2024-05-14T15:02:09.626 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-14T15:02:09.626 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} --verbose --name vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884 -- /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/orig.config.yaml 2024-05-14T15:02:09.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T15:02:09.781 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884 branch: wip-vshankar-testing-20240509.053109-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7705884' kernel: client: sha1: distro flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240509.053109-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: e2b755a64f0d72176b57299d02e880cf0620d780 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: e2b755a64f0d72176b57299d02e880cf0620d780 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240509.053109-debug sha1: e2b755a64f0d72176b57299d02e880cf0620d780 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6285 sha1: e2b755a64f0d72176b57299d02e880cf0620d780 sleep_before_teardown: 0 subset: 548/1024 suite: fs suite_branch: wip-vshankar-testing-20240509.053109-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e2b755a64f0d72176b57299d02e880cf0620d780 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFhUIHoGJpf4PEVsOBwGVRXX/J30I2pAaQ+g7bp70PFPF9v4/4PBrrNsOcLn4GyVaW2dqbCD1OfL5u+1YMEmnQ8= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRl4etdRzBgBfNEyMDJttwPsxBLFttqNA+87n/JJuaWyi7lGXTVeIohMJtspAxcrNT0H54EG8wqxKuCRxc5vFk= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLFa8bh5LHZxueTxQq8bSyPOLQsmxgbb6E99sDhMcIc//Rh3En1ZhQlKt+Co23GpBoffNr024DBJ7xff8Uq/aU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-14_07:04:04 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-14T15:02:09.781 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa; will attempt to use it 2024-05-14T15:02:09.782 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks 2024-05-14T15:02:09.782 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-14T15:02:09.784 INFO:teuthology.task.internal:Checking packages... 2024-05-14T15:02:09.806 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e2b755a64f0d72176b57299d02e880cf0620d780' 2024-05-14T15:02:09.806 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-14T15:02:09.806 INFO:teuthology.packaging:ref: None 2024-05-14T15:02:09.806 INFO:teuthology.packaging:tag: None 2024-05-14T15:02:09.806 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240509.053109-debug 2024-05-14T15:02:09.806 INFO:teuthology.packaging:sha1: e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:02:09.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240509.053109-debug 2024-05-14T15:02:10.024 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3588.ge2b755a6 2024-05-14T15:02:10.026 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-14T15:02:10.033 INFO:teuthology.task.internal:no buildpackages task found 2024-05-14T15:02:10.033 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-14T15:02:10.040 INFO:teuthology.task.internal:Saving configuration 2024-05-14T15:02:10.063 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-14T15:02:10.069 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T15:02:10.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 14:57:07.434175', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFhUIHoGJpf4PEVsOBwGVRXX/J30I2pAaQ+g7bp70PFPF9v4/4PBrrNsOcLn4GyVaW2dqbCD1OfL5u+1YMEmnQ8='} 2024-05-14T15:02:10.118 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 14:57:07.431505', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRl4etdRzBgBfNEyMDJttwPsxBLFttqNA+87n/JJuaWyi7lGXTVeIohMJtspAxcrNT0H54EG8wqxKuCRxc5vFk='} 2024-05-14T15:02:10.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 14:57:07.432933', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLFa8bh5LHZxueTxQq8bSyPOLQsmxgbb6E99sDhMcIc//Rh3En1ZhQlKt+Co23GpBoffNr024DBJ7xff8Uq/aU='} 2024-05-14T15:02:10.142 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-14T15:02:10.148 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-14T15:02:10.148 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-14T15:02:10.148 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-14T15:02:10.148 INFO:teuthology.run_tasks:Running task console_log... 2024-05-14T15:02:10.258 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7e2ded00d0>, signals=[15]) 2024-05-14T15:02:10.259 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-14T15:02:10.265 INFO:teuthology.task.internal:Opening connections... 2024-05-14T15:02:10.265 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:02:10.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:02:10.342 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-05-14T15:02:10.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:02:10.415 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-05-14T15:02:10.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:02:10.486 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-14T15:02:10.492 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-05-14T15:02:10.511 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-05-14T15:02:10.511 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-05-14T15:02:10.569 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-05-14T15:02:10.570 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T15:02:10.570 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-05-14T15:02:10.570 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T15:02:10.570 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T15:02:10.570 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T15:02:10.570 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-05-14T15:02:10.604 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-05-14T15:02:10.621 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-05-14T15:02:10.621 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-05-14T15:02:10.678 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T15:02:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T15:02:10.680 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T15:02:10.680 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-05-14T15:02:10.704 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-05-14T15:02:10.721 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-05-14T15:02:10.721 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-05-14T15:02:10.779 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-05-14T15:02:10.779 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-05-14T15:02:10.779 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-05-14T15:02:10.779 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T15:02:10.780 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T15:02:10.780 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-05-14T15:02:10.806 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-14T15:02:10.816 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-14T15:02:10.821 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-14T15:02:10.822 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-05-14T15:02:10.824 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-05-14T15:02:10.826 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-05-14T15:02:10.843 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-14T15:02:10.849 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-14T15:02:10.849 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-05-14T15:02:10.881 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-05-14T15:02:10.884 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-05-14T15:02:10.911 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-14T15:02:10.991 INFO:teuthology.run_tasks:Running task kernel... 2024-05-14T15:02:11.006 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-14T15:02:11.006 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-14T15:02:11.006 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-14T15:02:11.007 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T15:02:11.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:02:11.026 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-14T15:02:11.084 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-05-14T15:02:11.084 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-437.el9.x86_64 2024-05-14T15:02:11.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-05-14T15:02:13.884 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.1 MB 00:01 2024-05-14T15:02:15.585 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-05-14T15:02:20.127 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.2 MB 00:00 2024-05-14T15:02:21.644 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-05-14T15:02:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-14T15:02:28.190 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-05-14T15:02:30.194 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-14T15:02:30.194 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-14T15:02:30.274 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-14T15:02:30.280 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-14T15:02:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-05-14T15:02:30.280 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-05-14T15:02:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:02:30.282 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 78 M 2024-05-14T15:02:30.282 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 123 M 2024-05-14T15:02:30.282 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-14T15:02:30.748 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 2.8 MB/s | 550 kB 00:00 2024-05-14T15:02:31.848 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 12 MB/s | 15 MB 00:01 2024-05-14T15:02:32.773 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-core-5.14.0-446.el9.x86_6 14 MB/s | 28 MB 00:02 2024-05-14T15:02:33.365 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-14T15:02:33.365 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-14T15:02:33.365 INFO:teuthology.orchestra.run.smithi005.stdout:Total 25 MB/s | 78 MB 00:03 2024-05-14T15:02:34.077 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-14T15:02:34.476 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-14T15:02:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-14T15:02:35.447 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-14T15:02:35.447 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-14T15:02:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-14T15:02:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-14T15:02:38.707 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T15:02:41.530 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T15:02:41.667 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T15:02:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T15:02:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T15:02:51.883 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T15:03:23.018 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T15:03:23.058 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-14T15:03:23.360 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T15:03:23.360 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-14T15:03:23.360 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T15:03:23.360 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T15:03:24.163 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:03:24.164 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-14T15:03:24.534 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T15:03:24.606 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T15:03:24.607 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T15:03:24.607 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-14T15:03:24.607 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-14T15:03:24.607 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-14T15:03:24.607 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-14T15:03:24.607 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-05-14T15:03:24.607 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T15:03:24.659 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T15:03:24.659 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T15:03:24.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-05-14T15:03:24.690 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-05-14T15:03:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:03:24.692 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-05-14T15:03:24.693 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-14T15:03:24.693 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-446.el9.x86_64 2024-05-14T15:03:24.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-14T15:03:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-05-14T15:03:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-14T15:03:26.675 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-05-14T15:03:26.677 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-14T15:03:26.692 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.OtuAf0U7x5 2024-05-14T15:03:26.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OtuAf0U7x5 2024-05-14T15:03:26.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.OtuAf0U7x5 2024-05-14T15:03:26.904 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.OtuAf0U7x5 is 6KB 2024-05-14T15:03:26.954 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.OtuAf0U7x5 2024-05-14T15:03:26.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-05-14T15:03:27.035 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-14T15:03:27.035 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-14T15:03:27.035 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-14T15:03:27.035 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-14T15:03:27.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-14T15:03:27.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-05-14T15:03:57.176 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-14T15:03:57.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-14T15:03:57.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:04:57.181 DEBUG:teuthology.orchestra.remote:timed out 2024-05-14T15:05:06.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-14T15:05:06.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:05:06.508 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-14T15:05:06.868 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-14T15:05:06.868 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-14T15:05:06.868 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-14T15:05:06.883 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-446.el9.x86_64 2024-05-14T15:05:06.883 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-14T15:05:06.883 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T15:05:06.883 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-14T15:05:07.884 DEBUG:teuthology.parallel:result is None 2024-05-14T15:05:07.885 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-14T15:05:07.892 INFO:teuthology.task.internal:Creating test directory... 2024-05-14T15:05:07.893 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T15:05:07.895 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T15:05:07.897 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T15:05:07.915 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-14T15:05:07.922 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-14T15:05:07.933 INFO:teuthology.task.internal:Creating archive directory... 2024-05-14T15:05:07.933 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T15:05:07.954 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T15:05:07.957 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T15:05:08.070 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-14T15:05:08.077 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-14T15:05:08.077 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T15:05:08.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:05:08.094 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T15:05:08.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:05:08.110 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T15:05:08.112 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T15:05:08.136 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T15:05:08.166 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T15:05:08.182 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T15:05:08.184 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T15:05:08.202 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T15:05:08.207 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T15:05:08.219 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T15:05:08.221 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-14T15:05:08.229 INFO:teuthology.task.internal:Configuring sudo... 2024-05-14T15:05:08.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T15:05:08.232 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T15:05:08.246 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T15:05:08.288 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-14T15:05:08.297 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-14T15:05:08.298 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T15:05:08.301 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T15:05:08.313 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T15:05:08.344 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T15:05:08.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T15:05:08.448 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T15:05:08.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T15:05:08.579 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:05:08.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T15:05:08.647 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T15:05:08.690 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T15:05:08.737 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T15:05:08.820 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T15:05:08.861 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:05:08.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T15:05:08.926 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T15:05:08.970 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T15:05:09.021 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T15:05:09.105 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T15:05:09.146 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:05:09.146 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T15:05:09.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-05-14T15:05:09.218 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-05-14T15:05:09.221 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-05-14T15:05:09.252 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T15:05:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T15:05:09.301 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T15:05:09.669 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-14T15:05:09.680 INFO:teuthology.task.internal:Starting timer... 2024-05-14T15:05:09.680 INFO:teuthology.run_tasks:Running task pcp... 2024-05-14T15:05:09.734 INFO:teuthology.run_tasks:Running task selinux... 2024-05-14T15:05:09.887 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-14T15:05:09.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-05-14T15:05:09.996 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-05-14T15:05:10.001 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-05-14T15:05:10.070 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-05-14T15:05:10.072 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-05-14T15:05:10.140 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-05-14T15:05:10.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-14T15:05:10.142 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-05-14T15:05:10.172 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-05-14T15:05:10.172 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-05-14T15:05:10.201 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-05-14T15:05:10.202 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-05-14T15:05:10.221 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-05-14T15:05:10.221 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-05-14T15:05:10.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T15:05:10.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:05:10.268 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T15:05:10.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:05:10.295 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T15:05:10.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:05:10.323 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-14T15:05:10.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-05-14T15:05:10.659 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-05-14T15:05:10.682 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-05-14T15:05:10.715 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-14T15:05:10.727 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-14T15:05:10.754 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-14T15:05:10.915 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-14T15:05:10.968 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-14T15:05:10.969 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi144.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-14T15:09:53.747 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-05-14T15:09:53.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-14T15:09:53.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:09:53.830 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-14T15:09:53.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-14T15:09:53.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-14T15:09:53.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:09:53.986 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-05-14T15:09:54.073 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-05-14T15:09:54.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-14T15:09:54.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T15:09:54.204 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-05-14T15:09:54.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-05-14T15:09:54.282 INFO:teuthology.run_tasks:Running task clock... 2024-05-14T15:09:54.295 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-14T15:09:54.295 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T15:09:54.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T15:09:54.298 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T15:09:54.298 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T15:09:54.301 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T15:09:54.301 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T15:09:54.326 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T15:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T15:09:54.339 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T15:09:54.348 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T15:09:54.352 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T15:09:54.362 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-05-14T15:09:54.366 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T15:09:54.372 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-05-14T15:09:54.376 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-05-14T15:09:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T15:09:54.388 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-05-14T15:09:54.389 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-05-14T15:09:54.397 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T15:09:54.400 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-05-14T15:09:54.404 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T15:09:54.413 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T15:09:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T15:09:54.426 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T15:09:54.450 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-05-14T15:09:54.454 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T15:09:54.454 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-05-14T15:09:54.454 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.454 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.454 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.454 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-14T15:09:54.483 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-05-14T15:09:54.485 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T15:09:54.485 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-14T15:09:54.485 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.485 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.485 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.485 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.486 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T15:09:54.486 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-05-14T15:09:54.487 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.487 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.487 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.487 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:09:54.487 INFO:teuthology.run_tasks:Running task install... 2024-05-14T15:09:54.498 DEBUG:teuthology.task.install:project ceph 2024-05-14T15:09:54.498 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'}} 2024-05-14T15:09:54.499 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780'} 2024-05-14T15:09:54.499 INFO:teuthology.task.install:Using flavor: default 2024-05-14T15:09:54.504 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-14T15:09:54.505 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-14T15:09:54.505 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'tag': None, 'wait_for_package': False} 2024-05-14T15:09:54.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:09:54.508 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'tag': None, 'wait_for_package': False} 2024-05-14T15:09:54.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:09:54.509 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'tag': None, 'wait_for_package': False} 2024-05-14T15:09:54.509 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:09:54.656 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/ 2024-05-14T15:09:54.656 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3588.ge2b755a6 2024-05-14T15:09:54.770 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T15:09:54.770 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:09:54.770 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T15:09:54.795 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T15:09:54.795 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e2b755a64f0d72176b57299d02e880cf0620d780/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T15:09:54.803 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/ 2024-05-14T15:09:54.803 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3588.ge2b755a6 2024-05-14T15:09:54.806 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/ 2024-05-14T15:09:54.806 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3588.ge2b755a6 2024-05-14T15:09:54.866 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T15:09:54.917 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T15:09:54.918 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:09:54.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T15:09:54.921 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T15:09:54.922 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:09:54.922 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T15:09:54.939 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T15:09:54.945 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T15:09:54.946 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e2b755a64f0d72176b57299d02e880cf0620d780/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T15:09:54.947 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T15:09:54.947 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e2b755a64f0d72176b57299d02e880cf0620d780/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T15:09:55.002 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-05-14T15:09:55.003 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-05-14T15:09:55.012 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T15:09:55.014 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T15:09:55.085 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T15:09:55.092 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T15:09:55.147 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-05-14T15:09:55.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-05-14T15:09:55.157 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-05-14T15:09:55.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-05-14T15:09:55.229 INFO:teuthology.orchestra.run.smithi193.stdout:57 files removed 2024-05-14T15:09:55.256 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T15:09:55.377 INFO:teuthology.orchestra.run.smithi144.stdout:57 files removed 2024-05-14T15:09:55.404 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T15:09:55.411 INFO:teuthology.orchestra.run.smithi005.stdout:57 files removed 2024-05-14T15:09:55.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T15:09:55.924 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 207 kB/s | 84 kB 00:00 2024-05-14T15:09:56.067 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 209 kB/s | 84 kB 00:00 2024-05-14T15:09:56.143 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-05-14T15:09:56.367 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-14T15:09:56.468 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-05-14T15:09:56.576 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-14T15:09:56.692 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-05-14T15:09:56.910 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-14T15:09:56.944 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-14T15:09:57.494 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-14T15:09:57.759 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.1 MB 00:01 2024-05-14T15:09:58.777 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.1 MB 00:01 2024-05-14T15:10:00.703 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-14T15:10:01.303 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 19 MB 00:02 2024-05-14T15:10:01.393 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-05-14T15:10:05.095 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-05-14T15:10:06.594 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-14T15:10:06.796 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-05-14T15:10:06.904 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-05-14T15:10:08.128 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-14T15:10:08.248 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-14T15:10:08.405 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-14T15:10:09.656 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-14T15:10:11.563 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-05-14T15:10:13.398 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-14T15:10:15.090 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-14T15:10:15.649 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T15:10:15.649 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T15:10:15.660 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T15:10:15.661 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T15:10:15.662 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T15:10:15.663 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T15:10:15.664 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T15:10:15.667 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T15:10:15.667 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T15:10:15.728 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-05-14T15:10:15.752 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-14T15:10:15.752 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-05-14T15:10:15.752 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 6.6 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.4 M 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 863 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 147 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 1.5 M 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 142 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 3.5 M 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 7.4 M 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 49 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 10 M 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 43 M 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 273 k 2024-05-14T15:10:15.753 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 763 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 224 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 27 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 18 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 32 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 756 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 127 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T15:10:15.754 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 162 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 321 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 301 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 100 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 86 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.0 M 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 170 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T15:10:15.755 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.3 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.1 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 20 M 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 34 k 2024-05-14T15:10:15.756 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 2.3 M 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 274 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 4.9 M 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 18 M 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 16 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 25 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T15:10:15.757 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 157 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 497 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.2 M 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T15:10:15.758 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T15:10:15.759 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 45 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 135 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T15:10:15.760 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T15:10:15.761 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T15:10:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T15:10:15.763 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-05-14T15:10:15.764 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-05-14T15:10:15.765 INFO:teuthology.orchestra.run.smithi144.stdout:Install 148 Packages 2024-05-14T15:10:15.765 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 8 Packages 2024-05-14T15:10:15.765 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:10:15.765 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 219 M 2024-05-14T15:10:15.765 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-05-14T15:10:16.549 INFO:teuthology.orchestra.run.smithi144.stdout:(1/156): ceph-19.0.0-3588.ge2b755a6.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-05-14T15:10:16.746 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-05-14T15:10:16.866 INFO:teuthology.orchestra.run.smithi144.stdout:(2/156): ceph-fuse-19.0.0-3588.ge2b755a6.el9.x8 2.7 MB/s | 863 kB 00:00 2024-05-14T15:10:16.958 INFO:teuthology.orchestra.run.smithi144.stdout:(3/156): ceph-immutable-object-cache-19.0.0-358 1.6 MB/s | 147 kB 00:00 2024-05-14T15:10:17.200 INFO:teuthology.orchestra.run.smithi144.stdout:(4/156): ceph-base-19.0.0-3588.ge2b755a6.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-05-14T15:10:17.341 INFO:teuthology.orchestra.run.smithi144.stdout:(5/156): ceph-mds-19.0.0-3588.ge2b755a6.el9.x86 5.9 MB/s | 2.3 MB 00:00 2024-05-14T15:10:17.383 INFO:teuthology.orchestra.run.smithi144.stdout:(6/156): ceph-mgr-19.0.0-3588.ge2b755a6.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-05-14T15:10:17.583 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T15:10:17.583 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T15:10:17.595 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T15:10:17.595 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T15:10:17.596 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T15:10:17.597 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T15:10:17.599 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T15:10:17.601 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T15:10:17.602 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T15:10:17.667 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 6.6 k 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.4 M 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 863 k 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 147 k 2024-05-14T15:10:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 1.5 M 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 142 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 3.5 M 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 7.4 M 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 49 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 10 M 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 43 M 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 273 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 763 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 224 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 27 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 18 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T15:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 32 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 756 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 127 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 162 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 321 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 301 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 100 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 86 k 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.0 M 2024-05-14T15:10:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 170 k 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.3 M 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.1 M 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T15:10:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 20 M 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 34 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 2.3 M 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 274 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 4.9 M 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 18 M 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 16 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 25 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T15:10:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 157 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 497 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.2 M 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T15:10:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 45 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 135 k 2024-05-14T15:10:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T15:10:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T15:10:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T15:10:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T15:10:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-05-14T15:10:17.694 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-05-14T15:10:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:10:17.695 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 219 M 2024-05-14T15:10:17.695 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-14T15:10:18.134 INFO:teuthology.orchestra.run.smithi144.stdout:(7/156): ceph-mon-19.0.0-3588.ge2b755a6.el9.x86 6.2 MB/s | 4.9 MB 00:00 2024-05-14T15:10:18.316 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.0.0-3588.ge2b755a6.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-14T15:10:18.501 INFO:teuthology.orchestra.run.smithi144.stdout:(8/156): ceph-common-19.0.0-3588.ge2b755a6.el9. 9.0 MB/s | 20 MB 00:02 2024-05-14T15:10:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-fuse-19.0.0-3588.ge2b755a6.el9.x8 3.9 MB/s | 863 kB 00:00 2024-05-14T15:10:18.593 INFO:teuthology.orchestra.run.smithi144.stdout:(9/156): ceph-selinux-19.0.0-3588.ge2b755a6.el9 275 kB/s | 25 kB 00:00 2024-05-14T15:10:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-immutable-object-cache-19.0.0-358 2.2 MB/s | 147 kB 00:00 2024-05-14T15:10:18.988 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T15:10:18.989 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T15:10:19.000 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T15:10:19.001 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T15:10:19.001 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T15:10:19.002 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T15:10:19.004 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T15:10:19.006 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T15:10:19.007 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T15:10:19.067 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 6.6 k 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.4 M 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 863 k 2024-05-14T15:10:19.088 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 147 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 1.5 M 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 142 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 3.5 M 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 7.4 M 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 49 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 10 M 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 43 M 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 273 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 763 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 224 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 27 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 18 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T15:10:19.089 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 32 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 756 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 127 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 162 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 321 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 301 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 100 k 2024-05-14T15:10:19.090 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 86 k 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.0 M 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 170 k 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.3 M 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 3.1 M 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T15:10:19.091 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 20 M 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 34 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 2.3 M 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 274 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 4.9 M 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 18 M 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3588.ge2b755a6.el9 ceph-noarch 16 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 25 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T15:10:19.092 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 157 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 497 k 2024-05-14T15:10:19.093 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 5.2 M 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T15:10:19.094 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 45 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-3588.ge2b755a6.el9 ceph 135 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T15:10:19.095 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T15:10:19.096 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T15:10:19.097 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T15:10:19.098 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T15:10:19.099 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T15:10:19.099 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T15:10:19.099 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T15:10:19.099 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T15:10:19.099 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T15:10:19.100 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T15:10:19.101 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T15:10:19.101 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T15:10:19.101 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T15:10:19.101 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T15:10:19.101 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 8 Packages 2024-05-14T15:10:19.102 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:10:19.103 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 219 M 2024-05-14T15:10:19.103 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-05-14T15:10:19.117 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-mds-19.0.0-3588.ge2b755a6.el9.x86 4.4 MB/s | 2.3 MB 00:00 2024-05-14T15:10:19.293 INFO:teuthology.orchestra.run.smithi144.stdout:(10/156): ceph-osd-19.0.0-3588.ge2b755a6.el9.x8 9.3 MB/s | 18 MB 00:01 2024-05-14T15:10:19.309 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-mgr-19.0.0-3588.ge2b755a6.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-05-14T15:10:19.376 INFO:teuthology.orchestra.run.smithi144.stdout:(11/156): cephfs-mirror-19.0.0-3588.ge2b755a6.e 2.6 MB/s | 224 kB 00:00 2024-05-14T15:10:19.460 INFO:teuthology.orchestra.run.smithi144.stdout:(12/156): cephfs-shell-19.0.0-3588.ge2b755a6.el 324 kB/s | 27 kB 00:00 2024-05-14T15:10:19.552 INFO:teuthology.orchestra.run.smithi144.stdout:(13/156): libcephfs-devel-19.0.0-3588.ge2b755a6 345 kB/s | 32 kB 00:00 2024-05-14T15:10:19.697 INFO:teuthology.orchestra.run.smithi193.stdout:(1/156): ceph-19.0.0-3588.ge2b755a6.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-14T15:10:19.727 INFO:teuthology.orchestra.run.smithi144.stdout:(14/156): libcephfs2-19.0.0-3588.ge2b755a6.el9. 4.2 MB/s | 756 kB 00:00 2024-05-14T15:10:19.819 INFO:teuthology.orchestra.run.smithi144.stdout:(15/156): libcephsqlite-19.0.0-3588.ge2b755a6.e 1.7 MB/s | 157 kB 00:00 2024-05-14T15:10:19.961 INFO:teuthology.orchestra.run.smithi144.stdout:(16/156): ceph-radosgw-19.0.0-3588.ge2b755a6.el 5.5 MB/s | 10 MB 00:01 2024-05-14T15:10:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-mon-19.0.0-3588.ge2b755a6.el9.x86 7.2 MB/s | 4.9 MB 00:00 2024-05-14T15:10:19.986 INFO:teuthology.orchestra.run.smithi144.stdout:(17/156): librados-devel-19.0.0-3588.ge2b755a6. 759 kB/s | 127 kB 00:00 2024-05-14T15:10:20.120 INFO:teuthology.orchestra.run.smithi144.stdout:(18/156): libradosstriper1-19.0.0-3588.ge2b755a 3.1 MB/s | 497 kB 00:00 2024-05-14T15:10:20.220 INFO:teuthology.orchestra.run.smithi144.stdout:(19/156): python3-ceph-argparse-19.0.0-3588.ge2 451 kB/s | 45 kB 00:00 2024-05-14T15:10:20.337 INFO:teuthology.orchestra.run.smithi144.stdout:(20/156): python3-ceph-common-19.0.0-3588.ge2b7 1.1 MB/s | 135 kB 00:00 2024-05-14T15:10:20.455 INFO:teuthology.orchestra.run.smithi144.stdout:(21/156): python3-cephfs-19.0.0-3588.ge2b755a6. 1.4 MB/s | 162 kB 00:00 2024-05-14T15:10:20.680 INFO:teuthology.orchestra.run.smithi193.stdout:(2/156): ceph-fuse-19.0.0-3588.ge2b755a6.el9.x8 878 kB/s | 863 kB 00:00 2024-05-14T15:10:20.780 INFO:teuthology.orchestra.run.smithi144.stdout:(22/156): librgw2-19.0.0-3588.ge2b755a6.el9.x86 6.5 MB/s | 5.2 MB 00:00 2024-05-14T15:10:20.831 INFO:teuthology.orchestra.run.smithi193.stdout:(3/156): ceph-immutable-object-cache-19.0.0-358 980 kB/s | 147 kB 00:00 2024-05-14T15:10:20.838 INFO:teuthology.orchestra.run.smithi144.stdout:(23/156): python3-rados-19.0.0-3588.ge2b755a6.e 837 kB/s | 321 kB 00:00 2024-05-14T15:10:20.872 INFO:teuthology.orchestra.run.smithi144.stdout:(24/156): python3-rbd-19.0.0-3588.ge2b755a6.el9 3.2 MB/s | 301 kB 00:00 2024-05-14T15:10:20.922 INFO:teuthology.orchestra.run.smithi144.stdout:(25/156): python3-rgw-19.0.0-3588.ge2b755a6.el9 1.2 MB/s | 100 kB 00:00 2024-05-14T15:10:20.948 INFO:teuthology.orchestra.run.smithi144.stdout:(26/156): rbd-fuse-19.0.0-3588.ge2b755a6.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-14T15:10:21.048 INFO:teuthology.orchestra.run.smithi144.stdout:(27/156): rbd-nbd-19.0.0-3588.ge2b755a6.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-14T15:10:21.132 INFO:teuthology.orchestra.run.smithi144.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3588.g 411 kB/s | 34 kB 00:00 2024-05-14T15:10:21.232 INFO:teuthology.orchestra.run.smithi144.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3588.ge2b755a 1.4 MB/s | 142 kB 00:00 2024-05-14T15:10:21.865 INFO:teuthology.orchestra.run.smithi144.stdout:(30/156): rbd-mirror-19.0.0-3588.ge2b755a6.el9. 3.2 MB/s | 3.0 MB 00:00 2024-05-14T15:10:21.999 INFO:teuthology.orchestra.run.smithi144.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3588.ge2b75 4.6 MB/s | 3.5 MB 00:00 2024-05-14T15:10:22.116 INFO:teuthology.orchestra.run.smithi144.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3588.ge2 2.3 MB/s | 274 kB 00:00 2024-05-14T15:10:22.181 INFO:teuthology.orchestra.run.smithi193.stdout:(4/156): ceph-mds-19.0.0-3588.ge2b755a6.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-05-14T15:10:22.199 INFO:teuthology.orchestra.run.smithi144.stdout:(33/156): ceph-mgr-rook-19.0.0-3588.ge2b755a6.e 592 kB/s | 49 kB 00:00 2024-05-14T15:10:22.317 INFO:teuthology.orchestra.run.smithi144.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3588.ge 141 kB/s | 16 kB 00:00 2024-05-14T15:10:22.339 INFO:teuthology.orchestra.run.smithi193.stdout:(5/156): ceph-base-19.0.0-3588.ge2b755a6.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-05-14T15:10:22.417 INFO:teuthology.orchestra.run.smithi144.stdout:(35/156): ceph-volume-19.0.0-3588.ge2b755a6.el9 2.7 MB/s | 273 kB 00:00 2024-05-14T15:10:22.625 INFO:teuthology.orchestra.run.smithi144.stdout:(36/156): cephadm-19.0.0-3588.ge2b755a6.el9.noa 3.6 MB/s | 763 kB 00:00 2024-05-14T15:10:22.726 INFO:teuthology.orchestra.run.smithi144.stdout:(37/156): cephfs-top-19.0.0-3588.ge2b755a6.el9. 182 kB/s | 18 kB 00:00 2024-05-14T15:10:22.748 INFO:teuthology.orchestra.run.smithi193.stdout:(6/156): ceph-mgr-19.0.0-3588.ge2b755a6.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-05-14T15:10:22.817 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-base-19.0.0-3588.ge2b755a6.el9.x8 1.1 MB/s | 5.4 MB 00:04 2024-05-14T15:10:22.884 INFO:teuthology.orchestra.run.smithi144.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 508 kB/s | 80 kB 00:00 2024-05-14T15:10:23.043 INFO:teuthology.orchestra.run.smithi144.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-05-14T15:10:23.110 INFO:teuthology.orchestra.run.smithi144.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-14T15:10:23.177 INFO:teuthology.orchestra.run.smithi144.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-14T15:10:23.319 INFO:teuthology.orchestra.run.smithi144.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-05-14T15:10:23.378 INFO:teuthology.orchestra.run.smithi144.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-14T15:10:23.436 INFO:teuthology.orchestra.run.smithi144.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-14T15:10:23.486 INFO:teuthology.orchestra.run.smithi144.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-14T15:10:23.509 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-osd-19.0.0-3588.ge2b755a6.el9.x86 5.0 MB/s | 18 MB 00:03 2024-05-14T15:10:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-14T15:10:23.601 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.0.0-3588.ge2b755a6.el9 275 kB/s | 25 kB 00:00 2024-05-14T15:10:23.846 INFO:teuthology.orchestra.run.smithi144.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-05-14T15:10:23.873 INFO:teuthology.orchestra.run.smithi193.stdout:(7/156): ceph-mon-19.0.0-3588.ge2b755a6.el9.x86 3.2 MB/s | 4.9 MB 00:01 2024-05-14T15:10:24.071 INFO:teuthology.orchestra.run.smithi144.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-14T15:10:24.488 INFO:teuthology.orchestra.run.smithi144.stdout:(49/156): ceph-test-19.0.0-3588.ge2b755a6.el9.x 7.3 MB/s | 43 MB 00:05 2024-05-14T15:10:24.507 INFO:teuthology.orchestra.run.smithi193.stdout:(8/156): ceph-common-19.0.0-3588.ge2b755a6.el9. 3.8 MB/s | 20 MB 00:05 2024-05-14T15:10:24.513 INFO:teuthology.orchestra.run.smithi144.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 242 kB/s | 107 kB 00:00 2024-05-14T15:10:24.597 INFO:teuthology.orchestra.run.smithi144.stdout:(51/156): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-05-14T15:10:24.598 INFO:teuthology.orchestra.run.smithi193.stdout:(9/156): ceph-selinux-19.0.0-3588.ge2b755a6.el9 275 kB/s | 25 kB 00:00 2024-05-14T15:10:24.639 INFO:teuthology.orchestra.run.smithi144.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-14T15:10:24.656 INFO:teuthology.orchestra.run.smithi144.stdout:(53/156): elfutils-libelf-devel-0.190-2.el9.x86 128 kB/s | 21 kB 00:00 2024-05-14T15:10:24.698 INFO:teuthology.orchestra.run.smithi144.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-05-14T15:10:24.714 INFO:teuthology.orchestra.run.smithi144.stdout:(55/156): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-05-14T15:10:24.740 INFO:teuthology.orchestra.run.smithi144.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-05-14T15:10:24.782 INFO:teuthology.orchestra.run.smithi144.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-14T15:10:24.849 INFO:teuthology.orchestra.run.smithi144.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-14T15:10:24.974 INFO:teuthology.orchestra.run.smithi144.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-05-14T15:10:24.999 INFO:teuthology.orchestra.run.smithi144.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-14T15:10:25.025 INFO:teuthology.orchestra.run.smithi144.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-05-14T15:10:25.050 INFO:teuthology.orchestra.run.smithi144.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-05-14T15:10:25.109 INFO:teuthology.orchestra.run.smithi144.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-05-14T15:10:25.134 INFO:teuthology.orchestra.run.smithi144.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-14T15:10:25.167 INFO:teuthology.orchestra.run.smithi144.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-14T15:10:25.226 INFO:teuthology.orchestra.run.smithi144.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.1 MB/s | 3.0 MB 00:00 2024-05-14T15:10:25.349 INFO:teuthology.orchestra.run.smithi193.stdout:(10/156): ceph-osd-19.0.0-3588.ge2b755a6.el9.x8 6.8 MB/s | 18 MB 00:02 2024-05-14T15:10:25.440 INFO:teuthology.orchestra.run.smithi193.stdout:(11/156): cephfs-mirror-19.0.0-3588.ge2b755a6.e 2.4 MB/s | 224 kB 00:00 2024-05-14T15:10:25.502 INFO:teuthology.orchestra.run.smithi144.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-05-14T15:10:25.516 INFO:teuthology.orchestra.run.smithi193.stdout:(12/156): cephfs-shell-19.0.0-3588.ge2b755a6.el 360 kB/s | 27 kB 00:00 2024-05-14T15:10:25.591 INFO:teuthology.orchestra.run.smithi193.stdout:(13/156): libcephfs-devel-19.0.0-3588.ge2b755a6 422 kB/s | 32 kB 00:00 2024-05-14T15:10:25.677 INFO:teuthology.orchestra.run.smithi144.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-05-14T15:10:25.744 INFO:teuthology.orchestra.run.smithi144.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-14T15:10:25.766 INFO:teuthology.orchestra.run.smithi193.stdout:(14/156): libcephfs2-19.0.0-3588.ge2b755a6.el9. 4.2 MB/s | 756 kB 00:00 2024-05-14T15:10:25.859 INFO:teuthology.orchestra.run.smithi193.stdout:(15/156): libcephsqlite-19.0.0-3588.ge2b755a6.e 1.7 MB/s | 157 kB 00:00 2024-05-14T15:10:25.919 INFO:teuthology.orchestra.run.smithi144.stdout:(70/156): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-05-14T15:10:25.942 INFO:teuthology.orchestra.run.smithi193.stdout:(16/156): librados-devel-19.0.0-3588.ge2b755a6. 1.5 MB/s | 127 kB 00:00 2024-05-14T15:10:25.978 INFO:teuthology.orchestra.run.smithi144.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-05-14T15:10:26.036 INFO:teuthology.orchestra.run.smithi144.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-14T15:10:26.042 INFO:teuthology.orchestra.run.smithi193.stdout:(17/156): libradosstriper1-19.0.0-3588.ge2b755a 4.8 MB/s | 497 kB 00:00 2024-05-14T15:10:26.128 INFO:teuthology.orchestra.run.smithi144.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-05-14T15:10:26.221 INFO:teuthology.orchestra.run.smithi144.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-14T15:10:26.268 INFO:teuthology.orchestra.run.smithi193.stdout:(18/156): ceph-radosgw-19.0.0-3588.ge2b755a6.el 4.2 MB/s | 10 MB 00:02 2024-05-14T15:10:26.304 INFO:teuthology.orchestra.run.smithi144.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-14T15:10:26.355 INFO:teuthology.orchestra.run.smithi144.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-14T15:10:26.368 INFO:teuthology.orchestra.run.smithi193.stdout:(19/156): python3-ceph-argparse-19.0.0-3588.ge2 451 kB/s | 45 kB 00:00 2024-05-14T15:10:26.413 INFO:teuthology.orchestra.run.smithi144.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-05-14T15:10:26.472 INFO:teuthology.orchestra.run.smithi144.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-14T15:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout:(20/156): python3-ceph-common-19.0.0-3588.ge2b7 1.2 MB/s | 135 kB 00:00 2024-05-14T15:10:26.585 INFO:teuthology.orchestra.run.smithi193.stdout:(21/156): python3-cephfs-19.0.0-3588.ge2b755a6. 1.5 MB/s | 162 kB 00:00 2024-05-14T15:10:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-05-14T15:10:26.669 INFO:teuthology.orchestra.run.smithi193.stdout:(22/156): librgw2-19.0.0-3588.ge2b755a6.el9.x86 8.2 MB/s | 5.2 MB 00:00 2024-05-14T15:10:26.694 INFO:teuthology.orchestra.run.smithi193.stdout:(23/156): python3-rados-19.0.0-3588.ge2b755a6.e 2.9 MB/s | 321 kB 00:00 2024-05-14T15:10:26.706 INFO:teuthology.orchestra.run.smithi144.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 6.0 MB/s | 5.8 MB 00:00 2024-05-14T15:10:26.753 INFO:teuthology.orchestra.run.smithi193.stdout:(24/156): python3-rbd-19.0.0-3588.ge2b755a6.el9 3.5 MB/s | 301 kB 00:00 2024-05-14T15:10:26.786 INFO:teuthology.orchestra.run.smithi193.stdout:(25/156): python3-rgw-19.0.0-3588.ge2b755a6.el9 1.1 MB/s | 100 kB 00:00 2024-05-14T15:10:26.789 INFO:teuthology.orchestra.run.smithi144.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-14T15:10:26.828 INFO:teuthology.orchestra.run.smithi193.stdout:(26/156): rbd-fuse-19.0.0-3588.ge2b755a6.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-14T15:10:26.929 INFO:teuthology.orchestra.run.smithi193.stdout:(27/156): rbd-nbd-19.0.0-3588.ge2b755a6.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-14T15:10:26.973 INFO:teuthology.orchestra.run.smithi144.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.4 MB/s | 6.0 MB 00:00 2024-05-14T15:10:26.998 INFO:teuthology.orchestra.run.smithi144.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-05-14T15:10:27.021 INFO:teuthology.orchestra.run.smithi193.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3588.g 374 kB/s | 34 kB 00:00 2024-05-14T15:10:27.024 INFO:teuthology.orchestra.run.smithi144.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-14T15:10:27.040 INFO:teuthology.orchestra.run.smithi144.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-14T15:10:27.099 INFO:teuthology.orchestra.run.smithi144.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-14T15:10:27.121 INFO:teuthology.orchestra.run.smithi193.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3588.ge2b755a 1.4 MB/s | 142 kB 00:00 2024-05-14T15:10:27.124 INFO:teuthology.orchestra.run.smithi144.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-14T15:10:27.167 INFO:teuthology.orchestra.run.smithi144.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-14T15:10:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-14T15:10:27.209 INFO:teuthology.orchestra.run.smithi144.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-05-14T15:10:27.250 INFO:teuthology.orchestra.run.smithi144.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-14T15:10:27.301 INFO:teuthology.orchestra.run.smithi144.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-05-14T15:10:27.385 INFO:teuthology.orchestra.run.smithi144.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-14T15:10:27.560 INFO:teuthology.orchestra.run.smithi144.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 205 kB/s | 36 kB 00:00 2024-05-14T15:10:27.579 INFO:teuthology.orchestra.run.smithi193.stdout:(30/156): rbd-mirror-19.0.0-3588.ge2b755a6.el9. 3.8 MB/s | 3.0 MB 00:00 2024-05-14T15:10:27.638 INFO:teuthology.orchestra.run.smithi193.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3588.ge2b75 6.8 MB/s | 3.5 MB 00:00 2024-05-14T15:10:27.643 INFO:teuthology.orchestra.run.smithi144.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-05-14T15:10:27.738 INFO:teuthology.orchestra.run.smithi193.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3588.ge2 2.7 MB/s | 274 kB 00:00 2024-05-14T15:10:27.760 INFO:teuthology.orchestra.run.smithi144.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-14T15:10:27.811 INFO:teuthology.orchestra.run.smithi144.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-14T15:10:27.830 INFO:teuthology.orchestra.run.smithi193.stdout:(33/156): ceph-mgr-rook-19.0.0-3588.ge2b755a6.e 537 kB/s | 49 kB 00:00 2024-05-14T15:10:27.861 INFO:teuthology.orchestra.run.smithi144.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-14T15:10:27.920 INFO:teuthology.orchestra.run.smithi144.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-14T15:10:27.987 INFO:teuthology.orchestra.run.smithi144.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-05-14T15:10:28.054 INFO:teuthology.orchestra.run.smithi144.stdout:(101/156): python3-numpy-1.20.1-5.el9.x86_64.rp 3.5 MB/s | 5.1 MB 00:01 2024-05-14T15:10:28.204 INFO:teuthology.orchestra.run.smithi144.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-05-14T15:10:28.272 INFO:teuthology.orchestra.run.smithi193.stdout:(34/156): ceph-test-19.0.0-3588.ge2b755a6.el9.x 12 MB/s | 43 MB 00:03 2024-05-14T15:10:28.296 INFO:teuthology.orchestra.run.smithi144.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-05-14T15:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3588.ge 35 kB/s | 16 kB 00:00 2024-05-14T15:10:28.339 INFO:teuthology.orchestra.run.smithi193.stdout:(36/156): ceph-volume-19.0.0-3588.ge2b755a6.el9 4.0 MB/s | 273 kB 00:00 2024-05-14T15:10:28.376 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-radosgw-19.0.0-3588.ge2b755a6.el 1.8 MB/s | 10 MB 00:05 2024-05-14T15:10:28.406 INFO:teuthology.orchestra.run.smithi193.stdout:(37/156): cephadm-19.0.0-3588.ge2b755a6.el9.noa 6.9 MB/s | 763 kB 00:00 2024-05-14T15:10:28.423 INFO:teuthology.orchestra.run.smithi193.stdout:(38/156): cephfs-top-19.0.0-3588.ge2b755a6.el9. 218 kB/s | 18 kB 00:00 2024-05-14T15:10:28.476 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): cephfs-mirror-19.0.0-3588.ge2b755a6.e 2.2 MB/s | 224 kB 00:00 2024-05-14T15:10:28.535 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): cephfs-shell-19.0.0-3588.ge2b755a6.el 465 kB/s | 27 kB 00:00 2024-05-14T15:10:28.602 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): libcephfs-devel-19.0.0-3588.ge2b755a6 477 kB/s | 32 kB 00:00 2024-05-14T15:10:28.827 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs2-19.0.0-3588.ge2b755a6.el9. 3.3 MB/s | 756 kB 00:00 2024-05-14T15:10:28.849 INFO:teuthology.orchestra.run.smithi193.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-05-14T15:10:28.927 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephsqlite-19.0.0-3588.ge2b755a6.e 1.5 MB/s | 157 kB 00:00 2024-05-14T15:10:28.988 INFO:teuthology.orchestra.run.smithi144.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 700 kB/s | 548 kB 00:00 2024-05-14T15:10:28.994 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): librados-devel-19.0.0-3588.ge2b755a6. 1.9 MB/s | 127 kB 00:00 2024-05-14T15:10:29.013 INFO:teuthology.orchestra.run.smithi144.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 448 kB/s | 309 kB 00:00 2024-05-14T15:10:29.097 INFO:teuthology.orchestra.run.smithi144.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-05-14T15:10:29.132 INFO:teuthology.orchestra.run.smithi193.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-05-14T15:10:29.206 INFO:teuthology.orchestra.run.smithi144.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-05-14T15:10:29.211 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): ceph-common-19.0.0-3588.ge2b755a6.el9 1.8 MB/s | 20 MB 00:11 2024-05-14T15:10:29.286 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): libradosstriper1-19.0.0-3588.ge2b755a 1.7 MB/s | 497 kB 00:00 2024-05-14T15:10:29.339 INFO:teuthology.orchestra.run.smithi144.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-05-14T15:10:29.353 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): python3-ceph-argparse-19.0.0-3588.ge2 678 kB/s | 45 kB 00:00 2024-05-14T15:10:29.420 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-common-19.0.0-3588.ge2b7 2.0 MB/s | 135 kB 00:00 2024-05-14T15:10:29.465 INFO:teuthology.orchestra.run.smithi144.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 539 kB/s | 67 kB 00:00 2024-05-14T15:10:29.466 INFO:teuthology.orchestra.run.smithi193.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 770 kB/s | 803 kB 00:01 2024-05-14T15:10:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): python3-cephfs-19.0.0-3588.ge2b755a6. 1.6 MB/s | 162 kB 00:00 2024-05-14T15:10:29.531 INFO:teuthology.orchestra.run.smithi144.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-05-14T15:10:29.615 INFO:teuthology.orchestra.run.smithi144.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-14T15:10:29.658 INFO:teuthology.orchestra.run.smithi193.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 242 kB/s | 195 kB 00:00 2024-05-14T15:10:29.662 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-rados-19.0.0-3588.ge2b755a6.e 2.2 MB/s | 321 kB 00:00 2024-05-14T15:10:29.665 INFO:teuthology.orchestra.run.smithi144.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-14T15:10:29.771 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): python3-rbd-19.0.0-3588.ge2b755a6.el9 2.7 MB/s | 301 kB 00:00 2024-05-14T15:10:29.791 INFO:teuthology.orchestra.run.smithi144.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-05-14T15:10:29.816 INFO:teuthology.orchestra.run.smithi144.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-14T15:10:29.838 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rgw-19.0.0-3588.ge2b755a6.el9 1.5 MB/s | 100 kB 00:00 2024-05-14T15:10:29.900 INFO:teuthology.orchestra.run.smithi144.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-05-14T15:10:29.914 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): rbd-fuse-19.0.0-3588.ge2b755a6.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-14T15:10:29.925 INFO:teuthology.orchestra.run.smithi144.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-14T15:10:29.975 INFO:teuthology.orchestra.run.smithi193.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 301 kB/s | 253 kB 00:00 2024-05-14T15:10:30.025 INFO:teuthology.orchestra.run.smithi144.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-14T15:10:30.092 INFO:teuthology.orchestra.run.smithi144.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-05-14T15:10:30.176 INFO:teuthology.orchestra.run.smithi144.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-14T15:10:30.193 INFO:teuthology.orchestra.run.smithi144.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-05-14T15:10:30.197 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): librgw2-19.0.0-3588.ge2b755a6.el9.x86 5.2 MB/s | 5.2 MB 00:00 2024-05-14T15:10:30.264 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): rbd-nbd-19.0.0-3588.ge2b755a6.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-14T15:10:30.301 INFO:teuthology.orchestra.run.smithi144.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-14T15:10:30.314 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3588.g 688 kB/s | 34 kB 00:00 2024-05-14T15:10:30.365 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3588.ge2b755a 2.8 MB/s | 142 kB 00:00 2024-05-14T15:10:30.418 INFO:teuthology.orchestra.run.smithi144.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-14T15:10:30.442 INFO:teuthology.orchestra.run.smithi193.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 136 kB/s | 106 kB 00:00 2024-05-14T15:10:30.493 INFO:teuthology.orchestra.run.smithi144.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-14T15:10:30.519 INFO:teuthology.orchestra.run.smithi144.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-14T15:10:30.550 INFO:teuthology.orchestra.run.smithi193.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-14T15:10:30.594 INFO:teuthology.orchestra.run.smithi144.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-14T15:10:30.619 INFO:teuthology.orchestra.run.smithi144.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-14T15:10:30.695 INFO:teuthology.orchestra.run.smithi144.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-14T15:10:30.729 INFO:teuthology.orchestra.run.smithi144.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 180 kB/s | 20 kB 00:00 2024-05-14T15:10:30.792 INFO:teuthology.orchestra.run.smithi193.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 165 kB/s | 135 kB 00:00 2024-05-14T15:10:30.823 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): rbd-mirror-19.0.0-3588.ge2b755a6.el9. 3.3 MB/s | 3.0 MB 00:00 2024-05-14T15:10:30.899 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3588.ge2b75 6.6 MB/s | 3.5 MB 00:00 2024-05-14T15:10:30.928 INFO:teuthology.orchestra.run.smithi144.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-05-14T15:10:30.982 INFO:teuthology.orchestra.run.smithi144.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2024-05-14T15:10:30.983 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3588.ge2 3.2 MB/s | 274 kB 00:00 2024-05-14T15:10:31.024 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-rook-19.0.0-3588.ge2b755a6.e 1.2 MB/s | 49 kB 00:00 2024-05-14T15:10:31.046 INFO:teuthology.orchestra.run.smithi144.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 669 kB/s | 79 kB 00:00 2024-05-14T15:10:31.080 INFO:teuthology.orchestra.run.smithi144.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 574 kB/s | 58 kB 00:00 2024-05-14T15:10:31.172 INFO:teuthology.orchestra.run.smithi144.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-14T15:10:31.192 INFO:teuthology.orchestra.run.smithi193.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 166 kB/s | 125 kB 00:00 2024-05-14T15:10:31.197 INFO:teuthology.orchestra.run.smithi144.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-05-14T15:10:31.290 INFO:teuthology.orchestra.run.smithi144.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-05-14T15:10:31.314 INFO:teuthology.orchestra.run.smithi144.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-05-14T15:10:31.376 INFO:teuthology.orchestra.run.smithi193.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 260 kB/s | 215 kB 00:00 2024-05-14T15:10:31.415 INFO:teuthology.orchestra.run.smithi144.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-05-14T15:10:31.440 INFO:teuthology.orchestra.run.smithi144.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-14T15:10:31.450 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-test-19.0.0-3588.ge2b755a6.el9.x 5.5 MB/s | 43 MB 00:07 2024-05-14T15:10:31.475 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3588.ge 36 kB/s | 16 kB 00:00 2024-05-14T15:10:31.517 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): ceph-volume-19.0.0-3588.ge2b755a6.el9 4.0 MB/s | 273 kB 00:00 2024-05-14T15:10:31.540 INFO:teuthology.orchestra.run.smithi144.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-05-14T15:10:31.558 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): cephfs-top-19.0.0-3588.ge2b755a6.el9. 440 kB/s | 18 kB 00:00 2024-05-14T15:10:31.565 INFO:teuthology.orchestra.run.smithi144.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-05-14T15:10:31.609 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): cephadm-19.0.0-3588.ge2b755a6.el9.noa 5.6 MB/s | 763 kB 00:00 2024-05-14T15:10:31.725 INFO:teuthology.orchestra.run.smithi144.stdout:(141/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.6 MB/s | 16 MB 00:04 2024-05-14T15:10:31.749 INFO:teuthology.orchestra.run.smithi144.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 244 kB/s | 51 kB 00:00 2024-05-14T15:10:31.751 INFO:teuthology.orchestra.run.smithi193.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 740 kB/s | 689 kB 00:00 2024-05-14T15:10:31.775 INFO:teuthology.orchestra.run.smithi144.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-05-14T15:10:31.875 INFO:teuthology.orchestra.run.smithi144.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-14T15:10:31.900 INFO:teuthology.orchestra.run.smithi144.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-05-14T15:10:31.984 INFO:teuthology.orchestra.run.smithi144.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-14T15:10:32.060 INFO:teuthology.orchestra.run.smithi144.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-14T15:10:32.085 INFO:teuthology.orchestra.run.smithi193.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 151 kB/s | 107 kB 00:00 2024-05-14T15:10:32.168 INFO:teuthology.orchestra.run.smithi193.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 967 kB/s | 943 kB 00:00 2024-05-14T15:10:32.234 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 119 kB/s | 80 kB 00:00 2024-05-14T15:10:32.252 INFO:teuthology.orchestra.run.smithi144.stdout:(148/156): python3-websocket-client-1.2.3-2.el9 170 kB/s | 90 kB 00:00 2024-05-14T15:10:32.369 INFO:teuthology.orchestra.run.smithi193.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 35 kB/s | 21 kB 00:00 2024-05-14T15:10:32.527 INFO:teuthology.orchestra.run.smithi144.stdout:(149/156): librbd1-19.0.0-3588.ge2b755a6.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-05-14T15:10:32.559 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 845 kB/s | 803 kB 00:00 2024-05-14T15:10:32.635 INFO:teuthology.orchestra.run.smithi144.stdout:(150/156): NetworkManager-1.47.90-1.el9.x86_64. 6.0 MB/s | 2.3 MB 00:00 2024-05-14T15:10:32.677 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-05-14T15:10:32.686 INFO:teuthology.orchestra.run.smithi144.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 835 kB/s | 41 kB 00:00 2024-05-14T15:10:32.744 INFO:teuthology.orchestra.run.smithi144.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-14T15:10:32.769 INFO:teuthology.orchestra.run.smithi193.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-05-14T15:10:32.878 INFO:teuthology.orchestra.run.smithi144.stdout:(153/156): librados2-19.0.0-3588.ge2b755a6.el9. 3.7 MB/s | 3.3 MB 00:00 2024-05-14T15:10:32.912 INFO:teuthology.orchestra.run.smithi144.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-14T15:10:32.927 INFO:teuthology.orchestra.run.smithi193.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 373 kB/s | 313 kB 00:00 2024-05-14T15:10:32.954 INFO:teuthology.orchestra.run.smithi144.stdout:(155/156): NetworkManager-libnm-1.47.90-1.el9.x 4.2 MB/s | 1.8 MB 00:00 2024-05-14T15:10:33.076 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 232 kB/s | 195 kB 00:00 2024-05-14T15:10:33.079 INFO:teuthology.orchestra.run.smithi144.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-05-14T15:10:33.084 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-05-14T15:10:33.084 INFO:teuthology.orchestra.run.smithi144.stdout:Total 13 MB/s | 219 MB 00:17 2024-05-14T15:10:33.311 INFO:teuthology.orchestra.run.smithi193.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-05-14T15:10:33.452 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 284 kB/s | 253 kB 00:00 2024-05-14T15:10:33.478 INFO:teuthology.orchestra.run.smithi193.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-05-14T15:10:33.503 INFO:teuthology.orchestra.run.smithi193.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 19 kB/s | 11 kB 00:00 2024-05-14T15:10:33.738 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-14T15:10:33.794 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 149 kB/s | 106 kB 00:00 2024-05-14T15:10:34.162 INFO:teuthology.orchestra.run.smithi193.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-05-14T15:10:34.187 INFO:teuthology.orchestra.run.smithi193.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 183 kB/s | 160 kB 00:00 2024-05-14T15:10:34.210 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-05-14T15:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 165 kB/s | 135 kB 00:00 2024-05-14T15:10:34.362 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-05-14T15:10:34.362 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-05-14T15:10:34.462 INFO:teuthology.orchestra.run.smithi193.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 691 kB/s | 662 kB 00:00 2024-05-14T15:10:34.520 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 165 kB/s | 125 kB 00:00 2024-05-14T15:10:34.612 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 263 kB/s | 215 kB 00:00 2024-05-14T15:10:34.678 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-05-14T15:10:34.704 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-14T15:10:34.737 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-05-14T15:10:34.762 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-05-14T15:10:34.770 INFO:teuthology.orchestra.run.smithi193.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-05-14T15:10:34.804 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-05-14T15:10:34.830 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-14T15:10:34.872 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-05-14T15:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-14T15:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-05-14T15:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-14T15:10:35.031 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-05-14T15:10:35.062 INFO:teuthology.orchestra.run.smithi193.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 647 kB/s | 582 kB 00:00 2024-05-14T15:10:35.081 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-05-14T15:10:35.087 INFO:teuthology.orchestra.run.smithi193.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-05-14T15:10:35.115 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-14T15:10:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-05-14T15:10:35.174 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-05-14T15:10:35.224 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-14T15:10:35.241 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-05-14T15:10:35.267 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-14T15:10:35.400 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-14T15:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-05-14T15:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-14T15:10:35.613 INFO:teuthology.orchestra.run.smithi193.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-05-14T15:10:35.618 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-14T15:10:35.660 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-05-14T15:10:35.688 INFO:teuthology.orchestra.run.smithi193.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-14T15:10:35.735 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-14T15:10:35.777 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-05-14T15:10:35.855 INFO:teuthology.orchestra.run.smithi193.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 238 kB/s | 188 kB 00:00 2024-05-14T15:10:35.894 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-05-14T15:10:35.919 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-05-14T15:10:35.969 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-14T15:10:36.003 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-14T15:10:36.045 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-05-14T15:10:36.078 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-14T15:10:36.112 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-05-14T15:10:36.185 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-05-14T15:10:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-05-14T15:10:36.187 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-05-14T15:10:36.254 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-05-14T15:10:36.279 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-14T15:10:36.338 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 923 kB/s | 77 kB 00:00 2024-05-14T15:10:36.363 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-05-14T15:10:36.447 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-14T15:10:36.489 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-14T15:10:36.514 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-05-14T15:10:36.556 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-14T15:10:36.582 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 551 kB/s | 51 kB 00:00 2024-05-14T15:10:36.599 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 637 kB/s | 54 kB 00:00 2024-05-14T15:10:36.649 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-05-14T15:10:36.666 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-05-14T15:10:36.708 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-05-14T15:10:36.733 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-14T15:10:36.783 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-14T15:10:36.789 INFO:teuthology.orchestra.run.smithi193.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-05-14T15:10:36.834 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-14T15:10:36.859 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 766 kB/s | 58 kB 00:00 2024-05-14T15:10:36.926 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-05-14T15:10:36.939 INFO:teuthology.orchestra.run.smithi193.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-05-14T15:10:36.951 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 801 kB/s | 94 kB 00:00 2024-05-14T15:10:36.981 INFO:teuthology.orchestra.run.smithi193.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-14T15:10:37.010 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 538 kB/s | 45 kB 00:00 2024-05-14T15:10:37.035 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-05-14T15:10:37.202 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-14T15:10:37.436 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-14T15:10:37.461 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-14T15:10:37.486 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-05-14T15:10:37.520 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-05-14T15:10:37.545 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-05-14T15:10:37.603 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-14T15:10:37.662 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-14T15:10:37.687 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 290 kB/s | 49 kB 00:00 2024-05-14T15:10:37.706 INFO:teuthology.orchestra.run.smithi193.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 174 kB/s | 133 kB 00:00 2024-05-14T15:10:37.755 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-05-14T15:10:37.796 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-14T15:10:37.815 INFO:teuthology.orchestra.run.smithi193.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 889 kB/s | 911 kB 00:01 2024-05-14T15:10:37.822 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-05-14T15:10:37.847 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-05-14T15:10:37.872 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-14T15:10:37.906 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-05-14T15:10:37.931 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-14T15:10:37.957 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-14T15:10:37.982 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-14T15:10:38.007 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 466 kB/s | 35 kB 00:00 2024-05-14T15:10:38.035 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-14T15:10:38.066 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-14T15:10:38.091 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-05-14T15:10:38.117 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-05-14T15:10:38.167 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-14T15:10:38.193 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 229 kB/s | 23 kB 00:00 2024-05-14T15:10:38.215 INFO:teuthology.orchestra.run.smithi193.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-14T15:10:38.218 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-05-14T15:10:38.235 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-05-14T15:10:38.420 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 230 kB/s | 46 kB 00:00 2024-05-14T15:10:38.486 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 315 kB/s | 79 kB 00:00 2024-05-14T15:10:38.503 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 701 kB/s | 58 kB 00:00 2024-05-14T15:10:38.523 INFO:teuthology.orchestra.run.smithi193.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 123 kB/s | 87 kB 00:00 2024-05-14T15:10:38.536 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-05-14T15:10:38.553 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-05-14T15:10:38.578 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-14T15:10:38.603 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-14T15:10:38.629 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-14T15:10:38.654 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-05-14T15:10:38.688 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-14T15:10:38.691 INFO:teuthology.orchestra.run.smithi193.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-05-14T15:10:38.713 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-05-14T15:10:38.738 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-05-14T15:10:38.763 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-14T15:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-05-14T15:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 669 kB/s | 90 kB 00:00 2024-05-14T15:10:38.906 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-14T15:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-05-14T15:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-14T15:10:39.023 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-14T15:10:39.366 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): librados2-19.0.0-3588.ge2b755a6.el9. 7.7 MB/s | 3.3 MB 00:00 2024-05-14T15:10:39.391 INFO:teuthology.orchestra.run.smithi193.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 282 kB/s | 244 kB 00:00 2024-05-14T15:10:39.474 INFO:teuthology.orchestra.run.smithi193.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-05-14T15:10:39.583 INFO:teuthology.orchestra.run.smithi193.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 278 kB/s | 248 kB 00:00 2024-05-14T15:10:39.666 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): librbd1-19.0.0-3588.ge2b755a6.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-05-14T15:10:40.008 INFO:teuthology.orchestra.run.smithi193.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-05-14T15:10:40.033 INFO:teuthology.orchestra.run.smithi193.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-05-14T15:10:40.099 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-1.47.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-05-14T15:10:40.325 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): NetworkManager-team-1.47.90-1.el9.x8 63 kB/s | 41 kB 00:00 2024-05-14T15:10:40.350 INFO:teuthology.orchestra.run.smithi193.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 225 kB/s | 172 kB 00:00 2024-05-14T15:10:40.425 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): NetworkManager-libnm-1.47.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-05-14T15:10:40.634 INFO:teuthology.orchestra.run.smithi193.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-05-14T15:10:40.933 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): NetworkManager-tui-1.47.90-1.el9.x86 297 kB/s | 248 kB 00:00 2024-05-14T15:10:41.209 INFO:teuthology.orchestra.run.smithi193.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 292 kB/s | 250 kB 00:00 2024-05-14T15:10:41.276 INFO:teuthology.orchestra.run.smithi193.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-14T15:10:41.367 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-14T15:10:41.493 INFO:teuthology.orchestra.run.smithi193.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 259 kB/s | 222 kB 00:00 2024-05-14T15:10:41.517 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-14T15:10:41.521 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-14T15:10:41.521 INFO:teuthology.orchestra.run.smithi005.stdout:Total 9.2 MB/s | 219 MB 00:23 2024-05-14T15:10:41.893 INFO:teuthology.orchestra.run.smithi193.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 113 kB/s | 77 kB 00:00 2024-05-14T15:10:41.919 INFO:teuthology.orchestra.run.smithi193.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 66 kB/s | 42 kB 00:00 2024-05-14T15:10:42.207 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-05-14T15:10:42.308 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T15:10:42.346 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T15:10:42.352 INFO:teuthology.orchestra.run.smithi193.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 314 kB/s | 269 kB 00:00 2024-05-14T15:10:42.399 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T15:10:42.438 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T15:10:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T15:10:42.510 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T15:10:42.644 INFO:teuthology.orchestra.run.smithi193.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 212 kB/s | 159 kB 00:00 2024-05-14T15:10:42.753 INFO:teuthology.orchestra.run.smithi193.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 334 kB/s | 279 kB 00:00 2024-05-14T15:10:42.784 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-14T15:10:42.904 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T15:10:42.953 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:10:42.953 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-14T15:10:42.953 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-14T15:10:42.994 INFO:teuthology.orchestra.run.smithi193.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-05-14T15:10:43.087 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:10:43.120 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:10:43.194 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:10:43.225 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T15:10:43.295 INFO:teuthology.orchestra.run.smithi193.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-05-14T15:10:43.370 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T15:10:43.422 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T15:10:43.450 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:10:43.553 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:10:43.576 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T15:10:43.638 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T15:10:43.653 INFO:teuthology.orchestra.run.smithi193.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 66 kB/s | 42 kB 00:00 2024-05-14T15:10:43.686 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T15:10:43.705 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T15:10:43.876 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T15:10:43.928 INFO:teuthology.orchestra.run.smithi193.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 70 kB/s | 44 kB 00:00 2024-05-14T15:10:44.001 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T15:10:44.150 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T15:10:44.193 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T15:10:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:10:44.337 INFO:teuthology.orchestra.run.smithi193.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-14T15:10:44.355 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:10:44.545 INFO:teuthology.orchestra.run.smithi193.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 247 kB/s | 221 kB 00:00 2024-05-14T15:10:44.556 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:10:44.612 INFO:teuthology.orchestra.run.smithi193.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-05-14T15:10:44.659 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T15:10:44.684 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T15:10:44.719 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T15:10:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T15:10:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T15:10:44.878 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T15:10:44.911 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T15:10:44.939 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T15:10:44.962 INFO:teuthology.orchestra.run.smithi193.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-05-14T15:10:45.007 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T15:10:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T15:10:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T15:10:45.127 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T15:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-14T15:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-14T15:10:45.194 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T15:10:45.230 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 32/164 2024-05-14T15:10:45.273 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 33/164 2024-05-14T15:10:45.296 INFO:teuthology.orchestra.run.smithi193.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 85 kB/s | 58 kB 00:00 2024-05-14T15:10:45.302 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 34/164 2024-05-14T15:10:45.370 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T15:10:45.405 INFO:teuthology.orchestra.run.smithi193.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 355 kB/s | 305 kB 00:00 2024-05-14T15:10:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T15:10:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T15:10:45.622 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T15:10:45.668 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T15:10:45.680 INFO:teuthology.orchestra.run.smithi193.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 131 kB/s | 94 kB 00:00 2024-05-14T15:10:45.947 INFO:teuthology.orchestra.run.smithi193.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-05-14T15:10:45.972 INFO:teuthology.orchestra.run.smithi193.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-05-14T15:10:46.040 INFO:teuthology.orchestra.run.smithi193.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-14T15:10:46.165 INFO:teuthology.orchestra.run.smithi193.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 MB/s | 548 kB 00:00 2024-05-14T15:10:46.181 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T15:10:46.206 INFO:teuthology.orchestra.run.smithi193.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 469 kB/s | 19 kB 00:00 2024-05-14T15:10:46.232 INFO:teuthology.orchestra.run.smithi193.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-05-14T15:10:46.257 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T15:10:46.286 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T15:10:46.328 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T15:10:46.366 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 44/164 2024-05-14T15:10:46.408 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T15:10:46.424 INFO:teuthology.orchestra.run.smithi193.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 135 kB/s | 26 kB 00:00 2024-05-14T15:10:46.474 INFO:teuthology.orchestra.run.smithi193.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-05-14T15:10:46.524 INFO:teuthology.orchestra.run.smithi193.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-14T15:10:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T15:10:46.575 INFO:teuthology.orchestra.run.smithi193.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-05-14T15:10:46.619 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T15:10:46.683 INFO:teuthology.orchestra.run.smithi193.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-05-14T15:10:46.708 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T15:10:46.708 INFO:teuthology.orchestra.run.smithi193.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-14T15:10:46.750 INFO:teuthology.orchestra.run.smithi193.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-14T15:10:46.774 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T15:10:46.792 INFO:teuthology.orchestra.run.smithi193.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 772 kB/s | 32 kB 00:00 2024-05-14T15:10:46.839 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T15:10:46.876 INFO:teuthology.orchestra.run.smithi193.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-05-14T15:10:46.893 INFO:teuthology.orchestra.run.smithi193.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-14T15:10:46.896 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T15:10:46.919 INFO:teuthology.orchestra.run.smithi193.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-05-14T15:10:46.932 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T15:10:46.944 INFO:teuthology.orchestra.run.smithi193.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-14T15:10:46.969 INFO:teuthology.orchestra.run.smithi193.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-05-14T15:10:46.979 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T15:10:47.002 INFO:teuthology.orchestra.run.smithi193.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-14T15:10:47.015 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T15:10:47.028 INFO:teuthology.orchestra.run.smithi193.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-05-14T15:10:47.053 INFO:teuthology.orchestra.run.smithi193.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-14T15:10:47.058 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T15:10:47.078 INFO:teuthology.orchestra.run.smithi193.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-14T15:10:47.104 INFO:teuthology.orchestra.run.smithi193.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-14T15:10:47.110 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T15:10:47.129 INFO:teuthology.orchestra.run.smithi193.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-05-14T15:10:47.146 INFO:teuthology.orchestra.run.smithi193.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-05-14T15:10:47.171 INFO:teuthology.orchestra.run.smithi193.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 463 kB/s | 19 kB 00:00 2024-05-14T15:10:47.188 INFO:teuthology.orchestra.run.smithi193.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-05-14T15:10:47.238 INFO:teuthology.orchestra.run.smithi193.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-05-14T15:10:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T15:10:47.263 INFO:teuthology.orchestra.run.smithi193.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-14T15:10:47.289 INFO:teuthology.orchestra.run.smithi193.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-14T15:10:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T15:10:47.314 INFO:teuthology.orchestra.run.smithi193.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-14T15:10:47.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T15:10:47.339 INFO:teuthology.orchestra.run.smithi193.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-14T15:10:47.360 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T15:10:47.365 INFO:teuthology.orchestra.run.smithi193.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-14T15:10:47.382 INFO:teuthology.orchestra.run.smithi193.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-14T15:10:47.415 INFO:teuthology.orchestra.run.smithi193.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-05-14T15:10:47.441 INFO:teuthology.orchestra.run.smithi193.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-14T15:10:47.454 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T15:10:47.466 INFO:teuthology.orchestra.run.smithi193.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-05-14T15:10:47.486 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T15:10:47.491 INFO:teuthology.orchestra.run.smithi193.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-14T15:10:47.516 INFO:teuthology.orchestra.run.smithi193.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-05-14T15:10:47.526 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T15:10:47.541 INFO:teuthology.orchestra.run.smithi193.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 787 kB/s | 59 kB 00:00 2024-05-14T15:10:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T15:10:47.567 INFO:teuthology.orchestra.run.smithi193.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-14T15:10:47.576 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T15:10:47.592 INFO:teuthology.orchestra.run.smithi193.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-14T15:10:47.642 INFO:teuthology.orchestra.run.smithi193.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-14T15:10:47.649 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T15:10:47.668 INFO:teuthology.orchestra.run.smithi193.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-14T15:10:47.693 INFO:teuthology.orchestra.run.smithi193.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-14T15:10:47.718 INFO:teuthology.orchestra.run.smithi193.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-14T15:10:47.785 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T15:10:47.867 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T15:10:47.877 INFO:teuthology.orchestra.run.smithi193.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-05-14T15:10:47.895 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T15:10:47.929 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T15:10:47.968 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T15:10:47.999 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T15:10:48.029 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T15:10:48.064 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T15:10:48.098 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T15:10:48.139 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T15:10:48.144 INFO:teuthology.orchestra.run.smithi193.stdout:(149/156): librados2-19.0.0-3588.ge2b755a6.el9. 7.4 MB/s | 3.3 MB 00:00 2024-05-14T15:10:48.173 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T15:10:48.258 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T15:10:48.303 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T15:10:48.319 INFO:teuthology.orchestra.run.smithi193.stdout:(150/156): librbd1-19.0.0-3588.ge2b755a6.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-05-14T15:10:48.335 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T15:10:48.381 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T15:10:48.480 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T15:10:48.528 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T15:10:48.944 INFO:teuthology.orchestra.run.smithi193.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 66 kB/s | 41 kB 00:00 2024-05-14T15:10:49.078 INFO:teuthology.orchestra.run.smithi193.stdout:(152/156): NetworkManager-1.47.90-1.el9.x86_64. 1.9 MB/s | 2.3 MB 00:01 2024-05-14T15:10:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T15:10:49.271 INFO:teuthology.orchestra.run.smithi193.stdout:(153/156): NetworkManager-libnm-1.47.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-05-14T15:10:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T15:10:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T15:10:49.431 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T15:10:49.483 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T15:10:49.570 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T15:10:49.611 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T15:10:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T15:10:49.820 INFO:teuthology.orchestra.run.smithi193.stdout:(154/156): NetworkManager-tui-1.47.90-1.el9.x86 283 kB/s | 248 kB 00:00 2024-05-14T15:10:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T15:10:49.903 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T15:10:50.187 INFO:teuthology.orchestra.run.smithi193.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-14T15:10:50.437 INFO:teuthology.orchestra.run.smithi193.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-05-14T15:10:50.441 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-05-14T15:10:50.441 INFO:teuthology.orchestra.run.smithi193.stdout:Total 7.0 MB/s | 219 MB 00:31 2024-05-14T15:10:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T15:10:50.648 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T15:10:51.562 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-14T15:10:51.627 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-05-14T15:10:51.675 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T15:10:51.722 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T15:10:51.777 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-05-14T15:10:51.777 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-05-14T15:10:51.783 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T15:10:51.826 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T15:10:51.851 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T15:10:51.854 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T15:10:51.907 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T15:10:51.938 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T15:10:51.965 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T15:10:51.999 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T15:10:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T15:10:52.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T15:10:52.362 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T15:10:52.385 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:10:52.435 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T15:10:52.466 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 102/164 2024-05-14T15:10:52.503 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T15:10:52.513 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:10:52.544 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:10:52.637 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:10:52.665 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T15:10:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T15:10:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T15:10:52.932 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:10:52.989 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T15:10:53.015 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T15:10:53.039 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:10:53.066 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T15:10:53.086 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T15:10:53.108 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 106/164 2024-05-14T15:10:53.121 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T15:10:53.134 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T15:10:53.158 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T15:10:53.354 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T15:10:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T15:10:53.565 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-05-14T15:10:53.565 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-05-14T15:10:53.673 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T15:10:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T15:10:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:10:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:10:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:10:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T15:10:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T15:10:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T15:10:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T15:10:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T15:10:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T15:10:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T15:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T15:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T15:10:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T15:10:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T15:10:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T15:10:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T15:10:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 32/164 2024-05-14T15:10:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 33/164 2024-05-14T15:10:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 34/164 2024-05-14T15:10:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T15:10:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T15:10:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T15:10:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T15:10:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T15:10:55.320 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:10:55.354 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:10:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T15:10:55.911 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:10:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T15:10:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T15:10:55.949 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T15:10:55.991 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T15:10:56.029 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 44/164 2024-05-14T15:10:56.043 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T15:10:56.043 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T15:10:56.043 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T15:10:56.043 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:10:56.076 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T15:10:56.076 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T15:10:56.250 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T15:10:56.315 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T15:10:56.410 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T15:10:56.472 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T15:10:56.547 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T15:10:56.582 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T15:10:56.619 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T15:10:56.666 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T15:10:56.718 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T15:10:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T15:10:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T15:10:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T15:10:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T15:10:57.014 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T15:10:57.056 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T15:10:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T15:10:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T15:10:57.231 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T15:10:57.263 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T15:10:57.287 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T15:10:57.371 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T15:10:57.501 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T15:10:57.536 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T15:10:57.584 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T15:10:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T15:10:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T15:10:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T15:10:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T15:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T15:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T15:10:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T15:10:57.885 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T15:10:57.974 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T15:10:58.029 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T15:10:58.071 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T15:10:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T15:10:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T15:10:58.290 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T15:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T15:10:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T15:10:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T15:10:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T15:10:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T15:10:59.385 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T15:10:59.414 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T15:10:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T15:10:59.619 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T15:10:59.652 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-05-14T15:10:59.701 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T15:10:59.758 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T15:10:59.796 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T15:10:59.857 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T15:10:59.891 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T15:10:59.917 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T15:10:59.959 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T15:11:00.295 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T15:11:00.398 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T15:11:00.410 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:11:00.518 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:11:00.529 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T15:11:00.544 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:11:00.626 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:11:00.656 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T15:11:00.803 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T15:11:00.854 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T15:11:00.882 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:11:00.986 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:11:01.008 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T15:11:01.070 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 14/164 2024-05-14T15:11:01.092 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T15:11:01.116 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T15:11:01.259 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T15:11:01.384 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T15:11:01.548 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T15:11:01.608 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T15:11:01.701 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:11:01.784 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:11:01.854 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T15:11:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T15:11:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T15:11:01.974 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T15:11:01.994 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T15:11:02.080 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T15:11:02.113 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T15:11:02.148 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T15:11:02.220 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T15:11:02.255 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T15:11:02.307 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T15:11:02.332 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T15:11:02.357 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T15:11:02.373 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T15:11:02.385 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T15:11:02.452 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T15:11:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T15:11:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 102/164 2024-05-14T15:11:02.506 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T15:11:02.537 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T15:11:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T15:11:02.581 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T15:11:02.656 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T15:11:02.691 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 32/164 2024-05-14T15:11:02.734 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 33/164 2024-05-14T15:11:02.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 34/164 2024-05-14T15:11:02.833 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T15:11:02.878 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T15:11:02.952 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T15:11:03.055 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T15:11:03.083 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T15:11:03.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T15:11:03.138 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T15:11:03.189 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T15:11:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 106/164 2024-05-14T15:11:03.646 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T15:11:03.734 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T15:11:03.780 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T15:11:03.823 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T15:11:03.861 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 44/164 2024-05-14T15:11:03.902 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T15:11:04.054 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T15:11:04.122 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T15:11:04.211 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T15:11:04.269 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T15:11:04.350 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T15:11:04.382 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T15:11:04.421 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T15:11:04.465 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T15:11:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T15:11:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T15:11:04.585 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T15:11:04.720 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T15:11:04.768 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T15:11:04.799 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T15:11:04.839 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T15:11:04.933 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T15:11:04.965 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T15:11:05.006 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T15:11:05.033 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T15:11:05.055 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T15:11:05.130 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T15:11:05.256 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T15:11:05.287 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T15:11:05.315 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T15:11:05.349 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T15:11:05.388 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T15:11:05.420 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T15:11:05.451 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T15:11:05.485 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T15:11:05.518 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T15:11:05.559 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T15:11:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:11:05.594 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T15:11:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:11:05.679 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T15:11:05.724 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T15:11:05.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T15:11:05.811 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T15:11:05.907 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T15:11:05.956 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T15:11:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:11:06.428 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T15:11:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T15:11:06.521 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T15:11:06.521 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T15:11:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:06.545 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T15:11:06.673 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T15:11:06.720 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T15:11:06.763 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T15:11:06.859 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T15:11:06.912 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T15:11:06.990 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T15:11:07.040 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T15:11:07.216 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T15:11:07.246 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T15:11:07.323 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-05-14T15:11:07.881 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:07.961 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T15:11:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T15:11:08.064 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T15:11:08.064 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:08.064 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T15:11:08.064 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T15:11:09.307 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T15:11:09.400 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T15:11:09.427 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T15:11:09.461 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T15:11:09.635 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T15:11:09.660 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T15:11:09.660 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:09.660 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T15:11:09.661 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T15:11:09.661 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T15:11:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:09.781 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T15:11:09.828 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T15:11:09.886 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:09.922 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 101/164 2024-05-14T15:11:09.954 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 102/164 2024-05-14T15:11:09.990 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T15:11:10.370 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T15:11:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T15:11:10.392 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:10.392 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T15:11:10.392 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T15:11:10.392 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T15:11:10.392 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:10.481 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T15:11:10.508 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T15:11:10.510 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T15:11:10.565 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T15:11:10.581 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 105/164 2024-05-14T15:11:10.603 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 106/164 2024-05-14T15:11:10.621 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T15:11:10.806 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T15:11:10.856 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T15:11:10.932 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T15:11:11.021 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T15:11:11.151 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T15:11:11.198 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T15:11:11.238 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T15:11:11.266 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T15:11:11.643 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T15:11:11.673 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T15:11:11.705 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 126/164 2024-05-14T15:11:11.766 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 127/164 2024-05-14T15:11:11.796 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T15:11:12.814 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:11:12.833 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T15:11:12.850 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:11:12.881 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T15:11:13.410 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 107/164 2024-05-14T15:11:13.453 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T15:11:13.551 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 108/164 2024-05-14T15:11:13.552 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T15:11:13.552 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T15:11:13.552 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:13.580 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T15:11:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T15:11:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T15:11:13.985 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T15:11:14.142 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 131/164 2024-05-14T15:11:14.175 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T15:11:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T15:11:14.199 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:14.199 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T15:11:14.199 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T15:11:14.199 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T15:11:14.199 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:14.243 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T15:11:14.254 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T15:11:14.352 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 134/164 2024-05-14T15:11:14.399 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 135/164 2024-05-14T15:11:15.338 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T15:11:15.370 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T15:11:15.393 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T15:11:15.393 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:15.393 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T15:11:15.393 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T15:11:15.393 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T15:11:15.393 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:15.437 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T15:11:15.459 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T15:11:15.459 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:15.459 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T15:11:15.459 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:15.505 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T15:11:15.527 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T15:11:15.527 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:15.527 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T15:11:15.527 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T15:11:15.527 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T15:11:15.527 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:15.830 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T15:11:15.853 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T15:11:15.853 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:15.853 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T15:11:15.853 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T15:11:15.853 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T15:11:15.853 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:18.655 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T15:11:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-05-14T15:11:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-05-14T15:11:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-05-14T15:11:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-05-14T15:11:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-05-14T15:11:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-05-14T15:11:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-05-14T15:11:18.656 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:18.845 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T15:11:18.863 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T15:11:18.863 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:18.863 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T15:11:18.863 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:20.134 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 141/164 2024-05-14T15:11:20.184 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 142/164 2024-05-14T15:11:20.224 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 143/164 2024-05-14T15:11:20.256 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T15:11:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T15:11:20.464 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 146/164 2024-05-14T15:11:20.488 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T15:11:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T15:11:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T15:11:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T15:11:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T15:11:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T15:11:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:20.552 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 148/164 2024-05-14T15:11:20.590 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T15:11:20.628 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T15:11:20.755 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T15:11:20.780 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T15:11:20.781 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:20.781 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T15:11:20.781 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T15:11:20.781 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T15:11:20.781 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:21.192 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:21.234 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:21.253 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:21.273 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T15:11:21.297 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T15:11:21.298 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:21.298 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T15:11:21.298 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T15:11:21.298 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T15:11:21.298 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T15:11:21.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T15:11:21.525 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T15:11:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T15:11:21.774 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T15:11:21.845 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T15:11:21.934 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T15:11:22.076 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T15:11:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 152/164 2024-05-14T15:11:22.124 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T15:11:22.156 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T15:11:22.183 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T15:11:22.197 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T15:11:22.247 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T15:11:22.327 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T15:11:22.376 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T15:11:22.402 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T15:11:22.402 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T15:11:22.469 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T15:11:22.502 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T15:11:22.502 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T15:11:22.595 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T15:11:22.613 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T15:11:22.620 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T15:11:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:22.640 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T15:11:22.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 126/164 2024-05-14T15:11:22.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 127/164 2024-05-14T15:11:22.763 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T15:11:22.814 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:22.814 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T15:11:22.890 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T15:11:22.952 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:23.855 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T15:11:23.890 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T15:11:24.905 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T15:11:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T15:11:25.058 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T15:11:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 131/164 2024-05-14T15:11:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T15:11:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T15:11:25.294 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:25.294 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T15:11:25.294 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T15:11:25.294 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T15:11:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T15:11:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T15:11:25.457 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 134/164 2024-05-14T15:11:25.498 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 109/164 2024-05-14T15:11:25.498 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-05-14T15:11:25.499 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:25.515 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 135/164 2024-05-14T15:11:25.669 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T15:11:25.684 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 110/164 2024-05-14T15:11:25.684 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:25.684 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T15:11:25.684 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:26.506 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T15:11:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T15:11:26.562 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T15:11:26.562 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:26.562 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T15:11:26.562 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T15:11:26.562 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T15:11:26.562 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:26.604 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T15:11:26.629 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T15:11:26.629 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:26.629 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T15:11:26.629 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:26.680 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T15:11:26.705 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T15:11:26.705 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:26.705 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T15:11:26.705 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T15:11:26.705 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T15:11:26.705 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:27.031 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T15:11:27.056 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T15:11:27.056 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:27.056 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T15:11:27.056 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T15:11:27.056 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T15:11:27.056 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:27.262 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T15:11:27.288 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 111/164 2024-05-14T15:11:27.289 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:27.289 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T15:11:27.289 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T15:11:27.289 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T15:11:27.289 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:27.539 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T15:11:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 112/164 2024-05-14T15:11:27.562 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:27.562 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T15:11:27.562 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T15:11:27.562 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T15:11:27.562 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:28.032 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T15:11:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 113/164 2024-05-14T15:11:28.055 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:28.055 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T15:11:28.055 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T15:11:28.055 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T15:11:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:28.154 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T15:11:28.203 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T15:11:28.258 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T15:11:28.442 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T15:11:28.491 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T15:11:28.568 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 1/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 3/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 4/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 5/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 6/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 7/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 11/164 2024-05-14T15:11:28.588 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 12/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 14/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 15/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 16/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 17/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 18/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 19/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 20/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 21/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 22/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 23/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 24/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 25/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 26/164 2024-05-14T15:11:28.589 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 27/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 28/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 29/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 30/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 31/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 32/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 33/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 34/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 35/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 36/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 37/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 38/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 39/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T15:11:28.590 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T15:11:28.591 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T15:11:28.592 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T15:11:28.593 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T15:11:28.594 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T15:11:28.595 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T15:11:28.596 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T15:11:28.597 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 149/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 151/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T15:11:28.598 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T15:11:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T15:11:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T15:11:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T15:11:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T15:11:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T15:11:28.599 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T15:11:28.657 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T15:11:28.797 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T15:11:28.833 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T15:11:28.871 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T15:11:28.898 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T15:11:29.385 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T15:11:29.414 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 125/164 2024-05-14T15:11:29.446 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 126/164 2024-05-14T15:11:29.508 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 127/164 2024-05-14T15:11:29.538 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T15:11:30.568 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 128/164 2024-05-14T15:11:30.598 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T15:11:30.950 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:30.951 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.952 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.953 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T15:11:30.954 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.955 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T15:11:30.956 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.957 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T15:11:30.958 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T15:11:30.959 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T15:11:30.961 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T15:11:30.962 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T15:11:30.963 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T15:11:30.963 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-05-14T15:11:30.963 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-05-14T15:11:31.381 DEBUG:teuthology.parallel:result is None 2024-05-14T15:11:31.568 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 129/164 2024-05-14T15:11:31.590 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T15:11:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 141/164 2024-05-14T15:11:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 142/164 2024-05-14T15:11:31.700 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 130/164 2024-05-14T15:11:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 143/164 2024-05-14T15:11:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T15:11:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T15:11:31.865 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 131/164 2024-05-14T15:11:31.901 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T15:11:31.924 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 132/164 2024-05-14T15:11:31.924 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:31.924 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T15:11:31.925 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T15:11:31.925 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T15:11:31.925 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:31.968 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T15:11:31.980 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 133/164 2024-05-14T15:11:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 146/164 2024-05-14T15:11:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T15:11:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 148/164 2024-05-14T15:11:32.079 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 134/164 2024-05-14T15:11:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T15:11:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T15:11:32.133 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 135/164 2024-05-14T15:11:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:33.068 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T15:11:33.104 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T15:11:33.127 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 137/164 2024-05-14T15:11:33.127 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:33.127 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T15:11:33.127 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T15:11:33.127 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T15:11:33.127 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:33.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T15:11:33.194 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 138/164 2024-05-14T15:11:33.194 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:33.194 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T15:11:33.194 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:33.239 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T15:11:33.261 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 139/164 2024-05-14T15:11:33.261 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:33.261 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T15:11:33.261 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T15:11:33.262 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T15:11:33.262 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:33.573 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T15:11:33.596 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 140/164 2024-05-14T15:11:33.596 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T15:11:33.596 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T15:11:33.596 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T15:11:33.596 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T15:11:33.596 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 152/164 2024-05-14T15:11:33.826 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T15:11:33.872 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T15:11:33.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T15:11:33.996 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T15:11:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T15:11:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T15:11:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T15:11:34.139 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T15:11:34.139 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T15:11:34.207 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T15:11:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T15:11:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:34.443 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:34.479 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T15:11:34.544 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T15:11:34.620 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:37.869 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 141/164 2024-05-14T15:11:37.910 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 142/164 2024-05-14T15:11:37.951 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 143/164 2024-05-14T15:11:37.983 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T15:11:38.010 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T15:11:38.195 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 146/164 2024-05-14T15:11:38.248 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T15:11:38.278 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 148/164 2024-05-14T15:11:38.308 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T15:11:38.346 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T15:11:38.945 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:38.986 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:39.006 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T15:11:39.836 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 152/164 2024-05-14T15:11:39.909 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T15:11:39.950 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T15:11:40.032 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T15:11:40.062 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T15:11:40.102 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T15:11:40.103 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T15:11:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 1/164 2024-05-14T15:11:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 3/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 4/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 5/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 6/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 7/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 11/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 12/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 14/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 15/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 16/164 2024-05-14T15:11:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 17/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 18/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 19/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 20/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 21/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 22/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 23/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 24/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 25/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 26/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 27/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 28/164 2024-05-14T15:11:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 29/164 2024-05-14T15:11:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 30/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 31/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 32/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 33/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 34/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 35/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 36/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 37/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 38/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 39/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T15:11:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T15:11:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T15:11:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T15:11:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T15:11:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T15:11:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T15:11:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 149/164 2024-05-14T15:11:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 151/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T15:11:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T15:11:40.189 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T15:11:40.214 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T15:11:40.214 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T15:11:40.299 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T15:11:40.323 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T15:11:40.329 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:40.329 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:40.509 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T15:11:40.509 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T15:11:40.593 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T15:11:40.657 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:42.741 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T15:11:42.742 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:42.744 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T15:11:42.745 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T15:11:42.746 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T15:11:42.747 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T15:11:42.748 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T15:11:42.749 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T15:11:42.750 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T15:11:42.751 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T15:11:42.752 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T15:11:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T15:11:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T15:11:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T15:11:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:11:42.753 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-14T15:11:43.172 DEBUG:teuthology.parallel:result is None 2024-05-14T15:11:46.274 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 1/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 3/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 4/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3588.ge2b75 5/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 6/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 7/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 8/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 9/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 10/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 11/164 2024-05-14T15:11:46.275 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 12/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 13/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 14/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_ 15/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 16/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 17/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_6 18/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86 19/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 20/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el 21/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9. 22/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_6 23/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 24/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 25/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 26/164 2024-05-14T15:11:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 27/164 2024-05-14T15:11:46.277 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 28/164 2024-05-14T15:11:46.277 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 29/164 2024-05-14T15:11:46.277 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6. 30/164 2024-05-14T15:11:46.277 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noa 31/164 2024-05-14T15:11:46.277 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.n 32/164 2024-05-14T15:11:46.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b 33/164 2024-05-14T15:11:46.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el 34/164 2024-05-14T15:11:46.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 35/164 2024-05-14T15:11:46.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.e 36/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 37/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 38/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 39/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T15:11:46.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T15:11:46.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T15:11:46.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T15:11:46.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T15:11:46.294 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T15:11:46.295 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T15:11:46.296 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 149/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 151/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T15:11:46.297 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T15:11:46.298 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T15:11:46.298 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T15:11:46.298 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T15:11:46.298 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T15:11:46.298 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T15:11:46.298 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.956 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.957 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.958 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-3588.ge2b755a6.el9.noarch 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.959 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T15:11:48.960 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T15:11:48.961 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T15:11:48.962 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T15:11:48.963 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T15:11:48.964 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T15:11:48.965 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-3588.ge2b755a6.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T15:11:48.966 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T15:11:48.967 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T15:11:48.967 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T15:11:48.967 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T15:11:48.967 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:11:48.967 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-05-14T15:11:49.376 DEBUG:teuthology.parallel:result is None 2024-05-14T15:11:49.377 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:11:49.520 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T15:11:49.548 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-3588.ge2b755a6.el9 2024-05-14T15:11:49.548 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588.ge2b755a6.el9 2024-05-14T15:11:49.548 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588.ge2b755a6 is installed. 2024-05-14T15:11:49.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:11:49.685 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T15:11:49.710 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-3588.ge2b755a6.el9 2024-05-14T15:11:49.710 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588.ge2b755a6.el9 2024-05-14T15:11:49.710 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588.ge2b755a6 is installed. 2024-05-14T15:11:49.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:11:49.937 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T15:11:49.960 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-3588.ge2b755a6.el9 2024-05-14T15:11:49.960 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3588.ge2b755a6.el9 2024-05-14T15:11:49.960 INFO:teuthology.task.install:The correct ceph version 19.0.0-3588.ge2b755a6 is installed. 2024-05-14T15:11:49.962 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-14T15:11:49.962 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:11:49.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T15:11:49.996 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:11:49.996 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T15:11:50.027 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:11:50.027 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T15:11:50.054 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-14T15:11:50.054 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:11:50.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T15:11:50.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T15:11:50.153 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:11:50.153 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T15:11:50.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T15:11:50.247 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:11:50.247 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T15:11:50.272 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T15:11:50.336 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-14T15:11:50.336 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:11:50.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T15:11:50.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T15:11:50.435 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:11:50.435 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T15:11:50.459 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T15:11:50.523 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:11:50.523 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T15:11:50.548 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T15:11:50.613 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-14T15:11:50.613 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:11:50.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T15:11:50.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T15:11:50.706 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:11:50.706 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T15:11:50.737 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T15:11:50.800 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:11:50.801 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T15:11:50.827 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T15:11:50.892 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-14T15:11:50.987 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-14T15:11:50.987 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:11:50.988 INFO:tasks.cephadm:Cluster fsid is 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:11:50.988 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-14T15:11:50.988 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.144', 'mon.c': '172.21.15.193'} 2024-05-14T15:11:50.988 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-05-14T15:11:50.988 INFO:tasks.cephadm:First mgr is x 2024-05-14T15:11:50.989 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-14T15:11:50.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-05-14T15:11:51.016 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-05-14T15:11:51.040 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-05-14T15:11:51.065 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-14T15:11:51.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:11:51.289 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79237/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3588-ge2b755a6', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3588.ge2b755a6'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 07:03:10.506903', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240509.053109-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'e2b755a64f0d72176b57299d02e880cf0620d780', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79236/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3588-ge2b755a6', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3588.ge2b755a6'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 06:43:33.316648', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240509.053109-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-14T15:11:51.433 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240509.053109-debug, sha1 e2b755a64f0d72176b57299d02e880cf0620d780 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:11:51.435 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm 2024-05-14T15:11:51.435 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:52.205 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 14 15:11 /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:52.206 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:52.902 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 14 15:11 /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:52.902 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240509.053109-debug/e2b755a64f0d72176b57299d02e880cf0620d780/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:53.407 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 782846 May 14 15:11 /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:53.408 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:53.428 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:53.447 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T15:11:53.483 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 on all hosts... 2024-05-14T15:11:53.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 pull 2024-05-14T15:11:53.486 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 pull 2024-05-14T15:11:53.490 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 pull 2024-05-14T15:11:54.280 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T15:11:54.346 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T15:11:54.403 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345", 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430" 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-05-14T15:12:24.015 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345", 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430" 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-05-14T15:12:27.791 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-14T15:12:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:12:29.236 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T15:12:29.237 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345", 2024-05-14T15:12:29.237 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-05-14T15:12:29.237 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430" 2024-05-14T15:12:29.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:12:29.237 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:12:29.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-05-14T15:12:29.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-05-14T15:12:29.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-05-14T15:12:29.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-05-14T15:12:29.400 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-05-14T15:12:29.426 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-05-14T15:12:29.452 INFO:tasks.cephadm:Writing seed config... 2024-05-14T15:12:29.453 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-14T15:12:29.453 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-14T15:12:29.453 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-14T15:12:29.453 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-14T15:12:29.454 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-14T15:12:29.455 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-14T15:12:29.456 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-14T15:12:29.457 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:12:29.457 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-14T15:12:29.474 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4a7148fe-1204-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-14T15:12:29.474 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a.service 2024-05-14T15:12:29.517 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x.service 2024-05-14T15:12:29.560 INFO:tasks.cephadm:Bootstrapping... 2024-05-14T15:12:29.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 -v bootstrap --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:12:29.758 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-14T15:12:29.758 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780', '-v', 'bootstrap', '--fsid', '4a7148fe-1204-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-05-14T15:12:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-14T15:12:29.759 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-05-14T15:12:29.785 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.0.2 2024-05-14T15:12:29.785 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-05-14T15:12:29.785 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-05-14T15:12:29.793 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T15:12:29.793 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T15:12:29.800 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T15:12:29.800 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-14T15:12:29.806 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-05-14T15:12:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-05-14T15:12:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-05-14T15:12:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-05-14T15:12:29.838 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.0.2 2024-05-14T15:12:29.838 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-14T15:12:29.838 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-05-14T15:12:29.838 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-05-14T15:12:29.845 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T15:12:29.845 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T15:12:29.851 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T15:12:29.852 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-14T15:12:29.858 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-05-14T15:12:29.865 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-05-14T15:12:29.865 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-05-14T15:12:29.865 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-05-14T15:12:29.865 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:29.865 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 139925689278416 on /run/cephadm/4a7148fe-1204-11ef-bc98-c7b262605968.lock 2024-05-14T15:12:29.865 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 139925689278416 acquired on /run/cephadm/4a7148fe-1204-11ef-bc98-c7b262605968.lock 2024-05-14T15:12:29.866 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-05-14T15:12:29.866 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-05-14T15:12:29.866 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-05-14T15:12:29.870 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-05-14T15:12:29.870 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-05-14T15:12:29.872 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-14T15:12:29.872 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-14T15:12:29.873 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-14T15:12:29.875 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-14T15:12:29.875 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-14T15:12:29.875 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T15:12:29.875 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-14T15:12:29.875 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-14T15:12:29.875 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T15:12:29.876 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-05-14T15:12:29.876 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-05-14T15:12:29.876 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-14T15:12:29.876 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-14T15:12:29.877 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780... 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:d2511a189d36a438cd7259d98515a840197dc63bdcef8f78efcfd8f43785c919 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 2024-05-14T15:12:30.307 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-14T15:12:31.269 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev) 2024-05-14T15:12:31.269 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev) 2024-05-14T15:12:31.269 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-05-14T15:12:32.159 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-05-14T15:12:32.159 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-05-14T15:12:32.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBgf0NmZF0cHRAAYAwKWf6mq1k1eMM0HROXzg== 2024-05-14T15:12:33.838 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBhf0NmH2gRFBAAHWwjPIXFi/0zcf9LT++bBA== 2024-05-14T15:12:34.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBif0NmTgJjChAAKmx4nIWc6X0W5QFNx4WKrw== 2024-05-14T15:12:34.779 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T15:12:35.642 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:12:35.643 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-05-14T15:12:36.566 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-05-14T15:12:36.784 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-14T15:12:36.927 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-14T15:12:37.099 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4a7148fe-1204-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-4a7148fe-1204-11ef-bc98-c7b262605968.target. 2024-05-14T15:12:37.099 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4a7148fe-1204-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-4a7148fe-1204-11ef-bc98-c7b262605968.target. 2024-05-14T15:12:37.360 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a 2024-05-14T15:12:37.360 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-14T15:12:37.543 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4a7148fe-1204-11ef-bc98-c7b262605968.target.wants/ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-4a7148fe-1204-11ef-bc98-c7b262605968@.service. 2024-05-14T15:12:38.082 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T15:12:38.083 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-14T15:12:38.090 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T15:12:38.090 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-14T15:12:38.090 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-14T15:12:38.090 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T15:12:38.090 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-05-14T15:12:38.091 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-05-14T15:12:38.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:38 smithi005 systemd[1]: Started Ceph mon.a for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:12:39.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T15:12:39.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: monmap epoch 1 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: last_changed 2024-05-14T15:12:35.137977+0000 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: min_mon_release 19 (squid) 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: election_strategy: 1 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: fsmap 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: mgrmap e1: no daemons active 2024-05-14T15:12:39.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:39 smithi005 ceph-mon[31191]: from='client.? 172.21.15.5:0/4149384913' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-14T15:12:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-05-14T15:12:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-14T15:12:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-05-14T15:12:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.543665s) 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.503+0000 7fe2ce128700 1 Processor -- start 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.503+0000 7fe2ce128700 1 -- start start 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2ce128700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c810c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:39.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2ce128700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c810ce90 con 0x7fe2c807cc20 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c810c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c810c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59390/0 (socket says 172.21.15.5:59390) 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2c77fe700 1 -- 172.21.15.5:0/3504664552 learned_addr learned my addr 172.21.15.5:0/3504664552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2c77fe700 1 -- 172.21.15.5:0/3504664552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c810d010 con 0x7fe2c807cc20 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.504+0000 7fe2c77fe700 1 --2- 172.21.15.5:0/3504664552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c810c8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009e80 tx=0x7fe2b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd993246332a175 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2c6ffd700 1 -- 172.21.15.5:0/3504664552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b8017070 con 0x7fe2c807cc20 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2c6ffd700 1 -- 172.21.15.5:0/3504664552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b8003a20 con 0x7fe2c807cc20 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2c6ffd700 1 -- 172.21.15.5:0/3504664552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b8003d20 con 0x7fe2c807cc20 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2ce128700 1 -- 172.21.15.5:0/3504664552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 msgr2=0x7fe2c810c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2ce128700 1 --2- 172.21.15.5:0/3504664552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c810c8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009e80 tx=0x7fe2b800b330 comp rx=0 tx=0).stop 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2ce128700 1 -- 172.21.15.5:0/3504664552 shutdown_connections 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2ce128700 1 --2- 172.21.15.5:0/3504664552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c810c8c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.505+0000 7fe2ce128700 1 -- 172.21.15.5:0/3504664552 >> 172.21.15.5:0/3504664552 conn(0x7fe2c8104290 msgr2=0x7fe2c81066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.506+0000 7fe2ce128700 1 -- 172.21.15.5:0/3504664552 shutdown_connections 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.506+0000 7fe2ce128700 1 -- 172.21.15.5:0/3504664552 wait complete. 2024-05-14T15:12:39.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.506+0000 7fe2ce128700 1 Processor -- start 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.506+0000 7fe2ce128700 1 -- start start 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2ce128700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2ce128700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c810d4e0 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59400/0 (socket says 172.21.15.5:59400) 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c77fe700 1 -- 172.21.15.5:0/4149384913 learned_addr learned my addr 172.21.15.5:0/4149384913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c77fe700 1 -- 172.21.15.5:0/4149384913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c819dd50 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c77fe700 1 --2- 172.21.15.5:0/4149384913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c819d810 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8004000 tx=0x7fe2b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b8022020 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b8018040 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c819dfe0 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.507+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b8017070 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.508+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c81a0bf0 con 0x7fe2c807cc20 2024-05-14T15:12:39.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.508+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2b8025020 con 0x7fe2c807cc20 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.508+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2b800f050 con 0x7fe2c807cc20 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.508+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c8064e20 con 0x7fe2c807cc20 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.511+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe2b802c030 con 0x7fe2c807cc20 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.584+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe2c802e860 con 0x7fe2c807cc20 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2c57fa700 1 -- 172.21.15.5:0/4149384913 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe2b801ede0 con 0x7fe2c807cc20 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 msgr2=0x7fe2c819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2ce128700 1 --2- 172.21.15.5:0/4149384913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c819d810 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8004000 tx=0x7fe2b8004ad0 comp rx=0 tx=0).stop 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 shutdown_connections 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2ce128700 1 --2- 172.21.15.5:0/4149384913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c807cc20 0x7fe2c819d810 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 >> 172.21.15.5:0/4149384913 conn(0x7fe2c8104290 msgr2=0x7fe2c8106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.585+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 shutdown_connections 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:38.586+0000 7fe2ce128700 1 -- 172.21.15.5:0/4149384913 wait complete. 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-05-14T15:12:39.195 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T15:12:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.627+0000 7fe48c1da700 1 Processor -- start 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.627+0000 7fe48c1da700 1 -- start start 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.627+0000 7fe48c1da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48410cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.627+0000 7fe48c1da700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe48407d660 con 0x7fe48410a850 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.627+0000 7fe489f76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48410cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe489f76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48410cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59402/0 (socket says 172.21.15.5:59402) 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe489f76700 1 -- 172.21.15.5:0/446458133 learned_addr learned my addr 172.21.15.5:0/446458133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe489f76700 1 -- 172.21.15.5:0/446458133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48410d1b0 con 0x7fe48410a850 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe489f76700 1 --2- 172.21.15.5:0/446458133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48410cc70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098a0 tx=0x7fe478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41589ac5bad11737 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe488f74700 1 -- 172.21.15.5:0/446458133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478004160 con 0x7fe48410a850 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe488f74700 1 -- 172.21.15.5:0/446458133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe478004300 con 0x7fe48410a850 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.628+0000 7fe488f74700 1 -- 172.21.15.5:0/446458133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4780036a0 con 0x7fe48410a850 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 -- 172.21.15.5:0/446458133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 msgr2=0x7fe48410cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 --2- 172.21.15.5:0/446458133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48410cc70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098a0 tx=0x7fe478009bb0 comp rx=0 tx=0).stop 2024-05-14T15:12:40.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 -- 172.21.15.5:0/446458133 shutdown_connections 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 --2- 172.21.15.5:0/446458133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48410cc70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 -- 172.21.15.5:0/446458133 >> 172.21.15.5:0/446458133 conn(0x7fe484104330 msgr2=0x7fe484106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 -- 172.21.15.5:0/446458133 shutdown_connections 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.629+0000 7fe48c1da700 1 -- 172.21.15.5:0/446458133 wait complete. 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe48c1da700 1 Processor -- start 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe48c1da700 1 -- start start 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe48c1da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe48c1da700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410d560 con 0x7fe48410a850 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe489f76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe489f76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59418/0 (socket says 172.21.15.5:59418) 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe489f76700 1 -- 172.21.15.5:0/354854558 learned_addr learned my addr 172.21.15.5:0/354854558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.630+0000 7fe489f76700 1 -- 172.21.15.5:0/354854558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48419dcc0 con 0x7fe48410a850 2024-05-14T15:12:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe489f76700 1 --2- 172.21.15.5:0/354854558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48419d780 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478003db0 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe478011630 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478011950 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48419df50 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4841a0b60 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe478011af0 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.631+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe47801c450 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.632+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe484064e20 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.634+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe478032020 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.704+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe48402e8f0 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.709+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+463 (secure 0 0 0) 0x7fe478004260 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.710+0000 7fe476ffd700 1 -- 172.21.15.5:0/354854558 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe47801c6a0 con 0x7fe48410a850 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 msgr2=0x7fe48419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 --2- 172.21.15.5:0/354854558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48419d780 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478004750 comp rx=0 tx=0).stop 2024-05-14T15:12:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 shutdown_connections 2024-05-14T15:12:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 --2- 172.21.15.5:0/354854558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe48410a850 0x7fe48419d780 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 >> 172.21.15.5:0/354854558 conn(0x7fe484104330 msgr2=0x7fe4841060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 shutdown_connections 2024-05-14T15:12:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:39.711+0000 7fe48c1da700 1 -- 172.21.15.5:0/354854558 wait complete. 2024-05-14T15:12:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-05-14T15:12:40.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:40 smithi005 ceph-mon[31191]: from='client.? 172.21.15.5:0/354854558' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T15:12:40.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:40 smithi005 ceph-mon[31191]: from='client.? 172.21.15.5:0/354854558' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.694+0000 7efe4d692700 1 Processor -- start 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.694+0000 7efe4d692700 1 -- start start 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.694+0000 7efe4d692700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.694+0000 7efe4d692700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe4810bcb0 con 0x7efe4810b2d0 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.695+0000 7efe46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.695+0000 7efe46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59432/0 (socket says 172.21.15.5:59432) 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.695+0000 7efe46ffd700 1 -- 172.21.15.5:0/149171273 learned_addr learned my addr 172.21.15.5:0/149171273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:41.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.695+0000 7efe46ffd700 1 -- 172.21.15.5:0/149171273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4810be30 con 0x7efe4810b2d0 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.695+0000 7efe46ffd700 1 --2- 172.21.15.5:0/149171273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810b6e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe300098a0 tx=0x7efe30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60f80456f215c1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe45ffb700 1 -- 172.21.15.5:0/149171273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe30004160 con 0x7efe4810b2d0 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe45ffb700 1 -- 172.21.15.5:0/149171273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe30004300 con 0x7efe4810b2d0 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe45ffb700 1 -- 172.21.15.5:0/149171273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe30011af0 con 0x7efe4810b2d0 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe4d692700 1 -- 172.21.15.5:0/149171273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 msgr2=0x7efe4810b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe4d692700 1 --2- 172.21.15.5:0/149171273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810b6e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe300098a0 tx=0x7efe30009bb0 comp rx=0 tx=0).stop 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe4d692700 1 -- 172.21.15.5:0/149171273 shutdown_connections 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.696+0000 7efe4d692700 1 --2- 172.21.15.5:0/149171273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810b6e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.697+0000 7efe4d692700 1 -- 172.21.15.5:0/149171273 >> 172.21.15.5:0/149171273 conn(0x7efe48103ad0 msgr2=0x7efe48105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.697+0000 7efe4d692700 1 -- 172.21.15.5:0/149171273 shutdown_connections 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.697+0000 7efe4d692700 1 -- 172.21.15.5:0/149171273 wait complete. 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.697+0000 7efe4d692700 1 Processor -- start 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.697+0000 7efe4d692700 1 -- start start 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe4d692700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe4d692700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe48108b10 con 0x7efe4810b2d0 2024-05-14T15:12:41.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59448/0 (socket says 172.21.15.5:59448) 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe46ffd700 1 -- 172.21.15.5:0/2189105438 learned_addr learned my addr 172.21.15.5:0/2189105438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe46ffd700 1 -- 172.21.15.5:0/2189105438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4810ad00 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe46ffd700 1 --2- 172.21.15.5:0/2189105438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810a7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe30009fb0 tx=0x7efe30011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe300036a0 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe3001a430 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe30019550 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.698+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48108f50 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.699+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe48109430 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.699+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe30019b90 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.699+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe30024730 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.699+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe48064e20 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.702+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efe30043ec0 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.773+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efe48109b20 con 0x7efe4810b2d0 2024-05-14T15:12:41.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.773+0000 7efe3ffff700 1 -- 172.21.15.5:0/2189105438 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7efe30032020 con 0x7efe4810b2d0 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 msgr2=0x7efe4810a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 --2- 172.21.15.5:0/2189105438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810a7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe30009fb0 tx=0x7efe30011940 comp rx=0 tx=0).stop 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 shutdown_connections 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 --2- 172.21.15.5:0/2189105438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810b2d0 0x7efe4810a7c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 >> 172.21.15.5:0/2189105438 conn(0x7efe48103ad0 msgr2=0x7efe48105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 shutdown_connections 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:40.774+0000 7efe4d692700 1 -- 172.21.15.5:0/2189105438 wait complete. 2024-05-14T15:12:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-05-14T15:12:41.304 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:41 smithi005 ceph-mon[31191]: from='client.? 172.21.15.5:0/2189105438' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:12:41.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:41 smithi005 systemd[1]: Stopping Ceph mon.a for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:12:41.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a[31187]: 2024-05-14T15:12:41.552+0000 7ff05d99b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:12:41.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a[31187]: 2024-05-14T15:12:41.552+0000 7ff05d99b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-14T15:12:42.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:41 smithi005 podman[31437]: 2024-05-14 15:12:41.737648777 +0000 UTC m=+0.278267850 container died 69576e83fb39d7cdf2323d454585a92f20246cdfc07334ece0f2e9a714eb4d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:12:42.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 podman[31437]: 2024-05-14 15:12:42.102989727 +0000 UTC m=+0.643608801 container cleanup 69576e83fb39d7cdf2323d454585a92f20246cdfc07334ece0f2e9a714eb4d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-14T15:12:42.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 bash[31437]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a 2024-05-14T15:12:42.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 podman[31449]: 2024-05-14 15:12:42.337330593 +0000 UTC m=+0.599508996 container remove 69576e83fb39d7cdf2323d454585a92f20246cdfc07334ece0f2e9a714eb4d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T15:12:42.693 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T15:12:42.693 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 systemd[1]: Stopped Ceph mon.a for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:12:42.693 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 systemd[1]: Starting Ceph mon.a for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:12:42.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 podman[31540]: 2024-05-14 15:12:42.696493164 +0000 UTC m=+0.112200873 container create 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T15:12:42.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 podman[31540]: 2024-05-14 15:12:42.602040996 +0000 UTC m=+0.017748712 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:12:42.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 podman[31540]: 2024-05-14 15:12:42.882541445 +0000 UTC m=+0.298249179 container init 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-14T15:12:42.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 podman[31540]: 2024-05-14 15:12:42.886143607 +0000 UTC m=+0.301851326 container start 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-14T15:12:42.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev), process ceph-mon, pid 2 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: pidfile_write: ignore empty --pid-file 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: load: jerasure load: lrc 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: RocksDB version: 7.9.2 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Git sha 0 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Compile date 2024-05-09 05:50:40 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: DB SUMMARY 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: DB Session ID: 3WLBAK9VF776U6V6CU8N 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: CURRENT file: CURRENT 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-14T15:12:42.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105646 ; 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.error_if_exists: 0 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.create_if_missing: 0 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.paranoid_checks: 1 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.env: 0x55e6c8ae6500 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.info_log: 0x55e6cb338020 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.statistics: (nil) 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.use_fsync: 0 2024-05-14T15:12:42.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_log_file_size: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.allow_fallocate: 1 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.use_direct_reads: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.db_log_dir: 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.wal_dir: 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T15:12:42.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.write_buffer_manager: 0x55e6cb2028c0 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T15:12:42.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T15:12:42.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T15:12:42.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T15:12:42.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.unordered_write: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.row_cache: None 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.wal_filter: None 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.two_write_queues: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.wal_compression: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.atomic_flush: 0 2024-05-14T15:12:42.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.log_readahead_size: 0 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_background_jobs: 2 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_background_compactions: -1 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_subcompactions: 1 2024-05-14T15:12:42.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_open_files: -1 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_background_flushes: -1 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Compression algorithms supported: 2024-05-14T15:12:42.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kZSTD supported: 0 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kXpressCompression supported: 0 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kZlibCompression supported: 1 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kSnappyCompression supported: 1 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kLZ4Compression supported: 1 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: kBZip2Compression supported: 0 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T15:12:42.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.merge_operator: 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_filter: None 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e6cb3381e0) 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-05-14T15:12:42.972 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x55e6cb2351f0 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-05-14T15:12:42.973 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-05-14T15:12:42.974 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression: NoCompression 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.num_levels: 7 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T15:12:42.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T15:12:42.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T15:12:42.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T15:12:42.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.inplace_update_support: 0 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.bloom_locality: 0 2024-05-14T15:12:42.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.max_successive_merges: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.ttl: 2592000 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.enable_blob_files: false 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.min_blob_size: 0 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T15:12:42.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 878bede0-6c78-448c-92db-69c62f0fd7d4 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715699562927950, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715699562929189, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100753, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98641, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90859, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715699562, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "878bede0-6c78-448c-92db-69c62f0fd7d4", "db_session_id": "3WLBAK9VF776U6V6CU8N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715699562929275, "job": 1, "event": "recovery_finished"} 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T15:12:42.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e6cb219c00 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: DB pointer 0x55e6d000c000 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-14T15:12:42.982 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 100.25 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 100.25 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-14T15:12:42.983 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 28.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 28.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x55e6cb2351f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???) e1 preinit fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).mds e1 new map 2024-05-14T15:12:42.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).mds e1 print_map 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 ceph-mon[31559]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 bash[31540]: 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 2024-05-14T15:12:42.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:42 smithi005 systemd[1]: Started Ceph mon.a for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:12:42.985 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-14T15:12:44.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.428+0000 7fdad137d700 1 Processor -- start 2024-05-14T15:12:44.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.428+0000 7fdad137d700 1 -- start start 2024-05-14T15:12:44.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.429+0000 7fdad137d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc109050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:44.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.429+0000 7fdad137d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc109640 con 0x7fdacc108c40 2024-05-14T15:12:44.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.429+0000 7fdacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:44.109 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.429+0000 7fdacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc109050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59456/0 (socket says 172.21.15.5:59456) 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.429+0000 7fdacaffd700 1 -- 172.21.15.5:0/3767205971 learned_addr learned my addr 172.21.15.5:0/3767205971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.429+0000 7fdacaffd700 1 -- 172.21.15.5:0/3767205971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc109d70 con 0x7fdacc108c40 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.430+0000 7fdacaffd700 1 --2- 172.21.15.5:0/3767205971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc109050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009e80 tx=0x7fdabc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5788d32cd19e7265 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.430+0000 7fdaca7fc700 1 -- 172.21.15.5:0/3767205971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdabc017070 con 0x7fdacc108c40 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.430+0000 7fdaca7fc700 1 -- 172.21.15.5:0/3767205971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdabc00b8c0 con 0x7fdacc108c40 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.430+0000 7fdaca7fc700 1 -- 172.21.15.5:0/3767205971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdabc004490 con 0x7fdacc108c40 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.430+0000 7fdad137d700 1 -- 172.21.15.5:0/3767205971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 msgr2=0x7fdacc109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.430+0000 7fdad137d700 1 --2- 172.21.15.5:0/3767205971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc109050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009e80 tx=0x7fdabc00b330 comp rx=0 tx=0).stop 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.431+0000 7fdad137d700 1 -- 172.21.15.5:0/3767205971 shutdown_connections 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.431+0000 7fdad137d700 1 --2- 172.21.15.5:0/3767205971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc109050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.431+0000 7fdad137d700 1 -- 172.21.15.5:0/3767205971 >> 172.21.15.5:0/3767205971 conn(0x7fdacc1042a0 msgr2=0x7fdacc106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.431+0000 7fdad137d700 1 -- 172.21.15.5:0/3767205971 shutdown_connections 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.431+0000 7fdad137d700 1 -- 172.21.15.5:0/3767205971 wait complete. 2024-05-14T15:12:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.431+0000 7fdad137d700 1 Processor -- start 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdad137d700 1 -- start start 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdad137d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdad137d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10cfa0 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdacaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc19cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59458/0 (socket says 172.21.15.5:59458) 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdacaffd700 1 -- 172.21.15.5:0/2487705559 learned_addr learned my addr 172.21.15.5:0/2487705559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdacaffd700 1 -- 172.21.15.5:0/2487705559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc19d4b0 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.432+0000 7fdacaffd700 1 --2- 172.21.15.5:0/2487705559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc19cf70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009fd0 tx=0x7fdabc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdabc020020 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdabc012430 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc19d740 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdabc017040 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a0350 con 0x7fdacc108c40 2024-05-14T15:12:44.111 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdabc023020 con 0x7fdacc108c40 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.433+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdabc00f050 con 0x7fdacc108c40 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.434+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc064e20 con 0x7fdacc108c40 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.436+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdabc02a020 con 0x7fdacc108c40 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.506+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fdacc07c9f0 con 0x7fdacc108c40 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.509+0000 7fdac8ff9700 1 -- 172.21.15.5:0/2487705559 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fdabc025400 con 0x7fdacc108c40 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 msgr2=0x7fdacc19cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 --2- 172.21.15.5:0/2487705559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc19cf70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009fd0 tx=0x7fdabc00bd50 comp rx=0 tx=0).stop 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 shutdown_connections 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 --2- 172.21.15.5:0/2487705559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdacc108c40 0x7fdacc19cf70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 >> 172.21.15.5:0/2487705559 conn(0x7fdacc1042a0 msgr2=0x7fdacc104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 shutdown_connections 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:43.510+0000 7fdad137d700 1 -- 172.21.15.5:0/2487705559 wait complete. 2024-05-14T15:12:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-14T15:12:44.134 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:12:44.135 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-05-14T15:12:44.135 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-14T15:12:44.136 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-14T15:12:44.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T15:12:44.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: monmap epoch 1 2024-05-14T15:12:44.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: last_changed 2024-05-14T15:12:35.137977+0000 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: min_mon_release 19 (squid) 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: election_strategy: 1 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: fsmap 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: mgrmap e1: no daemons active 2024-05-14T15:12:44.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:43 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2487705559' entity='client.admin' 2024-05-14T15:12:44.399 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x 2024-05-14T15:12:44.399 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-14T15:12:44.550 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4a7148fe-1204-11ef-bc98-c7b262605968.target.wants/ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-4a7148fe-1204-11ef-bc98-c7b262605968@.service. 2024-05-14T15:12:45.088 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T15:12:45.088 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-14T15:12:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T15:12:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-14T15:12:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-14T15:12:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T15:12:45.104 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T15:12:45.104 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-14T15:12:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T15:12:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-14T15:12:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-14T15:12:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-14T15:12:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-05-14T15:12:45.112 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-05-14T15:12:45.173 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:45 smithi005 systemd[1]: Started Ceph mgr.x for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:12:45.477 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:45.176+0000 7f5762d58200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:12:45.477 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:45.370+0000 7f5762d58200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:12:45.477 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:45.480+0000 7f5762d58200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:12:46.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:45 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2629100129' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T15:12:46.030 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:45.819+0000 7f5762d58200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:12:46.031 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:45.909+0000 7f5762d58200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:12:46.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:46.301 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T15:12:46.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T15:12:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T15:12:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:46.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T15:12:38.044493+0000", 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e1d03a700 1 Processor -- start 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e1d03a700 1 -- start start 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e1d03a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1810c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e1d03a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1810ce30 con 0x7f6e1807cc80 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e16d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e16d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1810c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59488/0 (socket says 172.21.15.5:59488) 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.602+0000 7f6e16d9d700 1 -- 172.21.15.5:0/4102806487 learned_addr learned my addr 172.21.15.5:0/4102806487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.603+0000 7f6e16d9d700 1 -- 172.21.15.5:0/4102806487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1810cfb0 con 0x7f6e1807cc80 2024-05-14T15:12:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.603+0000 7f6e16d9d700 1 --2- 172.21.15.5:0/4102806487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1810c860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e080098a0 tx=0x7f6e08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595262eb7a707ccb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.603+0000 7f6e15d9b700 1 -- 172.21.15.5:0/4102806487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e08004160 con 0x7f6e1807cc80 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.603+0000 7f6e15d9b700 1 -- 172.21.15.5:0/4102806487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e08004300 con 0x7f6e1807cc80 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.603+0000 7f6e15d9b700 1 -- 172.21.15.5:0/4102806487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e08011af0 con 0x7f6e1807cc80 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 -- 172.21.15.5:0/4102806487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 msgr2=0x7f6e1810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 --2- 172.21.15.5:0/4102806487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1810c860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e080098a0 tx=0x7f6e08009bb0 comp rx=0 tx=0).stop 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 -- 172.21.15.5:0/4102806487 shutdown_connections 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 --2- 172.21.15.5:0/4102806487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1810c860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 -- 172.21.15.5:0/4102806487 >> 172.21.15.5:0/4102806487 conn(0x7f6e18104290 msgr2=0x7f6e181066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 -- 172.21.15.5:0/4102806487 shutdown_connections 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.604+0000 7f6e1d03a700 1 -- 172.21.15.5:0/4102806487 wait complete. 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.605+0000 7f6e1d03a700 1 Processor -- start 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.605+0000 7f6e1d03a700 1 -- start start 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e1d03a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e1d03a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1810dfc0 con 0x7f6e1807cc80 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e16d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:46.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e16d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59490/0 (socket says 172.21.15.5:59490) 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e16d9d700 1 -- 172.21.15.5:0/2629100129 learned_addr learned my addr 172.21.15.5:0/2629100129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e16d9d700 1 -- 172.21.15.5:0/2629100129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1819e5e0 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.606+0000 7f6e16d9d700 1 --2- 172.21.15.5:0/2629100129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1819e0a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009fb0 tx=0x7f6e08011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e080036a0 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e0801a430 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1819e870 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e08019550 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e181a1480 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e08019b90 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.607+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e080248a0 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.608+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e18064e20 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.610+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6e08043ec0 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.687+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6e181a1650 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.687+0000 7f6dfffff700 1 -- 172.21.15.5:0/2629100129 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6e08032030 con 0x7f6e1807cc80 2024-05-14T15:12:46.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.688+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 msgr2=0x7f6e1819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.688+0000 7f6e1d03a700 1 --2- 172.21.15.5:0/2629100129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1819e0a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009fb0 tx=0x7f6e08011940 comp rx=0 tx=0).stop 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.688+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 shutdown_connections 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.689+0000 7f6e1d03a700 1 --2- 172.21.15.5:0/2629100129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e1807cc80 0x7f6e1819e0a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.689+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 >> 172.21.15.5:0/2629100129 conn(0x7f6e18104290 msgr2=0x7f6e18104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.689+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 shutdown_connections 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:45.689+0000 7f6e1d03a700 1 -- 172.21.15.5:0/2629100129 wait complete. 2024-05-14T15:12:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-05-14T15:12:47.532 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:47.181+0000 7f5762d58200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:12:47.533 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:47.358+0000 7f5762d58200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:12:47.533 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:47.536+0000 7f5762d58200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:12:48.144 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:47.837+0000 7f5762d58200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:12:48.144 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:47.936+0000 7f5762d58200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:12:48.144 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:48.147+0000 7f5762d58200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:12:48.424 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:48.352+0000 7f5762d58200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:12:49.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:48 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/779487619' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T15:12:49.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:49.008+0000 7f5762d58200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:12:49.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:49.243+0000 7f5762d58200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:12:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:12:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:12:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:12:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T15:12:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T15:12:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-14T15:12:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:49.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T15:12:49.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T15:12:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T15:12:38.044493+0000", 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.732+0000 7f99ac54d700 1 Processor -- start 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.733+0000 7f99ac54d700 1 -- start start 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.733+0000 7f99ac54d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.733+0000 7f99ac54d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99a407cdd0 con 0x7f99a410a7d0 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.733+0000 7f99aa2e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:49.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.733+0000 7f99aa2e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a410cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39610/0 (socket says 172.21.15.5:39610) 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.733+0000 7f99aa2e9700 1 -- 172.21.15.5:0/1484636924 learned_addr learned my addr 172.21.15.5:0/1484636924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.734+0000 7f99aa2e9700 1 -- 172.21.15.5:0/1484636924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a410d130 con 0x7f99a410a7d0 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.734+0000 7f99aa2e9700 1 --2- 172.21.15.5:0/1484636924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a410cbf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99a00098a0 tx=0x7f99a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc254e601a41f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.734+0000 7f99a92e7700 1 -- 172.21.15.5:0/1484636924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0004160 con 0x7f99a410a7d0 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.734+0000 7f99a92e7700 1 -- 172.21.15.5:0/1484636924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99a0004300 con 0x7f99a410a7d0 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.734+0000 7f99a92e7700 1 -- 172.21.15.5:0/1484636924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0011af0 con 0x7f99a410a7d0 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 -- 172.21.15.5:0/1484636924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 msgr2=0x7f99a410cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 --2- 172.21.15.5:0/1484636924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a410cbf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99a00098a0 tx=0x7f99a0009bb0 comp rx=0 tx=0).stop 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 -- 172.21.15.5:0/1484636924 shutdown_connections 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 --2- 172.21.15.5:0/1484636924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a410cbf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 -- 172.21.15.5:0/1484636924 >> 172.21.15.5:0/1484636924 conn(0x7f99a4104290 msgr2=0x7f99a41066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 -- 172.21.15.5:0/1484636924 shutdown_connections 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.735+0000 7f99ac54d700 1 -- 172.21.15.5:0/1484636924 wait complete. 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99ac54d700 1 Processor -- start 2024-05-14T15:12:49.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99ac54d700 1 -- start start 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99ac54d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a4109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99ac54d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99a410dfc0 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99aa2e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a4109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99aa2e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a4109f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39612/0 (socket says 172.21.15.5:39612) 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99aa2e9700 1 -- 172.21.15.5:0/779487619 learned_addr learned my addr 172.21.15.5:0/779487619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.736+0000 7f99aa2e9700 1 -- 172.21.15.5:0/779487619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a410a490 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f99aa2e9700 1 --2- 172.21.15.5:0/779487619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a4109f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009fb0 tx=0x7f99a0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a00036a0 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41086e0 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99a001a430 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0019550 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a4108bc0 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f99a0019be0 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.737+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99a00248b0 con 0x7f99a410a7d0 2024-05-14T15:12:49.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.738+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a4064e20 con 0x7f99a410a7d0 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.740+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f99a0043ec0 con 0x7f99a410a7d0 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.816+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f99a4108d90 con 0x7f99a410a7d0 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.816+0000 7f999b7fe700 1 -- 172.21.15.5:0/779487619 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f99a0032020 con 0x7f99a410a7d0 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 msgr2=0x7f99a4109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 --2- 172.21.15.5:0/779487619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a4109f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009fb0 tx=0x7f99a0011940 comp rx=0 tx=0).stop 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 shutdown_connections 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 --2- 172.21.15.5:0/779487619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a410a7d0 0x7f99a4109f50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 >> 172.21.15.5:0/779487619 conn(0x7f99a4104290 msgr2=0x7f99a4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 shutdown_connections 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:48.817+0000 7f99ac54d700 1 -- 172.21.15.5:0/779487619 wait complete. 2024-05-14T15:12:49.426 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-05-14T15:12:50.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:49.689+0000 7f5762d58200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:12:50.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:49.791+0000 7f5762d58200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:12:50.548 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:50.176+0000 7f5762d58200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:12:50.548 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:50.551+0000 7f5762d58200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:12:50.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:50.648+0000 7f5762d58200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:12:51.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:50.904+0000 7f5762d58200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:12:51.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:51.046+0000 7f5762d58200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:12:51.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:51.146+0000 7f5762d58200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:12:51.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:51.490+0000 7f5762d58200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:12:51.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:51.587+0000 7f5762d58200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:12:52.501 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:52 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:52.224+0000 7f5762d58200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:12:52.501 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:12:52 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:12:52.451+0000 7f5762d58200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:12:52.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: Activating manager daemon x 2024-05-14T15:12:52.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: mgrmap e2: x(active, starting, since 0.00382137s) 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: Manager daemon x is now available 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' 2024-05-14T15:12:52.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:52 smithi005 ceph-mon[31559]: from='mgr.14100 172.21.15.5:0/4275960240' entity='mgr.x' 2024-05-14T15:12:54.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:54 smithi005 ceph-mon[31559]: mgrmap e3: x(active, since 1.00854s) 2024-05-14T15:12:55.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:55 smithi005 ceph-mon[31559]: mgrmap e4: x(active, since 2s) 2024-05-14T15:12:55.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:12:55 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3031910275' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T15:13:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:13:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:13:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:13:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T15:13:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-14T15:13:00.669 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T15:13:00.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T15:13:00.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T15:13:00.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T15:12:38.044493+0000", 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.541+0000 7fa10cfe8700 1 Processor -- start 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.541+0000 7fa10cfe8700 1 -- start start 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.541+0000 7fa10cfe8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa108109080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.541+0000 7fa10cfe8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa108109670 con 0x7fa108108c70 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa108109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa108109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39688/0 (socket says 172.21.15.5:39688) 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10659c700 1 -- 172.21.15.5:0/3088448009 learned_addr learned my addr 172.21.15.5:0/3088448009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10659c700 1 -- 172.21.15.5:0/3088448009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa108109da0 con 0x7fa108108c70 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10659c700 1 --2- 172.21.15.5:0/3088448009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa108109080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80098a0 tx=0x7fa0f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc508960e5d0363 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10559a700 1 -- 172.21.15.5:0/3088448009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f8004160 con 0x7fa108108c70 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10559a700 1 -- 172.21.15.5:0/3088448009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa0f8004300 con 0x7fa108108c70 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.542+0000 7fa10559a700 1 -- 172.21.15.5:0/3088448009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f8011af0 con 0x7fa108108c70 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3088448009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 msgr2=0x7fa108109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 --2- 172.21.15.5:0/3088448009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa108109080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80098a0 tx=0x7fa0f8009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3088448009 shutdown_connections 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 --2- 172.21.15.5:0/3088448009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa108109080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3088448009 >> 172.21.15.5:0/3088448009 conn(0x7fa108104330 msgr2=0x7fa108106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3088448009 shutdown_connections 2024-05-14T15:13:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.543+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3088448009 wait complete. 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10cfe8700 1 Processor -- start 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10cfe8700 1 -- start start 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10cfe8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa10819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10cfe8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa10810cfd0 con 0x7fa108108c70 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa10819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa10819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39700/0 (socket says 172.21.15.5:39700) 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.544+0000 7fa10659c700 1 -- 172.21.15.5:0/3031910275 learned_addr learned my addr 172.21.15.5:0/3031910275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa10659c700 1 -- 172.21.15.5:0/3031910275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819d540 con 0x7fa108108c70 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa10659c700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa10819d000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009fb0 tx=0x7fa0f8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f80036a0 con 0x7fa108108c70 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa0f8003cd0 con 0x7fa108108c70 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819d7d0 con 0x7fa108108c70 2024-05-14T15:13:00.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f801a430 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.545+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a03e0 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.546+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa108064e20 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.546+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa0f801a6f0 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.546+0000 7fa0ff7fe700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fa0f403e3d0 0x7fa0f40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.546+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa0f80536b0 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.547+0000 7fa105d9b700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fa0f403e3d0 0x7fa0f40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.547+0000 7fa105d9b700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fa0f403e3d0 0x7fa0f40408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0006fd0 tx=0x7fa0f0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.552+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0f8017e90 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.865+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa1081a0a60 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.865+0000 7fa0ff7fe700 1 -- 172.21.15.5:0/3031910275 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa0f802c350 con 0x7fa108108c70 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.867+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fa0f403e3d0 msgr2=0x7fa0f40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:00.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.867+0000 7fa10cfe8700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fa0f403e3d0 0x7fa0f40408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0006fd0 tx=0x7fa0f0006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.867+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 msgr2=0x7fa10819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.867+0000 7fa10cfe8700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa10819d000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009fb0 tx=0x7fa0f8011890 comp rx=0 tx=0).stop 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.868+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 shutdown_connections 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.868+0000 7fa10cfe8700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fa0f403e3d0 0x7fa0f40408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.868+0000 7fa10cfe8700 1 --2- 172.21.15.5:0/3031910275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa108108c70 0x7fa10819d000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.868+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 >> 172.21.15.5:0/3031910275 conn(0x7fa108104330 msgr2=0x7fa108105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.868+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 shutdown_connections 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:12:54.868+0000 7fa10cfe8700 1 -- 172.21.15.5:0/3031910275 wait complete. 2024-05-14T15:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-05-14T15:13:04.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:03 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2170561462' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T15:13:05.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:13:05.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T15:13:05.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.458+0000 7f48865f6700 1 Processor -- start 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.458+0000 7f48865f6700 1 -- start start 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.458+0000 7f48865f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f488010c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.458+0000 7f48865f6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f488010ce30 con 0x7f488007ccc0 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f488010c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f488010c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34618/0 (socket says 172.21.15.5:34618) 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487ffff700 1 -- 172.21.15.5:0/1221443625 learned_addr learned my addr 172.21.15.5:0/1221443625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487ffff700 1 -- 172.21.15.5:0/1221443625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488010cfb0 con 0x7f488007ccc0 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487ffff700 1 --2- 172.21.15.5:0/1221443625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f488010c820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f48700098a0 tx=0x7f4870009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6262430cba15ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487f7fe700 1 -- 172.21.15.5:0/1221443625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4870004160 con 0x7f488007ccc0 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487f7fe700 1 -- 172.21.15.5:0/1221443625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4870004300 con 0x7f488007ccc0 2024-05-14T15:13:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.459+0000 7f487f7fe700 1 -- 172.21.15.5:0/1221443625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4870011af0 con 0x7f488007ccc0 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 -- 172.21.15.5:0/1221443625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 msgr2=0x7f488010c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 --2- 172.21.15.5:0/1221443625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f488010c820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f48700098a0 tx=0x7f4870009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 -- 172.21.15.5:0/1221443625 shutdown_connections 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 --2- 172.21.15.5:0/1221443625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f488010c820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 -- 172.21.15.5:0/1221443625 >> 172.21.15.5:0/1221443625 conn(0x7f4880104290 msgr2=0x7f48801066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 -- 172.21.15.5:0/1221443625 shutdown_connections 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.460+0000 7f48865f6700 1 -- 172.21.15.5:0/1221443625 wait complete. 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.461+0000 7f48865f6700 1 Processor -- start 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.461+0000 7f48865f6700 1 -- start start 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.461+0000 7f48865f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f4880109f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.461+0000 7f48865f6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f488010dfc0 con 0x7f488007ccc0 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f4880109f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f4880109f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34634/0 (socket says 172.21.15.5:34634) 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487ffff700 1 -- 172.21.15.5:0/2170561462 learned_addr learned my addr 172.21.15.5:0/2170561462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:05.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487ffff700 1 -- 172.21.15.5:0/2170561462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488010a4d0 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487ffff700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f4880109f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4870009fb0 tx=0x7f4870011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48700036a0 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f487001a430 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4870019550 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4880108720 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.462+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4880108c00 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.463+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f4870019800 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.463+0000 7f487dffb700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f486c03e420 0x7f486c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.464+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4870053940 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.464+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4880064e20 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.464+0000 7f4877fff700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f486c03e420 0x7f486c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.464+0000 7f4877fff700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f486c03e420 0x7f486c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4868006fd0 tx=0x7f4868006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.469+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4870017e90 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.683+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f48801092d0 con 0x7f488007ccc0 2024-05-14T15:13:05.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.684+0000 7f487dffb700 1 -- 172.21.15.5:0/2170561462 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f4870023390 con 0x7f488007ccc0 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f486c03e420 msgr2=0x7f486c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f486c03e420 0x7f486c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4868006fd0 tx=0x7f4868006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 msgr2=0x7f4880109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f4880109f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4870009fb0 tx=0x7f4870011940 comp rx=0 tx=0).stop 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 shutdown_connections 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f486c03e420 0x7f486c040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 --2- 172.21.15.5:0/2170561462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f488007ccc0 0x7f4880109f90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 >> 172.21.15.5:0/2170561462 conn(0x7f4880104290 msgr2=0x7f4880104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 shutdown_connections 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:03.687+0000 7f48865f6700 1 -- 172.21.15.5:0/2170561462 wait complete. 2024-05-14T15:13:05.193 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-05-14T15:13:06.177 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:05 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3737737394' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-14T15:13:06.178 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: ignoring --setuser ceph since I am not root 2024-05-14T15:13:06.178 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: ignoring --setgroup ceph since I am not root 2024-05-14T15:13:06.178 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:06.180+0000 7fd30e8c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:13:06.482 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:06.377+0000 7fd30e8c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.638+0000 7f1af7036700 1 Processor -- start 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.638+0000 7f1af7036700 1 -- start start 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.638+0000 7f1af7036700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af010c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.638+0000 7f1af7036700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010cf30 con 0x7f1af007d4f0 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.638+0000 7f1af4dd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af010c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.638+0000 7f1af4dd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af010c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34638/0 (socket says 172.21.15.5:34638) 2024-05-14T15:13:06.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.639+0000 7f1af4dd2700 1 -- 172.21.15.5:0/2294045607 learned_addr learned my addr 172.21.15.5:0/2294045607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.639+0000 7f1af4dd2700 1 -- 172.21.15.5:0/2294045607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af010d0b0 con 0x7f1af007d4f0 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.639+0000 7f1af4dd2700 1 --2- 172.21.15.5:0/2294045607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af010c960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40098a0 tx=0x7f1ae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863956b723f50b1f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.639+0000 7f1aef7fe700 1 -- 172.21.15.5:0/2294045607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae4004160 con 0x7f1af007d4f0 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.639+0000 7f1aef7fe700 1 -- 172.21.15.5:0/2294045607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ae4004300 con 0x7f1af007d4f0 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.639+0000 7f1aef7fe700 1 -- 172.21.15.5:0/2294045607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae4011af0 con 0x7f1af007d4f0 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 -- 172.21.15.5:0/2294045607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 msgr2=0x7f1af010c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 --2- 172.21.15.5:0/2294045607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af010c960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40098a0 tx=0x7f1ae4009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 -- 172.21.15.5:0/2294045607 shutdown_connections 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 --2- 172.21.15.5:0/2294045607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af010c960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 -- 172.21.15.5:0/2294045607 >> 172.21.15.5:0/2294045607 conn(0x7f1af0104370 msgr2=0x7f1af01067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 -- 172.21.15.5:0/2294045607 shutdown_connections 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.640+0000 7f1af7036700 1 -- 172.21.15.5:0/2294045607 wait complete. 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af7036700 1 Processor -- start 2024-05-14T15:13:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af7036700 1 -- start start 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af7036700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af7036700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1af010e050 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af4dd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af4dd2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af019e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34642/0 (socket says 172.21.15.5:34642) 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af4dd2700 1 -- 172.21.15.5:0/3737737394 learned_addr learned my addr 172.21.15.5:0/3737737394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.641+0000 7f1af4dd2700 1 -- 172.21.15.5:0/3737737394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af019e680 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.642+0000 7f1af4dd2700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af019e140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40117b0 tx=0x7f1ae4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.642+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae40039c0 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.642+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ae4003b60 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.642+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ae40196a0 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.642+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af019e910 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.642+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a1520 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.643+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af0064e20 con 0x7f1af007d4f0 2024-05-14T15:13:06.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.643+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f1ae401ab00 con 0x7f1af007d4f0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.643+0000 7f1aedffb700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f1ae003e420 0x7f1ae0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.644+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ae4053b40 con 0x7f1af007d4f0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.644+0000 7f1aeffff700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f1ae003e420 0x7f1ae0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.644+0000 7f1aeffff700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f1ae003e420 0x7f1ae0040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1adc006fd0 tx=0x7f1adc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.649+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ae4018d60 con 0x7f1af007d4f0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.923+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1af002e8f0 con 0x7f1af007d4f0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.974+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1ae4053560 con 0x7f1af007d4f0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.976+0000 7f1aedffb700 1 -- 172.21.15.5:0/3737737394 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f1ae402d440 con 0x7f1af007d4f0 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.977+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f1ae003e420 msgr2=0x7f1ae0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.977+0000 7f1af7036700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f1ae003e420 0x7f1ae0040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1adc006fd0 tx=0x7f1adc006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.977+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 msgr2=0x7f1af019e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:06.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.977+0000 7f1af7036700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af019e140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40117b0 tx=0x7f1ae4011b70 comp rx=0 tx=0).stop 2024-05-14T15:13:06.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.978+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 shutdown_connections 2024-05-14T15:13:06.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.978+0000 7f1af7036700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f1ae003e420 0x7f1ae0040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:06.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.978+0000 7f1af7036700 1 --2- 172.21.15.5:0/3737737394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af007d4f0 0x7f1af019e140 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:06.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.978+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 >> 172.21.15.5:0/3737737394 conn(0x7f1af0104370 msgr2=0x7f1af0104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:06.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.978+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 shutdown_connections 2024-05-14T15:13:06.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:05.978+0000 7f1af7036700 1 -- 172.21.15.5:0/3737737394 wait complete. 2024-05-14T15:13:06.803 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:06.485+0000 7fd30e8c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:13:07.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:06 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3737737394' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-14T15:13:07.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:06 smithi005 ceph-mon[31559]: mgrmap e5: x(active, since 13s) 2024-05-14T15:13:07.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:06.824+0000 7fd30e8c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:13:07.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:06.915+0000 7fd30e8c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:13:07.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b96495700 1 Processor -- start 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b96495700 1 -- start start 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b96495700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9010cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b96495700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9007cdd0 con 0x7f6b9010a7d0 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9010cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9010cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55776/0 (socket says 172.21.15.5:55776) 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.038+0000 7f6b8ffff700 1 -- 172.21.15.5:0/1691028568 learned_addr learned my addr 172.21.15.5:0/1691028568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:07.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.039+0000 7f6b8ffff700 1 -- 172.21.15.5:0/1691028568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9010d130 con 0x7f6b9010a7d0 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.039+0000 7f6b8ffff700 1 --2- 172.21.15.5:0/1691028568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9010cbf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b800098a0 tx=0x7f6b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed28231c97958123 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.039+0000 7f6b8effd700 1 -- 172.21.15.5:0/1691028568 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b80004160 con 0x7f6b9010a7d0 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.039+0000 7f6b8effd700 1 -- 172.21.15.5:0/1691028568 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b80004300 con 0x7f6b9010a7d0 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.039+0000 7f6b8effd700 1 -- 172.21.15.5:0/1691028568 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b80011af0 con 0x7f6b9010a7d0 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 -- 172.21.15.5:0/1691028568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 msgr2=0x7f6b9010cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 --2- 172.21.15.5:0/1691028568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9010cbf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b800098a0 tx=0x7f6b80009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 -- 172.21.15.5:0/1691028568 shutdown_connections 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 --2- 172.21.15.5:0/1691028568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9010cbf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 -- 172.21.15.5:0/1691028568 >> 172.21.15.5:0/1691028568 conn(0x7f6b90104290 msgr2=0x7f6b901066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 -- 172.21.15.5:0/1691028568 shutdown_connections 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.040+0000 7f6b96495700 1 -- 172.21.15.5:0/1691028568 wait complete. 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b96495700 1 Processor -- start 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b96495700 1 -- start start 2024-05-14T15:13:07.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b96495700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b96495700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010dfc0 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b8ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55782/0 (socket says 172.21.15.5:55782) 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b8ffff700 1 -- 172.21.15.5:0/448910701 learned_addr learned my addr 172.21.15.5:0/448910701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.041+0000 7f6b8ffff700 1 -- 172.21.15.5:0/448910701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9019e5a0 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.042+0000 7f6b8ffff700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9019e060 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009fb0 tx=0x7f6b80011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.042+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b800036a0 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.042+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b8001a430 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.042+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9019e830 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.042+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b80019550 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.042+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b901a1440 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.043+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f6b80019800 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.043+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b90064e20 con 0x7f6b9010a7d0 2024-05-14T15:13:07.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.043+0000 7f6b8d7fa700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 0x7f6b78040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.043+0000 7f6b8f7fe700 1 -- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 msgr2=0x7f6b78040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.043+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6b80054830 con 0x7f6b9010a7d0 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.043+0000 7f6b8f7fe700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 0x7f6b78040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.049+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b80017e90 con 0x7f6b9010a7d0 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.244+0000 7f6b8f7fe700 1 -- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 msgr2=0x7f6b78040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.244+0000 7f6b8f7fe700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 0x7f6b78040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.331+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6b901a1ae0 con 0x7f6b9010a7d0 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.332+0000 7f6b8d7fa700 1 -- 172.21.15.5:0/448910701 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6b80023390 con 0x7f6b9010a7d0 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.336+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 msgr2=0x7f6b78040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.336+0000 7f6b96495700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 0x7f6b78040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.336+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 msgr2=0x7f6b9019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.336+0000 7f6b96495700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9019e060 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009fb0 tx=0x7f6b80011940 comp rx=0 tx=0).stop 2024-05-14T15:13:07.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.337+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 shutdown_connections 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.337+0000 7f6b96495700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7f6b7803e470 0x7f6b78040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.337+0000 7f6b96495700 1 --2- 172.21.15.5:0/448910701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b9010a7d0 0x7f6b9019e060 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.337+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 >> 172.21.15.5:0/448910701 conn(0x7f6b90104290 msgr2=0x7f6b90104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.337+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 shutdown_connections 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:07.337+0000 7f6b96495700 1 -- 172.21.15.5:0/448910701 wait complete. 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-05-14T15:13:07.888 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-05-14T15:13:08.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:07 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/448910701' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T15:13:08.312 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:08.198+0000 7fd30e8c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:13:08.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:08.387+0000 7fd30e8c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:13:08.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:08.566+0000 7fd30e8c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:13:09.184 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:08.873+0000 7fd30e8c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:13:09.184 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:08.972+0000 7fd30e8c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:13:09.184 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:09.187+0000 7fd30e8c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:13:09.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:09.392+0000 7fd30e8c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:13:10.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:10.033+0000 7fd30e8c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:13:10.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:10.271+0000 7fd30e8c2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:13:11.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:10.725+0000 7fd30e8c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:13:11.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:10.828+0000 7fd30e8c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:13:11.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:11 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:11.239+0000 7fd30e8c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:13:11.980 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:11 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:11.619+0000 7fd30e8c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:13:11.980 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:11 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:11.719+0000 7fd30e8c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:13:12.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:11 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:11.983+0000 7fd30e8c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:13:12.312 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:12 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:12.125+0000 7fd30e8c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:13:12.312 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:12 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:12.228+0000 7fd30e8c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:13:13.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:12 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:12.580+0000 7fd30e8c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:13:13.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:12 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:12.678+0000 7fd30e8c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:13:13.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:13 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:13.318+0000 7fd30e8c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:13:13.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:13 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:13.549+0000 7fd30e8c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:13:14.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: Active manager daemon x restarted 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: Activating manager daemon x 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: osdmap e2: 0 total, 0 up, 0 in 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: mgrmap e6: x(active, starting, since 0.0042252s) 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: Manager daemon x is now available 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:14.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:13 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:14.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:14 smithi005 ceph-mon[31559]: Found migration_current of "None". Setting to last migration. 2024-05-14T15:13:14.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:14 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T15:13:14.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:14 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T15:13:14.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:14 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:14.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:14 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:14.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:14 smithi005 ceph-mon[31559]: mgrmap e7: x(active, since 1.00688s) 2024-05-14T15:13:15.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:13:15.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-14T15:13:15.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbbaae1f700 1 Processor -- start 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbbaae1f700 1 -- start start 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbbaae1f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba410ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbbaae1f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbba410f1c0 con 0x7fbba407cc60 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbba8bbb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba410ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbba8bbb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba410ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55786/0 (socket says 172.21.15.5:55786) 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.338+0000 7fbba8bbb700 1 -- 172.21.15.5:0/1193947232 learned_addr learned my addr 172.21.15.5:0/1193947232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.339+0000 7fbba8bbb700 1 -- 172.21.15.5:0/1193947232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba410f340 con 0x7fbba407cc60 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.339+0000 7fbba8bbb700 1 --2- 172.21.15.5:0/1193947232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba410ec80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098a0 tx=0x7fbb98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ede5560120960f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.339+0000 7fbba37fe700 1 -- 172.21.15.5:0/1193947232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98004160 con 0x7fbba407cc60 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.339+0000 7fbba37fe700 1 -- 172.21.15.5:0/1193947232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbb98004300 con 0x7fbba407cc60 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.339+0000 7fbba37fe700 1 -- 172.21.15.5:0/1193947232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98011af0 con 0x7fbba407cc60 2024-05-14T15:13:15.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1193947232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 msgr2=0x7fbba410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1193947232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba410ec80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098a0 tx=0x7fbb98009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1193947232 shutdown_connections 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1193947232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba410ec80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1193947232 >> 172.21.15.5:0/1193947232 conn(0x7fbba4104290 msgr2=0x7fbba41066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1193947232 shutdown_connections 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.340+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1193947232 wait complete. 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbbaae1f700 1 Processor -- start 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbbaae1f700 1 -- start start 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbbaae1f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba41a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbbaae1f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41101e0 con 0x7fbba407cc60 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbba8bbb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba41a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbba8bbb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba41a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55788/0 (socket says 172.21.15.5:55788) 2024-05-14T15:13:15.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbba8bbb700 1 -- 172.21.15.5:0/1401044358 learned_addr learned my addr 172.21.15.5:0/1401044358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.341+0000 7fbba8bbb700 1 -- 172.21.15.5:0/1401044358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba41a0810 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.342+0000 7fbba8bbb700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba41a02d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009fb0 tx=0x7fbb98011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.342+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb980036a0 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.342+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbb9801a430 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.342+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba41a0aa0 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.342+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb98019550 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.342+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba41a36b0 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.343+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fbb98019850 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.343+0000 7fbba1ffb700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.343+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 --> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb94041040 con 0x7fbb9403e420 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.343+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb98053890 con 0x7fbba407cc60 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.343+0000 7fbba3fff700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 msgr2=0x7fbb94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.343+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.544+0000 7fbba3fff700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 msgr2=0x7fbb94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.544+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.945+0000 7fbba3fff700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 msgr2=0x7fbb94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:08.945+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:09.746+0000 7fbba3fff700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 msgr2=0x7fbb94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:09.746+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:11.347+0000 7fbba3fff700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 msgr2=0x7fbb94040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2401136202 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:11.347+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:13.553+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fbb98019b90 con 0x7fbba407cc60 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:13.553+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 msgr2=0x7fbb94040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:13.553+0000 7fbba1ffb700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.555+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fbb980555c0 con 0x7fbba407cc60 2024-05-14T15:13:15.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.555+0000 7fbba1ffb700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fbb94041d60 0x7fbb94044180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.555+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb94041040 con 0x7fbb94041d60 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.556+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fbb94041d60 0x7fbb94044180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.556+0000 7fbba3fff700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fbb94041d60 0x7fbb94044180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbb90003a10 tx=0x7fbb90009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.557+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fbb94041040 con 0x7fbb94041d60 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.565+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fbba4064e20 con 0x7fbb94041d60 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbba1ffb700 1 -- 172.21.15.5:0/1401044358 <== mgr.14118 v2:172.21.15.5:6800/2154399460 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fbba4064e20 con 0x7fbb94041d60 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fbb94041d60 msgr2=0x7fbb94044180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fbb94041d60 0x7fbb94044180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbb90003a10 tx=0x7fbb90009290 comp rx=0 tx=0).stop 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 msgr2=0x7fbba41a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba41a02d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009fb0 tx=0x7fbb98011940 comp rx=0 tx=0).stop 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 shutdown_connections 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fbb94041d60 0x7fbb94044180 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbba407cc60 0x7fbba41a02d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:15.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 --2- 172.21.15.5:0/1401044358 >> [v2:172.21.15.5:6800/2401136202,v1:172.21.15.5:6801/2401136202] conn(0x7fbb9403e420 0x7fbb94040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 >> 172.21.15.5:0/1401044358 conn(0x7fbba4104290 msgr2=0x7fbba4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 shutdown_connections 2024-05-14T15:13:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:14.566+0000 7fbbaae1f700 1 -- 172.21.15.5:0/1401044358 wait complete. 2024-05-14T15:13:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-05-14T15:13:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-05-14T15:13:16.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:15 smithi005 ceph-mon[31559]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T15:13:16.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:15 smithi005 ceph-mon[31559]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T15:13:16.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:15 smithi005 ceph-mon[31559]: [14/May/2024:15:13:14] ENGINE Bus STARTING 2024-05-14T15:13:16.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:15 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.528+0000 7f84ab7b3700 1 Processor -- start 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.528+0000 7f84ab7b3700 1 -- start start 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.528+0000 7f84ab7b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a410a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.528+0000 7f84ab7b3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410ac00 con 0x7f84a410a220 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a954f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a410a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a954f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a410a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55874/0 (socket says 172.21.15.5:55874) 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a954f700 1 -- 172.21.15.5:0/2995546130 learned_addr learned my addr 172.21.15.5:0/2995546130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a954f700 1 -- 172.21.15.5:0/2995546130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a4108ea0 con 0x7f84a410a220 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a954f700 1 --2- 172.21.15.5:0/2995546130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a410a630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8494009e80 tx=0x7f849400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f789d667169a71f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a3fff700 1 -- 172.21.15.5:0/2995546130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8494017070 con 0x7f84a410a220 2024-05-14T15:13:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a3fff700 1 -- 172.21.15.5:0/2995546130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f849400b8c0 con 0x7f84a410a220 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.529+0000 7f84a3fff700 1 -- 172.21.15.5:0/2995546130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8494004490 con 0x7f84a410a220 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 -- 172.21.15.5:0/2995546130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 msgr2=0x7f84a410a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 --2- 172.21.15.5:0/2995546130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a410a630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8494009e80 tx=0x7f849400b330 comp rx=0 tx=0).stop 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 -- 172.21.15.5:0/2995546130 shutdown_connections 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 --2- 172.21.15.5:0/2995546130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a410a630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 -- 172.21.15.5:0/2995546130 >> 172.21.15.5:0/2995546130 conn(0x7f84a4103b00 msgr2=0x7f84a4105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 -- 172.21.15.5:0/2995546130 shutdown_connections 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.530+0000 7f84ab7b3700 1 -- 172.21.15.5:0/2995546130 wait complete. 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84ab7b3700 1 Processor -- start 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84ab7b3700 1 -- start start 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84ab7b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84ab7b3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410d0e0 con 0x7f84a410a220 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84a954f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84a954f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a419e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55878/0 (socket says 172.21.15.5:55878) 2024-05-14T15:13:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84a954f700 1 -- 172.21.15.5:0/900864310 learned_addr learned my addr 172.21.15.5:0/900864310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.531+0000 7f84a954f700 1 -- 172.21.15.5:0/900864310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419e610 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.532+0000 7f84a954f700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a419e0d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f849400bca0 tx=0x7f849400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.532+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8494020020 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.532+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8494012430 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.532+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419e8a0 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.532+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8494017040 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.532+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a14b0 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.533+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f8494023020 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.533+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a4064e20 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.533+0000 7f84a27fc700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f848c03e2f0 0x7f848c040800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.533+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f849400f050 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.533+0000 7f84a8d4e700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f848c03e2f0 0x7f848c040800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.534+0000 7f84a8d4e700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f848c03e2f0 0x7f848c040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8498006fd0 tx=0x7f8498006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.539+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f849402a020 con 0x7f84a410a220 2024-05-14T15:13:16.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.608+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8494025400 con 0x7f84a410a220 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.772+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f84a402e960 con 0x7f848c03e2f0 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.777+0000 7f84a27fc700 1 -- 172.21.15.5:0/900864310 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f84a402e960 con 0x7f848c03e2f0 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f848c03e2f0 msgr2=0x7f848c040800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f848c03e2f0 0x7f848c040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8498006fd0 tx=0x7f8498006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 msgr2=0x7f84a419e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a419e0d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f849400bca0 tx=0x7f849400bd80 comp rx=0 tx=0).stop 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 shutdown_connections 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f848c03e2f0 0x7f848c040800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 --2- 172.21.15.5:0/900864310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a410a220 0x7f84a419e0d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 >> 172.21.15.5:0/900864310 conn(0x7f84a4103b00 msgr2=0x7f84a41946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 shutdown_connections 2024-05-14T15:13:16.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:15.780+0000 7f84ab7b3700 1 -- 172.21.15.5:0/900864310 wait complete. 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: [14/May/2024:15:13:15] ENGINE Serving on http://172.21.15.5:8765 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: [14/May/2024:15:13:15] ENGINE Serving on https://172.21.15.5:7150 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: [14/May/2024:15:13:15] ENGINE Bus STARTED 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: mgrmap e8: x(active, since 2s) 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:16.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:16 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.743+0000 7fd7f5826700 1 Processor -- start 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.743+0000 7fd7f5826700 1 -- start start 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.744+0000 7fd7f5826700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f010cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.744+0000 7fd7f5826700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f007cdd0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.744+0000 7fd7eeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f010cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.744+0000 7fd7eeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f010cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41322/0 (socket says 172.21.15.5:41322) 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.744+0000 7fd7eeffd700 1 -- 172.21.15.5:0/616007445 learned_addr learned my addr 172.21.15.5:0/616007445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.744+0000 7fd7eeffd700 1 -- 172.21.15.5:0/616007445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f010d130 con 0x7fd7f010a7d0 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.745+0000 7fd7eeffd700 1 --2- 172.21.15.5:0/616007445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f010cbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00098a0 tx=0x7fd7e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2264aa1974249ccd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.745+0000 7fd7ee7fc700 1 -- 172.21.15.5:0/616007445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0004160 con 0x7fd7f010a7d0 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.745+0000 7fd7ee7fc700 1 -- 172.21.15.5:0/616007445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd7e0004300 con 0x7fd7f010a7d0 2024-05-14T15:13:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.745+0000 7fd7ee7fc700 1 -- 172.21.15.5:0/616007445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0011af0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.745+0000 7fd7f5826700 1 -- 172.21.15.5:0/616007445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 msgr2=0x7fd7f010cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.745+0000 7fd7f5826700 1 --2- 172.21.15.5:0/616007445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f010cbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00098a0 tx=0x7fd7e0009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 -- 172.21.15.5:0/616007445 shutdown_connections 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 --2- 172.21.15.5:0/616007445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f010cbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 -- 172.21.15.5:0/616007445 >> 172.21.15.5:0/616007445 conn(0x7fd7f0104290 msgr2=0x7fd7f01066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 -- 172.21.15.5:0/616007445 shutdown_connections 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 -- 172.21.15.5:0/616007445 wait complete. 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 Processor -- start 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.746+0000 7fd7f5826700 1 -- start start 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7f5826700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7f5826700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f010dfc0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7eeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7eeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41330/0 (socket says 172.21.15.5:41330) 2024-05-14T15:13:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7eeffd700 1 -- 172.21.15.5:0/950376558 learned_addr learned my addr 172.21.15.5:0/950376558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7eeffd700 1 -- 172.21.15.5:0/950376558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f019e5e0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7eeffd700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f019e0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00117b0 tx=0x7fd7e0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0003af0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.747+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd7e0020070 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.748+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f019e870 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.748+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e0019630 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.748+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a1480 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.749+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f0064e20 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.749+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd7e00198a0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.749+0000 7fd7f4824700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fd7dc03e4c0 0x7fd7dc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.749+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd7e0053ac0 con 0x7fd7f010a7d0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.749+0000 7fd7e67fc700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fd7dc03e4c0 0x7fd7dc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.750+0000 7fd7e67fc700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fd7dc03e4c0 0x7fd7dc0409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006fd0 tx=0x7fd7d8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:17.563 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.754+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd7e0016260 con 0x7fd7f010a7d0 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.976+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd7f0105510 con 0x7fd7dc03e4c0 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.977+0000 7fd7f4824700 1 -- 172.21.15.5:0/950376558 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd7f0105510 con 0x7fd7dc03e4c0 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fd7dc03e4c0 msgr2=0x7fd7dc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fd7dc03e4c0 0x7fd7dc0409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006fd0 tx=0x7fd7d8006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 msgr2=0x7fd7f019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f019e0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00117b0 tx=0x7fd7e0011b70 comp rx=0 tx=0).stop 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 shutdown_connections 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fd7dc03e4c0 0x7fd7dc0409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 --2- 172.21.15.5:0/950376558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7f010a7d0 0x7fd7f019e0a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.979+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 >> 172.21.15.5:0/950376558 conn(0x7fd7f0104290 msgr2=0x7fd7f0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.980+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 shutdown_connections 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:16.980+0000 7fd7f5826700 1 -- 172.21.15.5:0/950376558 wait complete. 2024-05-14T15:13:17.564 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-05-14T15:13:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:17 smithi005 ceph-mon[31559]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: Generating public/private rsa key pair. 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: Your identification has been saved in /tmp/tmpdgl55ote/key. 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: Your public key has been saved in /tmp/tmpdgl55ote/key.pub. 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: The key fingerprint is: 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: SHA256:ujBibGulYg3DoZzJ60sqtVkb1B2tNG7uVgmP5VTgyIc ceph-4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: The key's randomart image is: 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: +---[RSA 3072]----+ 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: | ... | 2024-05-14T15:13:18.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: | .++. . | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: | . +E+o. | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: | . . . *.o | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: |= +. oSB . | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: |.X. + .o = | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: | oX*oo.. . | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: |===+.o .o | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: |==o .. | 2024-05-14T15:13:18.563 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:18 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: +----[SHA256]-----+ 2024-05-14T15:13:19.120 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.024+0000 7f2927df0700 1 Processor -- start 2024-05-14T15:13:19.120 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2927df0700 1 -- start start 2024-05-14T15:13:19.120 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2927df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292010a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:19.120 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2927df0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f292010a770 con 0x7f2920109d70 2024-05-14T15:13:19.120 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2925b8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292010a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2925b8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292010a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41346/0 (socket says 172.21.15.5:41346) 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2925b8c700 1 -- 172.21.15.5:0/3329029623 learned_addr learned my addr 172.21.15.5:0/3329029623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.025+0000 7f2925b8c700 1 -- 172.21.15.5:0/3329029623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292010ae50 con 0x7f2920109d70 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.026+0000 7f2925b8c700 1 --2- 172.21.15.5:0/3329029623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292010a180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f291c0098a0 tx=0x7f291c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d570ebf5fb668289 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.026+0000 7f2924b8a700 1 -- 172.21.15.5:0/3329029623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c004160 con 0x7f2920109d70 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.026+0000 7f2924b8a700 1 -- 172.21.15.5:0/3329029623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f291c004300 con 0x7f2920109d70 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.026+0000 7f2924b8a700 1 -- 172.21.15.5:0/3329029623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c011af0 con 0x7f2920109d70 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.026+0000 7f2927df0700 1 -- 172.21.15.5:0/3329029623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 msgr2=0x7f292010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.026+0000 7f2927df0700 1 --2- 172.21.15.5:0/3329029623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292010a180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f291c0098a0 tx=0x7f291c009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 -- 172.21.15.5:0/3329029623 shutdown_connections 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 --2- 172.21.15.5:0/3329029623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292010a180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 -- 172.21.15.5:0/3329029623 >> 172.21.15.5:0/3329029623 conn(0x7f2920105450 msgr2=0x7f29201078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:19.121 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 -- 172.21.15.5:0/3329029623 shutdown_connections 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 -- 172.21.15.5:0/3329029623 wait complete. 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 Processor -- start 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.027+0000 7f2927df0700 1 -- start start 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2927df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2927df0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f292010e080 con 0x7f2920109d70 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2925b8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2925b8c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41354/0 (socket says 172.21.15.5:41354) 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2925b8c700 1 -- 172.21.15.5:0/3424136361 learned_addr learned my addr 172.21.15.5:0/3424136361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2925b8c700 1 -- 172.21.15.5:0/3424136361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292019e5f0 con 0x7f2920109d70 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2925b8c700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292019e0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f291c009fb0 tx=0x7f291c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c0036a0 con 0x7f2920109d70 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f291c003cd0 con 0x7f2920109d70 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292019e880 con 0x7f2920109d70 2024-05-14T15:13:19.122 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.028+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f291c01a430 con 0x7f2920109d70 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.029+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a1490 con 0x7f2920109d70 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.029+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920064e20 con 0x7f2920109d70 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.030+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f291c01a6f0 con 0x7f2920109d70 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.030+0000 7f2912ffd700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f290c03e420 0x7f290c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.030+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f291c0537f0 con 0x7f2920109d70 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.030+0000 7f292538b700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f290c03e420 0x7f290c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.030+0000 7f292538b700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f290c03e420 0x7f290c040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2914006fd0 tx=0x7f2914006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.035+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f291c02c350 con 0x7f2920109d70 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.257+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2920107660 con 0x7f290c03e420 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.472+0000 7f2912ffd700 1 -- 172.21.15.5:0/3424136361 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2920107660 con 0x7f290c03e420 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f290c03e420 msgr2=0x7f290c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f290c03e420 0x7f290c040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2914006fd0 tx=0x7f2914006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:19.123 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 msgr2=0x7f292019e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292019e0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f291c009fb0 tx=0x7f291c011890 comp rx=0 tx=0).stop 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 shutdown_connections 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f290c03e420 0x7f290c040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 --2- 172.21.15.5:0/3424136361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2920109d70 0x7f292019e0b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 >> 172.21.15.5:0/3424136361 conn(0x7f2920105450 msgr2=0x7f2920106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 shutdown_connections 2024-05-14T15:13:19.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:18.475+0000 7f2927df0700 1 -- 172.21.15.5:0/3424136361 wait complete. 2024-05-14T15:13:19.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:19 smithi005 ceph-mon[31559]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:19.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:19 smithi005 ceph-mon[31559]: Generating ssh key... 2024-05-14T15:13:19.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:19 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:19.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:19 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:20.318 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:13:20.318 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33d1411700 1 Processor -- start 2024-05-14T15:13:20.318 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33d1411700 1 -- start start 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33d1411700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc10b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33d1411700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc10bd20 con 0x7f33cc10b340 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc10b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc10b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41364/0 (socket says 172.21.15.5:41364) 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.496+0000 7f33caffd700 1 -- 172.21.15.5:0/3397615354 learned_addr learned my addr 172.21.15.5:0/3397615354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.497+0000 7f33caffd700 1 -- 172.21.15.5:0/3397615354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc10bea0 con 0x7f33cc10b340 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.497+0000 7f33caffd700 1 --2- 172.21.15.5:0/3397615354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc10b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0098a0 tx=0x7f33bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558ade3ec9e370b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.497+0000 7f33c9ffb700 1 -- 172.21.15.5:0/3397615354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc004160 con 0x7f33cc10b340 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.497+0000 7f33c9ffb700 1 -- 172.21.15.5:0/3397615354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f33bc004300 con 0x7f33cc10b340 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.497+0000 7f33c9ffb700 1 -- 172.21.15.5:0/3397615354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc011af0 con 0x7f33cc10b340 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 -- 172.21.15.5:0/3397615354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 msgr2=0x7f33cc10b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 --2- 172.21.15.5:0/3397615354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc10b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0098a0 tx=0x7f33bc009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 -- 172.21.15.5:0/3397615354 shutdown_connections 2024-05-14T15:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 --2- 172.21.15.5:0/3397615354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc10b750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 -- 172.21.15.5:0/3397615354 >> 172.21.15.5:0/3397615354 conn(0x7f33cc103b00 msgr2=0x7f33cc105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 -- 172.21.15.5:0/3397615354 shutdown_connections 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.498+0000 7f33d1411700 1 -- 172.21.15.5:0/3397615354 wait complete. 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33d1411700 1 Processor -- start 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33d1411700 1 -- start start 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33d1411700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33d1411700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc108b10 con 0x7f33cc10b340 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc19e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41376/0 (socket says 172.21.15.5:41376) 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33caffd700 1 -- 172.21.15.5:0/4219213996 learned_addr learned my addr 172.21.15.5:0/4219213996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.499+0000 7f33caffd700 1 -- 172.21.15.5:0/4219213996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc19e620 con 0x7f33cc10b340 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.500+0000 7f33caffd700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc19e0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009fb0 tx=0x7f33bc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.500+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc011bc0 con 0x7f33cc10b340 2024-05-14T15:13:20.320 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.500+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f33bc024440 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.500+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc19e8b0 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.500+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc01a5c0 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.500+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc108ea0 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.501+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc064e20 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.501+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f33bc01a880 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.501+0000 7f33b3fff700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f33b403e420 0x7f33b4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.501+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f33bc053820 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.501+0000 7f33ca7fc700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f33b403e420 0x7f33b4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.502+0000 7f33ca7fc700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f33b403e420 0x7f33b4040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006fd0 tx=0x7f33c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.507+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33bc00b3e0 con 0x7f33cc10b340 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.727+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f33cc105d90 con 0x7f33b403e420 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.728+0000 7f33b3fff700 1 -- 172.21.15.5:0/4219213996 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f33cc105d90 con 0x7f33b403e420 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f33b403e420 msgr2=0x7f33b4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f33b403e420 0x7f33b4040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006fd0 tx=0x7f33c0006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:20.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 msgr2=0x7f33cc19e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc19e0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009fb0 tx=0x7f33bc011890 comp rx=0 tx=0).stop 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 shutdown_connections 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f33b403e420 0x7f33b4040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 --2- 172.21.15.5:0/4219213996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33cc10b340 0x7f33cc19e0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 >> 172.21.15.5:0/4219213996 conn(0x7f33cc103b00 msgr2=0x7f33cc105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 shutdown_connections 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:19.730+0000 7f33d1411700 1 -- 172.21.15.5:0/4219213996 wait complete. 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-05-14T15:13:20.322 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-05-14T15:13:20.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:20 smithi005 ceph-mon[31559]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:21.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:21 smithi005 ceph-mon[31559]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:22.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:22 smithi005 ceph-mon[31559]: Deploying cephadm binary to smithi005 2024-05-14T15:13:25.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aae35f700 1 Processor -- start 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aae35f700 1 -- start start 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aae35f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa810a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aae35f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa810a770 con 0x7f2aa8109d70 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa810a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa810a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41388/0 (socket says 172.21.15.5:41388) 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.779+0000 7f2aa7fff700 1 -- 172.21.15.5:0/117647493 learned_addr learned my addr 172.21.15.5:0/117647493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:25.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.780+0000 7f2aa7fff700 1 -- 172.21.15.5:0/117647493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa810ae50 con 0x7f2aa8109d70 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.780+0000 7f2aa7fff700 1 --2- 172.21.15.5:0/117647493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa810a180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a980098a0 tx=0x7f2a98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a25acf256667f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.780+0000 7f2aa6ffd700 1 -- 172.21.15.5:0/117647493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a98004160 con 0x7f2aa8109d70 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.780+0000 7f2aa6ffd700 1 -- 172.21.15.5:0/117647493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a98004300 con 0x7f2aa8109d70 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.780+0000 7f2aa6ffd700 1 -- 172.21.15.5:0/117647493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a98011af0 con 0x7f2aa8109d70 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 -- 172.21.15.5:0/117647493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 msgr2=0x7f2aa810a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 --2- 172.21.15.5:0/117647493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa810a180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a980098a0 tx=0x7f2a98009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 -- 172.21.15.5:0/117647493 shutdown_connections 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 --2- 172.21.15.5:0/117647493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa810a180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 -- 172.21.15.5:0/117647493 >> 172.21.15.5:0/117647493 conn(0x7f2aa8105450 msgr2=0x7f2aa81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 -- 172.21.15.5:0/117647493 shutdown_connections 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.781+0000 7f2aae35f700 1 -- 172.21.15.5:0/117647493 wait complete. 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aae35f700 1 Processor -- start 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aae35f700 1 -- start start 2024-05-14T15:13:25.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aae35f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aae35f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa810e080 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aa7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41404/0 (socket says 172.21.15.5:41404) 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.782+0000 7f2aa7fff700 1 -- 172.21.15.5:0/2261847926 learned_addr learned my addr 172.21.15.5:0/2261847926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aa7fff700 1 -- 172.21.15.5:0/2261847926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa819e5f0 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aa7fff700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa819e0b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a98011890 tx=0x7f2a98011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a980039e0 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a98003b80 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a980246e0 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa819e880 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.783+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa81a1490 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.784+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa8064e20 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.784+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2a98022020 con 0x7f2aa8109d70 2024-05-14T15:13:25.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.785+0000 7f2aa57fa700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f2a9003e4c0 0x7f2a900409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.785+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a98053710 con 0x7f2aa8109d70 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.785+0000 7f2aa77fe700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f2a9003e4c0 0x7f2a900409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.785+0000 7f2aa77fe700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f2a9003e4c0 0x7f2a900409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006fd0 tx=0x7f2a9c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:20.790+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a980172b0 con 0x7f2aa8109d70 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:21.011+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7f2aa8107660 con 0x7f2a9003e4c0 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.297+0000 7f2aa57fa700 1 -- 172.21.15.5:0/2261847926 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f2aa8107660 con 0x7f2a9003e4c0 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f2a9003e4c0 msgr2=0x7f2a900409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f2a9003e4c0 0x7f2a900409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006fd0 tx=0x7f2a9c006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 msgr2=0x7f2aa819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa819e0b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a98011890 tx=0x7f2a98011970 comp rx=0 tx=0).stop 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 shutdown_connections 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f2a9003e4c0 0x7f2a900409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 --2- 172.21.15.5:0/2261847926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8109d70 0x7f2aa819e0b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:25.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.300+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 >> 172.21.15.5:0/2261847926 conn(0x7f2aa8105450 msgr2=0x7f2aa8106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:25.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.301+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 shutdown_connections 2024-05-14T15:13:25.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:25.301+0000 7f2aae35f700 1 -- 172.21.15.5:0/2261847926 wait complete. 2024-05-14T15:13:25.855 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-05-14T15:13:26.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:26 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:26.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:26 smithi005 ceph-mon[31559]: Added host smithi005 2024-05-14T15:13:26.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:26 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.289+0000 7fe1c0c93700 1 Processor -- start 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.289+0000 7fe1c0c93700 1 -- start start 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.289+0000 7fe1c0c93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc10a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.289+0000 7fe1c0c93700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10aba0 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc10a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc10a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43768/0 (socket says 172.21.15.5:43768) 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1ba59c700 1 -- 172.21.15.5:0/2649777164 learned_addr learned my addr 172.21.15.5:0/2649777164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1ba59c700 1 -- 172.21.15.5:0/2649777164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc108ec0 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1ba59c700 1 --2- 172.21.15.5:0/2649777164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc10a5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0098a0 tx=0x7fe1ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1475a8ed9f74b3ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1b959a700 1 -- 172.21.15.5:0/2649777164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ac004160 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1b959a700 1 -- 172.21.15.5:0/2649777164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe1ac004300 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.290+0000 7fe1b959a700 1 -- 172.21.15.5:0/2649777164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ac011af0 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 -- 172.21.15.5:0/2649777164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 msgr2=0x7fe1bc10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 --2- 172.21.15.5:0/2649777164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc10a5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0098a0 tx=0x7fe1ac009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 -- 172.21.15.5:0/2649777164 shutdown_connections 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 --2- 172.21.15.5:0/2649777164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc10a5b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 -- 172.21.15.5:0/2649777164 >> 172.21.15.5:0/2649777164 conn(0x7fe1bc103b20 msgr2=0x7fe1bc105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 -- 172.21.15.5:0/2649777164 shutdown_connections 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.291+0000 7fe1c0c93700 1 -- 172.21.15.5:0/2649777164 wait complete. 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.292+0000 7fe1c0c93700 1 Processor -- start 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.292+0000 7fe1c0c93700 1 -- start start 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1c0c93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:27.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1c0c93700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10ad20 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc19cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43782/0 (socket says 172.21.15.5:43782) 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ba59c700 1 -- 172.21.15.5:0/3563912849 learned_addr learned my addr 172.21.15.5:0/3563912849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ba59c700 1 -- 172.21.15.5:0/3563912849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc19d4d0 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ba59c700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc19cf90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009fb0 tx=0x7fe1ac011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ac0036a0 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe1ac024440 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc19d760 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ac01a5c0 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.293+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a0370 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.294+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe1ac01a830 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.294+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc064e20 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.295+0000 7fe1ab7fe700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fe1a403e470 0x7fe1a4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:27.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.295+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1ac053730 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.295+0000 7fe1b9d9b700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fe1a403e470 0x7fe1a4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.295+0000 7fe1b9d9b700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fe1a403e470 0x7fe1a4040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006fd0 tx=0x7fe1b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.300+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1ac017e90 con 0x7fe1bc10a1a0 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.534+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe1bc105cb0 con 0x7fe1a403e470 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.539+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3563912849 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe1bc105cb0 con 0x7fe1a403e470 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fe1a403e470 msgr2=0x7fe1a4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:27.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fe1a403e470 0x7fe1a4040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006fd0 tx=0x7fe1b0006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 msgr2=0x7fe1bc19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc19cf90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009fb0 tx=0x7fe1ac011890 comp rx=0 tx=0).stop 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 shutdown_connections 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fe1a403e470 0x7fe1a4040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 --2- 172.21.15.5:0/3563912849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1bc10a1a0 0x7fe1bc19cf90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 >> 172.21.15.5:0/3563912849 conn(0x7fe1bc103b20 msgr2=0x7fe1bc105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 shutdown_connections 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:26.541+0000 7fe1c0c93700 1 -- 172.21.15.5:0/3563912849 wait complete. 2024-05-14T15:13:27.291 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-05-14T15:13:27.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:27 smithi005 ceph-mon[31559]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:27.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:27 smithi005 ceph-mon[31559]: Saving service mon spec with placement count:5 2024-05-14T15:13:27.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:27 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:29.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:29 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:29.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:29 smithi005 ceph-mon[31559]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:29.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:29 smithi005 ceph-mon[31559]: Saving service mgr spec with placement count:2 2024-05-14T15:13:29.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:29 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:29.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:29 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:29.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-14T15:13:29.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.371+0000 7fa6ab2e4700 1 Processor -- start 2024-05-14T15:13:29.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.371+0000 7fa6ab2e4700 1 -- start start 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.372+0000 7fa6ab2e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.372+0000 7fa6ab2e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a407cdd0 con 0x7fa6a410a7d0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.372+0000 7fa6a9080700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.372+0000 7fa6a9080700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a410cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43788/0 (socket says 172.21.15.5:43788) 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.372+0000 7fa6a9080700 1 -- 172.21.15.5:0/27579831 learned_addr learned my addr 172.21.15.5:0/27579831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.373+0000 7fa6a9080700 1 -- 172.21.15.5:0/27579831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a410d130 con 0x7fa6a410a7d0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.373+0000 7fa6a9080700 1 --2- 172.21.15.5:0/27579831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a410cbf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00098a0 tx=0x7fa6a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43406f7069a7d1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.373+0000 7fa69bfff700 1 -- 172.21.15.5:0/27579831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0004160 con 0x7fa6a410a7d0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.373+0000 7fa69bfff700 1 -- 172.21.15.5:0/27579831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a0004300 con 0x7fa6a410a7d0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.373+0000 7fa69bfff700 1 -- 172.21.15.5:0/27579831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0011a20 con 0x7fa6a410a7d0 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/27579831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 msgr2=0x7fa6a410cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 --2- 172.21.15.5:0/27579831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a410cbf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00098a0 tx=0x7fa6a0009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:29.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/27579831 shutdown_connections 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 --2- 172.21.15.5:0/27579831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a410cbf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/27579831 >> 172.21.15.5:0/27579831 conn(0x7fa6a4104290 msgr2=0x7fa6a41066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/27579831 shutdown_connections 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.374+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/27579831 wait complete. 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6ab2e4700 1 Processor -- start 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6ab2e4700 1 -- start start 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6ab2e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a4109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6ab2e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a410e070 con 0x7fa6a410a7d0 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6a9080700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a4109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6a9080700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a4109f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43796/0 (socket says 172.21.15.5:43796) 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.375+0000 7fa6a9080700 1 -- 172.21.15.5:0/2814949400 learned_addr learned my addr 172.21.15.5:0/2814949400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa6a9080700 1 -- 172.21.15.5:0/2814949400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a410a490 con 0x7fa6a410a7d0 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa6a9080700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a4109f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0000c00 tx=0x7fa6a0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0003aa0 con 0x7fa6a410a7d0 2024-05-14T15:13:29.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a001b8f0 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a41086e0 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6a0023890 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.376+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a4108bc0 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.377+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a4064e20 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.377+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa6a002c3f0 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.378+0000 7fa69a7fc700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fa69003e4c0 0x7fa6900409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.378+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6a0053e10 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.378+0000 7fa6a887f700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fa69003e4c0 0x7fa6900409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.378+0000 7fa6a887f700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fa69003e4c0 0x7fa6900409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa694006fd0 tx=0x7fa694006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.383+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6a001f030 con 0x7fa6a410a7d0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.616+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa6a4109320 con 0x7fa69003e4c0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.621+0000 7fa69a7fc700 1 -- 172.21.15.5:0/2814949400 <== mgr.14118 v2:172.21.15.5:6800/2154399460 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa6a4109320 con 0x7fa69003e4c0 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fa69003e4c0 msgr2=0x7fa6900409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:29.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fa69003e4c0 0x7fa6900409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa694006fd0 tx=0x7fa694006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 msgr2=0x7fa6a4109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a4109f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0000c00 tx=0x7fa6a0011aa0 comp rx=0 tx=0).stop 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 shutdown_connections 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7fa69003e4c0 0x7fa6900409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 --2- 172.21.15.5:0/2814949400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a410a7d0 0x7fa6a4109f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 >> 172.21.15.5:0/2814949400 conn(0x7fa6a4104290 msgr2=0x7fa6a4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 shutdown_connections 2024-05-14T15:13:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:28.623+0000 7fa6ab2e4700 1 -- 172.21.15.5:0/2814949400 wait complete. 2024-05-14T15:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95c5d4d700 1 Processor -- start 2024-05-14T15:13:30.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95c5d4d700 1 -- start start 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95c5d4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c0109030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95c5d4d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0109620 con 0x7f95c0108c20 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c0109030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c0109030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43806/0 (socket says 172.21.15.5:43806) 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.656+0000 7f95bf7fe700 1 -- 172.21.15.5:0/3665163597 learned_addr learned my addr 172.21.15.5:0/3665163597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.657+0000 7f95bf7fe700 1 -- 172.21.15.5:0/3665163597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c0109d50 con 0x7f95c0108c20 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.657+0000 7f95bf7fe700 1 --2- 172.21.15.5:0/3665163597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c0109030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95b00098a0 tx=0x7f95b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6163682005062564 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.657+0000 7f95be7fc700 1 -- 172.21.15.5:0/3665163597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b0004160 con 0x7f95c0108c20 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.657+0000 7f95be7fc700 1 -- 172.21.15.5:0/3665163597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b0004300 con 0x7f95c0108c20 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.657+0000 7f95be7fc700 1 -- 172.21.15.5:0/3665163597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b0011a20 con 0x7f95c0108c20 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 -- 172.21.15.5:0/3665163597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 msgr2=0x7f95c0109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 --2- 172.21.15.5:0/3665163597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c0109030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95b00098a0 tx=0x7f95b0009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:30.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 -- 172.21.15.5:0/3665163597 shutdown_connections 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 --2- 172.21.15.5:0/3665163597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c0109030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 -- 172.21.15.5:0/3665163597 >> 172.21.15.5:0/3665163597 conn(0x7f95c0104280 msgr2=0x7f95c0106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 -- 172.21.15.5:0/3665163597 shutdown_connections 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.658+0000 7f95c5d4d700 1 -- 172.21.15.5:0/3665163597 wait complete. 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95c5d4d700 1 Processor -- start 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95c5d4d700 1 -- start start 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95c5d4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95c5d4d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95c010d100 con 0x7f95c0108c20 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c007d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43820/0 (socket says 172.21.15.5:43820) 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.659+0000 7f95bf7fe700 1 -- 172.21.15.5:0/1114558540 learned_addr learned my addr 172.21.15.5:0/1114558540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95bf7fe700 1 -- 172.21.15.5:0/1114558540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c007d8d0 con 0x7f95c0108c20 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95bf7fe700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c007d390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95b0003e70 tx=0x7f95b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b0003aa0 con 0x7f95c0108c20 2024-05-14T15:13:30.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c01a0e20 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b001a8e0 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95b0024870 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.660+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c01a12a0 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.661+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f95b0022020 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.661+0000 7f95bcff9700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f95a803e4c0 0x7f95a80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.661+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95b0053940 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.661+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c0064e20 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.662+0000 7f95beffd700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f95a803e4c0 0x7f95a80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.662+0000 7f95beffd700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f95a803e4c0 0x7f95a80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f95b4006fd0 tx=0x7f95b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.667+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95b0029df0 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.975+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f95c01a1520 con 0x7f95c0108c20 2024-05-14T15:13:30.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.978+0000 7f95bcff9700 1 -- 172.21.15.5:0/1114558540 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f95b001ad20 con 0x7f95c0108c20 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f95a803e4c0 msgr2=0x7f95a80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f95a803e4c0 0x7f95a80409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f95b4006fd0 tx=0x7f95b4006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 msgr2=0x7f95c007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c007d390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95b0003e70 tx=0x7f95b0011aa0 comp rx=0 tx=0).stop 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 shutdown_connections 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f95a803e4c0 0x7f95a80409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 --2- 172.21.15.5:0/1114558540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c0108c20 0x7f95c007d390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.981+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 >> 172.21.15.5:0/1114558540 conn(0x7f95c0104280 msgr2=0x7f95c0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.982+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 shutdown_connections 2024-05-14T15:13:30.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:29.982+0000 7f95c5d4d700 1 -- 172.21.15.5:0/1114558540 wait complete. 2024-05-14T15:13:31.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:30 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1114558540' entity='client.admin' 2024-05-14T15:13:31.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:30 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:32.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f5982a55700 1 Processor -- start 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f5982a55700 1 -- start start 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f5982a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c10c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f5982a55700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10ce70 con 0x7f597c07cc40 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f597bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c10c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f597bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c10c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43834/0 (socket says 172.21.15.5:43834) 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.403+0000 7f597bfff700 1 -- 172.21.15.5:0/4020297563 learned_addr learned my addr 172.21.15.5:0/4020297563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.404+0000 7f597bfff700 1 -- 172.21.15.5:0/4020297563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c10cff0 con 0x7f597c07cc40 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.404+0000 7f597bfff700 1 --2- 172.21.15.5:0/4020297563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c10c8a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098a0 tx=0x7f596c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ae9db0e92b8115 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.404+0000 7f597b7fe700 1 -- 172.21.15.5:0/4020297563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c004160 con 0x7f597c07cc40 2024-05-14T15:13:32.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.404+0000 7f597b7fe700 1 -- 172.21.15.5:0/4020297563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c004300 con 0x7f597c07cc40 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.404+0000 7f597b7fe700 1 -- 172.21.15.5:0/4020297563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c011a20 con 0x7f597c07cc40 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 -- 172.21.15.5:0/4020297563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 msgr2=0x7f597c10c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 --2- 172.21.15.5:0/4020297563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c10c8a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098a0 tx=0x7f596c009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 -- 172.21.15.5:0/4020297563 shutdown_connections 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 --2- 172.21.15.5:0/4020297563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c10c8a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 -- 172.21.15.5:0/4020297563 >> 172.21.15.5:0/4020297563 conn(0x7f597c104290 msgr2=0x7f597c1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 -- 172.21.15.5:0/4020297563 shutdown_connections 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.405+0000 7f5982a55700 1 -- 172.21.15.5:0/4020297563 wait complete. 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f5982a55700 1 Processor -- start 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f5982a55700 1 -- start start 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f5982a55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f5982a55700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10e0b0 con 0x7f597c07cc40 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f597bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:32.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f597bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c19e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43842/0 (socket says 172.21.15.5:43842) 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f597bfff700 1 -- 172.21.15.5:0/2407940440 learned_addr learned my addr 172.21.15.5:0/2407940440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.406+0000 7f597bfff700 1 -- 172.21.15.5:0/2407940440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c19e580 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.407+0000 7f597bfff700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c19e040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f596c0119f0 tx=0x7f596c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.407+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c0039e0 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.407+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596c01b8f0 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.407+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c19e810 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.407+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f596c0237a0 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.407+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a1420 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.408+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c064e20 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.408+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f596c023aa0 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.408+0000 7f5979ffb700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f596803e420 0x7f5968040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.408+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f596c053ed0 con 0x7f597c07cc40 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.408+0000 7f5973fff700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f596803e420 0x7f5968040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:32.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.409+0000 7f5973fff700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f596803e420 0x7f5968040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5964006fd0 tx=0x7f5964006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.414+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f596c01f030 con 0x7f597c07cc40 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.630+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f597c1a15f0 con 0x7f597c07cc40 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.633+0000 7f5979ffb700 1 -- 172.21.15.5:0/2407940440 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f596c019e20 con 0x7f597c07cc40 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f596803e420 msgr2=0x7f5968040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f596803e420 0x7f5968040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5964006fd0 tx=0x7f5964006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 msgr2=0x7f597c19e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c19e040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f596c0119f0 tx=0x7f596c011ad0 comp rx=0 tx=0).stop 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 shutdown_connections 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f596803e420 0x7f5968040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 --2- 172.21.15.5:0/2407940440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c07cc40 0x7f597c19e040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 >> 172.21.15.5:0/2407940440 conn(0x7f597c104290 msgr2=0x7f597c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 shutdown_connections 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:31.636+0000 7f5982a55700 1 -- 172.21.15.5:0/2407940440 wait complete. 2024-05-14T15:13:32.248 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-05-14T15:13:32.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:32 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2407940440' entity='client.admin' 2024-05-14T15:13:33.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:33 smithi005 ceph-mon[31559]: from='mgr.14118 172.21.15.5:0/3746158822' entity='mgr.x' 2024-05-14T15:13:33.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:33 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4207052260' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-14T15:13:33.967 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:33 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: ignoring --setuser ceph since I am not root 2024-05-14T15:13:33.967 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:33 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: ignoring --setgroup ceph since I am not root 2024-05-14T15:13:33.967 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:33 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:33.970+0000 7f05af614200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:13:34.274 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:34.167+0000 7f05af614200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:13:34.274 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:34.276+0000 7f05af614200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:13:34.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:34 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4207052260' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-14T15:13:34.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:34 smithi005 ceph-mon[31559]: mgrmap e9: x(active, since 20s) 2024-05-14T15:13:34.803 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:34.608+0000 7f05af614200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:13:34.804 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:34.702+0000 7f05af614200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:13:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.649+0000 7ffa98f9d700 1 Processor -- start 2024-05-14T15:13:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.649+0000 7ffa98f9d700 1 -- start start 2024-05-14T15:13:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa98f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa9410ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa98f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9410efb0 con 0x7ffa9407cc80 2024-05-14T15:13:34.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa9410ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa9410ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43854/0 (socket says 172.21.15.5:43854) 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa937fe700 1 -- 172.21.15.5:0/3834690718 learned_addr learned my addr 172.21.15.5:0/3834690718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa937fe700 1 -- 172.21.15.5:0/3834690718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9410f130 con 0x7ffa9407cc80 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa937fe700 1 --2- 172.21.15.5:0/3834690718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa9410ea70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa840098a0 tx=0x7ffa84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0ad1062e1887e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.650+0000 7ffa927fc700 1 -- 172.21.15.5:0/3834690718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa84004160 con 0x7ffa9407cc80 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa927fc700 1 -- 172.21.15.5:0/3834690718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa84004300 con 0x7ffa9407cc80 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa927fc700 1 -- 172.21.15.5:0/3834690718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa84011a20 con 0x7ffa9407cc80 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 -- 172.21.15.5:0/3834690718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 msgr2=0x7ffa9410ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 --2- 172.21.15.5:0/3834690718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa9410ea70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa840098a0 tx=0x7ffa84009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 -- 172.21.15.5:0/3834690718 shutdown_connections 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 --2- 172.21.15.5:0/3834690718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa9410ea70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 -- 172.21.15.5:0/3834690718 >> 172.21.15.5:0/3834690718 conn(0x7ffa941040e0 msgr2=0x7ffa94106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 -- 172.21.15.5:0/3834690718 shutdown_connections 2024-05-14T15:13:34.812 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.651+0000 7ffa98f9d700 1 -- 172.21.15.5:0/3834690718 wait complete. 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.652+0000 7ffa98f9d700 1 Processor -- start 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.652+0000 7ffa98f9d700 1 -- start start 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.652+0000 7ffa98f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa941a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.652+0000 7ffa98f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94110080 con 0x7ffa9407cc80 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa941a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa941a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43858/0 (socket says 172.21.15.5:43858) 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa937fe700 1 -- 172.21.15.5:0/4207052260 learned_addr learned my addr 172.21.15.5:0/4207052260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa937fe700 1 -- 172.21.15.5:0/4207052260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa941a0580 con 0x7ffa9407cc80 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa937fe700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa941a0040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009520 tx=0x7ffa84011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa84003b70 con 0x7ffa9407cc80 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa8401b8f0 con 0x7ffa9407cc80 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa941a0810 con 0x7ffa9407cc80 2024-05-14T15:13:34.813 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa84019800 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.653+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa941a3420 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.654+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa94064e20 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.654+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ffa84019af0 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.655+0000 7ffa90ff9700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7ffa8003e470 0x7ffa80040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.655+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa84054060 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.655+0000 7ffa92ffd700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7ffa8003e470 0x7ffa80040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.655+0000 7ffa92ffd700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7ffa8003e470 0x7ffa80040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c006fd0 tx=0x7ffa7c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.660+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa8401f030 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:32.962+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ffa941a35f0 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.751+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ffa8401baa0 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.753+0000 7ffa90ff9700 1 -- 172.21.15.5:0/4207052260 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7ffa8402a350 con 0x7ffa9407cc80 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7ffa8003e470 msgr2=0x7ffa80040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:34.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7ffa8003e470 0x7ffa80040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c006fd0 tx=0x7ffa7c006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 msgr2=0x7ffa941a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa941a0040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009520 tx=0x7ffa84011b10 comp rx=0 tx=0).stop 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 shutdown_connections 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7ffa8003e470 0x7ffa80040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 --2- 172.21.15.5:0/4207052260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa9407cc80 0x7ffa941a0040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 >> 172.21.15.5:0/4207052260 conn(0x7ffa941040e0 msgr2=0x7ffa94105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 shutdown_connections 2024-05-14T15:13:34.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:33.754+0000 7ffa98f9d700 1 -- 172.21.15.5:0/4207052260 wait complete. 2024-05-14T15:13:36.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4038871793' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T15:13:36.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:13:36.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-14T15:13:36.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T15:13:36.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.291+0000 7f608c649700 1 Processor -- start 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.291+0000 7f608c649700 1 -- start start 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608c649700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608410c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608c649700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f608410ce30 con 0x7f608407cca0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608a3e5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608410c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608a3e5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608410c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43884/0 (socket says 172.21.15.5:43884) 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608a3e5700 1 -- 172.21.15.5:0/1054757288 learned_addr learned my addr 172.21.15.5:0/1054757288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608a3e5700 1 -- 172.21.15.5:0/1054757288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608410cfb0 con 0x7f608407cca0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f608a3e5700 1 --2- 172.21.15.5:0/1054757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608410c840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60780098a0 tx=0x7f6078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc9a233f64c4cd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.292+0000 7f60893e3700 1 -- 172.21.15.5:0/1054757288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6078004160 con 0x7f608407cca0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.293+0000 7f60893e3700 1 -- 172.21.15.5:0/1054757288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6078004300 con 0x7f608407cca0 2024-05-14T15:13:36.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.293+0000 7f60893e3700 1 -- 172.21.15.5:0/1054757288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6078011a20 con 0x7f608407cca0 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.293+0000 7f608c649700 1 -- 172.21.15.5:0/1054757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 msgr2=0x7f608410c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.293+0000 7f608c649700 1 --2- 172.21.15.5:0/1054757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608410c840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f60780098a0 tx=0x7f6078009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 -- 172.21.15.5:0/1054757288 shutdown_connections 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 --2- 172.21.15.5:0/1054757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608410c840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 -- 172.21.15.5:0/1054757288 >> 172.21.15.5:0/1054757288 conn(0x7f6084104290 msgr2=0x7f60841066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 -- 172.21.15.5:0/1054757288 shutdown_connections 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 -- 172.21.15.5:0/1054757288 wait complete. 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 Processor -- start 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.294+0000 7f608c649700 1 -- start start 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608c649700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608c649700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f608410e070 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608a3e5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608a3e5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608419e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43896/0 (socket says 172.21.15.5:43896) 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608a3e5700 1 -- 172.21.15.5:0/4038871793 learned_addr learned my addr 172.21.15.5:0/4038871793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608a3e5700 1 -- 172.21.15.5:0/4038871793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608419e5c0 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608a3e5700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608419e080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60780119f0 tx=0x7f6078011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60780039e0 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f607801b8f0 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.295+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608419e850 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.296+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60780237a0 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.296+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60841a1460 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.296+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f607802c3f0 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.297+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6084064e20 con 0x7f608407cca0 2024-05-14T15:13:36.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.297+0000 7f60777fe700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 0x7f6070040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.297+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f607801b440 con 0x7f608407cca0 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.297+0000 7f6089be4700 1 -- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 msgr2=0x7f6070040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.297+0000 7f6089be4700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 0x7f6070040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.302+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f607801f030 con 0x7f608407cca0 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.498+0000 7f6089be4700 1 -- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 msgr2=0x7f6070040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.498+0000 7f6089be4700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 0x7f6070040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.594+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f60841a1630 con 0x7f608407cca0 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.594+0000 7f60777fe700 1 -- 172.21.15.5:0/4038871793 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6078056020 con 0x7f608407cca0 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 msgr2=0x7f6070040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 0x7f6070040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 msgr2=0x7f608419e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608419e080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60780119f0 tx=0x7f6078011ad0 comp rx=0 tx=0).stop 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 shutdown_connections 2024-05-14T15:13:36.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f607003e470 0x7f6070040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:36.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 --2- 172.21.15.5:0/4038871793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f608407cca0 0x7f608419e080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:36.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 >> 172.21.15.5:0/4038871793 conn(0x7f6084104290 msgr2=0x7f6084104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:36.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 shutdown_connections 2024-05-14T15:13:36.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:35.596+0000 7f608c649700 1 -- 172.21.15.5:0/4038871793 wait complete. 2024-05-14T15:13:36.216 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-05-14T15:13:36.216 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-05-14T15:13:36.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:36.010+0000 7f05af614200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:13:36.312 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:36.193+0000 7f05af614200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:13:36.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:36.380+0000 7f05af614200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:13:37.009 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:36.698+0000 7f05af614200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:13:37.009 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:36.797+0000 7f05af614200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:13:37.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:37 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:37.011+0000 7f05af614200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:13:37.312 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:37 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:37.218+0000 7f05af614200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:13:38.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:37 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:37.857+0000 7f05af614200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:13:38.312 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:38.093+0000 7f05af614200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:13:38.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:38.551+0000 7f05af614200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:13:38.812 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:38.655+0000 7f05af614200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:13:39.311 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:39.046+0000 7f05af614200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:13:39.786 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:39.425+0000 7f05af614200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:13:39.786 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:39.526+0000 7f05af614200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:13:39.786 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:39.788+0000 7f05af614200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:13:40.061 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:39.932+0000 7f05af614200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:13:40.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:40.036+0000 7f05af614200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:13:40.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:40.390+0000 7f05af614200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:13:40.562 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:40.489+0000 7f05af614200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:13:41.415 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:41.131+0000 7f05af614200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:13:41.416 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:13:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:13:41.365+0000 7f05af614200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: Active manager daemon x restarted 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: Activating manager daemon x 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: osdmap e3: 0 total, 0 up, 0 in 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: mgrmap e10: x(active, starting, since 0.00413413s) 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: Manager daemon x is now available 2024-05-14T15:13:41.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:42.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T15:13:42.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T15:13:42.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:42 smithi005 ceph-mon[31559]: mgrmap e11: x(active, since 1.00666s) 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.614+0000 7f5e6cf29700 1 Processor -- start 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.614+0000 7f5e6cf29700 1 -- start start 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.615+0000 7f5e6cf29700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e68109000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.615+0000 7f5e6cf29700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681095f0 con 0x7f5e68108bf0 2024-05-14T15:13:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.615+0000 7f5e6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e68109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.615+0000 7f5e6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e68109000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36884/0 (socket says 172.21.15.5:36884) 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.615+0000 7f5e6659c700 1 -- 172.21.15.5:0/2696480596 learned_addr learned my addr 172.21.15.5:0/2696480596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.615+0000 7f5e6659c700 1 -- 172.21.15.5:0/2696480596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e68109d20 con 0x7f5e68108bf0 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.616+0000 7f5e6659c700 1 --2- 172.21.15.5:0/2696480596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e68109000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800bb40 tx=0x7f5e5800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecd79f5ccd1fa9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.616+0000 7f5e6559a700 1 -- 172.21.15.5:0/2696480596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e5800dbe0 con 0x7f5e68108bf0 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.616+0000 7f5e6559a700 1 -- 172.21.15.5:0/2696480596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e580044e0 con 0x7f5e68108bf0 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.616+0000 7f5e6559a700 1 -- 172.21.15.5:0/2696480596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e58010470 con 0x7f5e68108bf0 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2696480596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 msgr2=0x7f5e68109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2696480596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e68109000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800bb40 tx=0x7f5e5800be50 comp rx=0 tx=0).stop 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2696480596 shutdown_connections 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2696480596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e68109000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2696480596 >> 172.21.15.5:0/2696480596 conn(0x7f5e68104290 msgr2=0x7f5e681066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2696480596 shutdown_connections 2024-05-14T15:13:43.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.617+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2696480596 wait complete. 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.618+0000 7f5e6cf29700 1 Processor -- start 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.618+0000 7f5e6cf29700 1 -- start start 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.618+0000 7f5e6cf29700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e6819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.618+0000 7f5e6cf29700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6807c140 con 0x7f5e68108bf0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e6819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e6819be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36898/0 (socket says 172.21.15.5:36898) 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6659c700 1 -- 172.21.15.5:0/2098482373 learned_addr learned my addr 172.21.15.5:0/2098482373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6659c700 1 -- 172.21.15.5:0/2098482373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6819c3d0 con 0x7f5e68108bf0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6659c700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e6819be90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800bb10 tx=0x7f5e58003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e58027070 con 0x7f5e68108bf0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6819c660 con 0x7f5e68108bf0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e58010c20 con 0x7f5e68108bf0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e58022b70 con 0x7f5e68108bf0 2024-05-14T15:13:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.619+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6819f270 con 0x7f5e68108bf0 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.621+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f5e5802a070 con 0x7f5e68108bf0 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.621+0000 7f5e5f7fe700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.621+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 --> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e54041090 con 0x7f5e5403e470 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.621+0000 7f5e65d9b700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 msgr2=0x7f5e54040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.621+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e5801f070 con 0x7f5e68108bf0 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.621+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.822+0000 7f5e65d9b700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 msgr2=0x7f5e54040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:36.822+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:37.223+0000 7f5e65d9b700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 msgr2=0x7f5e54040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:37.223+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:38.024+0000 7f5e65d9b700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 msgr2=0x7f5e54040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:38.024+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:39.625+0000 7f5e65d9b700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 msgr2=0x7f5e54040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2154399460 2024-05-14T15:13:43.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:39.625+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:41.369+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f5e58022410 con 0x7f5e68108bf0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:41.369+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 msgr2=0x7f5e54040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:41.369+0000 7f5e5f7fe700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.371+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f5e58057d60 con 0x7f5e68108bf0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.371+0000 7f5e5f7fe700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5e580558d0 0x7f5e54043f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.371+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e5801ce50 con 0x7f5e580558d0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.371+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5e580558d0 0x7f5e54043f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.372+0000 7f5e65d9b700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5e580558d0 0x7f5e54043f70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5e50003a10 tx=0x7f5e500092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.373+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5e5801ce50 con 0x7f5e580558d0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.381+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5e68064e20 con 0x7f5e580558d0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.381+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/2098482373 <== mgr.14150 v2:172.21.15.5:6800/887023568 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f5e68064e20 con 0x7f5e580558d0 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.381+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5e580558d0 msgr2=0x7f5e54043f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.381+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5e580558d0 0x7f5e54043f70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5e50003a10 tx=0x7f5e500092b0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 msgr2=0x7f5e6819be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:43.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e6819be90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800bb10 tx=0x7f5e58003dc0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 shutdown_connections 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5e580558d0 0x7f5e54043f70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68108bf0 0x7f5e6819be90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 --2- 172.21.15.5:0/2098482373 >> [v2:172.21.15.5:6800/2154399460,v1:172.21.15.5:6801/2154399460] conn(0x7f5e5403e470 0x7f5e54040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 >> 172.21.15.5:0/2098482373 conn(0x7f5e68104290 msgr2=0x7f5e68192160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 shutdown_connections 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:42.382+0000 7f5e6cf29700 1 -- 172.21.15.5:0/2098482373 wait complete. 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-05-14T15:13:43.159 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-05-14T15:13:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: [14/May/2024:15:13:42] ENGINE Bus STARTING 2024-05-14T15:13:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: [14/May/2024:15:13:42] ENGINE Serving on http://172.21.15.5:8765 2024-05-14T15:13:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: [14/May/2024:15:13:42] ENGINE Serving on https://172.21.15.5:7150 2024-05-14T15:13:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: [14/May/2024:15:13:42] ENGINE Bus STARTED 2024-05-14T15:13:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T15:13:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T15:13:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:44.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:44 smithi005 ceph-mon[31559]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:44.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:44 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:44.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:44 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:44.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:44 smithi005 ceph-mon[31559]: mgrmap e12: x(active, since 2s) 2024-05-14T15:13:44.491 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-14T15:13:44.491 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.591+0000 7f8e48acd700 1 Processor -- start 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.591+0000 7f8e48acd700 1 -- start start 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.592+0000 7f8e48acd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.592+0000 7f8e48acd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4407cdd0 con 0x7f8e4410a7d0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.592+0000 7f8e4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.592+0000 7f8e4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4410cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36990/0 (socket says 172.21.15.5:36990) 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.592+0000 7f8e4259c700 1 -- 172.21.15.5:0/4085517556 learned_addr learned my addr 172.21.15.5:0/4085517556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.592+0000 7f8e4259c700 1 -- 172.21.15.5:0/4085517556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4410d130 con 0x7f8e4410a7d0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.593+0000 7f8e4259c700 1 --2- 172.21.15.5:0/4085517556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4410cbf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e340098a0 tx=0x7f8e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1bea6d7d093348 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.593+0000 7f8e4159a700 1 -- 172.21.15.5:0/4085517556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e34004160 con 0x7f8e4410a7d0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.593+0000 7f8e4159a700 1 -- 172.21.15.5:0/4085517556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e34004300 con 0x7f8e4410a7d0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.593+0000 7f8e4159a700 1 -- 172.21.15.5:0/4085517556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e34011a20 con 0x7f8e4410a7d0 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 -- 172.21.15.5:0/4085517556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 msgr2=0x7f8e4410cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 --2- 172.21.15.5:0/4085517556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4410cbf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e340098a0 tx=0x7f8e34009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 -- 172.21.15.5:0/4085517556 shutdown_connections 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 --2- 172.21.15.5:0/4085517556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4410cbf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 -- 172.21.15.5:0/4085517556 >> 172.21.15.5:0/4085517556 conn(0x7f8e44104290 msgr2=0x7f8e441066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 -- 172.21.15.5:0/4085517556 shutdown_connections 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.594+0000 7f8e48acd700 1 -- 172.21.15.5:0/4085517556 wait complete. 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e48acd700 1 Processor -- start 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e48acd700 1 -- start start 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e48acd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e48acd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4410e070 con 0x7f8e4410a7d0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36996/0 (socket says 172.21.15.5:36996) 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e4259c700 1 -- 172.21.15.5:0/2555510116 learned_addr learned my addr 172.21.15.5:0/2555510116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.595+0000 7f8e4259c700 1 -- 172.21.15.5:0/2555510116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4419e540 con 0x7f8e4410a7d0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.596+0000 7f8e4259c700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4419e000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e34000c00 tx=0x7f8e34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.596+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e34003aa0 con 0x7f8e4410a7d0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.596+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4419e7d0 con 0x7f8e4410a7d0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.596+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e3401b8f0 con 0x7f8e4410a7d0 2024-05-14T15:13:44.493 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.596+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e34023890 con 0x7f8e4410a7d0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.596+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a13e0 con 0x7f8e4410a7d0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.597+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f8e3402c3f0 con 0x7f8e4410a7d0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.597+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44064e20 con 0x7f8e4410a7d0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.597+0000 7f8e3b7fe700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8e3003e3a0 0x7f8e300408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.597+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8e34054d30 con 0x7f8e4410a7d0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.597+0000 7f8e41d9b700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8e3003e3a0 0x7f8e300408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.598+0000 7f8e41d9b700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8e3003e3a0 0x7f8e300408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c006fd0 tx=0x7f8e2c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.603+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e3401f030 con 0x7f8e4410a7d0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.833+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8e441a1670 con 0x7f8e3003e3a0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.890+0000 7f8e3b7fe700 1 -- 172.21.15.5:0/2555510116 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8e441a1670 con 0x7f8e3003e3a0 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8e3003e3a0 msgr2=0x7f8e300408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8e3003e3a0 0x7f8e300408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c006fd0 tx=0x7f8e2c006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 msgr2=0x7f8e4419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4419e000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e34000c00 tx=0x7f8e34011aa0 comp rx=0 tx=0).stop 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 shutdown_connections 2024-05-14T15:13:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8e3003e3a0 0x7f8e300408b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:44.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 --2- 172.21.15.5:0/2555510116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4410a7d0 0x7f8e4419e000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:44.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 >> 172.21.15.5:0/2555510116 conn(0x7f8e44104290 msgr2=0x7f8e44104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:44.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 shutdown_connections 2024-05-14T15:13:44.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:43.893+0000 7f8e48acd700 1 -- 172.21.15.5:0/2555510116 wait complete. 2024-05-14T15:13:44.495 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7VsTj4EjnbRYvXIsWp6BqOopNME07f4QfRrngE3tuvfsnGO/ynvpe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715699625, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.899+0000 7f1f4cce1700 1 Processor -- start 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.899+0000 7f1f4cce1700 1 -- start start 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.899+0000 7f1f4cce1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f4810ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.899+0000 7f1f4cce1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810f1f0 con 0x7f1f4807d6b0 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f4810ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f4810ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37006/0 (socket says 172.21.15.5:37006) 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4659c700 1 -- 172.21.15.5:0/3268704301 learned_addr learned my addr 172.21.15.5:0/3268704301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:46.024 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4659c700 1 -- 172.21.15.5:0/3268704301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4810f370 con 0x7f1f4807d6b0 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4659c700 1 --2- 172.21.15.5:0/3268704301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f4810ecb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009e80 tx=0x7f1f3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5654dec36287ce2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4559a700 1 -- 172.21.15.5:0/3268704301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f38017070 con 0x7f1f4807d6b0 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4559a700 1 -- 172.21.15.5:0/3268704301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3800b8c0 con 0x7f1f4807d6b0 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.900+0000 7f1f4559a700 1 -- 172.21.15.5:0/3268704301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f38004490 con 0x7f1f4807d6b0 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 -- 172.21.15.5:0/3268704301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 msgr2=0x7f1f4810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 --2- 172.21.15.5:0/3268704301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f4810ecb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009e80 tx=0x7f1f3800b330 comp rx=0 tx=0).stop 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 -- 172.21.15.5:0/3268704301 shutdown_connections 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 --2- 172.21.15.5:0/3268704301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f4810ecb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 -- 172.21.15.5:0/3268704301 >> 172.21.15.5:0/3268704301 conn(0x7f1f481042a0 msgr2=0x7f1f48106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 -- 172.21.15.5:0/3268704301 shutdown_connections 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.901+0000 7f1f4cce1700 1 -- 172.21.15.5:0/3268704301 wait complete. 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4cce1700 1 Processor -- start 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4cce1700 1 -- start start 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4cce1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f481a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4cce1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481102c0 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f481a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f481a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37022/0 (socket says 172.21.15.5:37022) 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.902+0000 7f1f4659c700 1 -- 172.21.15.5:0/966651132 learned_addr learned my addr 172.21.15.5:0/966651132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f4659c700 1 -- 172.21.15.5:0/966651132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f481a0730 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f4659c700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f481a01f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009e50 tx=0x7f1f3800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f38020020 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f38012430 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f481a09c0 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f38017040 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.903+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481a35d0 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.904+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f1f38023020 con 0x7f1f4807d6b0 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.904+0000 7f1f3f7fe700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1f3403e4d0 0x7f1f340409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:46.026 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.904+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f3800f050 con 0x7f1f4807d6b0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.904+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f48064e20 con 0x7f1f4807d6b0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.904+0000 7f1f45d9b700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1f3403e4d0 0x7f1f340409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.905+0000 7f1f45d9b700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1f3403e4d0 0x7f1f340409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f30006fd0 tx=0x7f1f30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:44.910+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f3802a020 con 0x7f1f4807d6b0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.162+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1f481a37a0 con 0x7f1f3403e4d0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.408+0000 7f1f3f7fe700 1 -- 172.21.15.5:0/966651132 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1f481a37a0 con 0x7f1f3403e4d0 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.410+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1f3403e4d0 msgr2=0x7f1f340409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.410+0000 7f1f4cce1700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1f3403e4d0 0x7f1f340409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f30006fd0 tx=0x7f1f30006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.410+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 msgr2=0x7f1f481a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.410+0000 7f1f4cce1700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f481a01f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009e50 tx=0x7f1f3800bd50 comp rx=0 tx=0).stop 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.411+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 shutdown_connections 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.411+0000 7f1f4cce1700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1f3403e4d0 0x7f1f340409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.411+0000 7f1f4cce1700 1 --2- 172.21.15.5:0/966651132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4807d6b0 0x7f1f481a01f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.411+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 >> 172.21.15.5:0/966651132 conn(0x7f1f481042a0 msgr2=0x7f1f48104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:46.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.411+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 shutdown_connections 2024-05-14T15:13:46.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:45.411+0000 7f1f4cce1700 1 -- 172.21.15.5:0/966651132 wait complete. 2024-05-14T15:13:46.028 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-05-14T15:13:46.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:46.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:46 smithi005 ceph-mon[31559]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:46.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:47.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:47 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1910473792' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-14T15:13:47.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-05-14T15:13:47.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.446+0000 7f3212412700 1 Processor -- start 2024-05-14T15:13:47.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.446+0000 7f3212412700 1 -- start start 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.446+0000 7f3212412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c109000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.446+0000 7f3212412700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1095f0 con 0x7f320c108bf0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c109000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45918/0 (socket says 172.21.15.5:45918) 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320bfff700 1 -- 172.21.15.5:0/2739446479 learned_addr learned my addr 172.21.15.5:0/2739446479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320bfff700 1 -- 172.21.15.5:0/2739446479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c109d20 con 0x7f320c108bf0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320bfff700 1 --2- 172.21.15.5:0/2739446479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c109000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0098a0 tx=0x7f31fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0e4a24801489b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320b7fe700 1 -- 172.21.15.5:0/2739446479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31fc004160 con 0x7f320c108bf0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320b7fe700 1 -- 172.21.15.5:0/2739446479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc004300 con 0x7f320c108bf0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.447+0000 7f320b7fe700 1 -- 172.21.15.5:0/2739446479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31fc011a20 con 0x7f320c108bf0 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.448+0000 7f3212412700 1 -- 172.21.15.5:0/2739446479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 msgr2=0x7f320c109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.448+0000 7f3212412700 1 --2- 172.21.15.5:0/2739446479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c109000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0098a0 tx=0x7f31fc009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:47.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.448+0000 7f3212412700 1 -- 172.21.15.5:0/2739446479 shutdown_connections 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.448+0000 7f3212412700 1 --2- 172.21.15.5:0/2739446479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c109000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.448+0000 7f3212412700 1 -- 172.21.15.5:0/2739446479 >> 172.21.15.5:0/2739446479 conn(0x7f320c104290 msgr2=0x7f320c1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.449+0000 7f3212412700 1 -- 172.21.15.5:0/2739446479 shutdown_connections 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.449+0000 7f3212412700 1 -- 172.21.15.5:0/2739446479 wait complete. 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.449+0000 7f3212412700 1 Processor -- start 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.449+0000 7f3212412700 1 -- start start 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.449+0000 7f3212412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c07c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.449+0000 7f3212412700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f320c10d0d0 con 0x7f320c108bf0 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.450+0000 7f320bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c07c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.450+0000 7f320bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c07c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45930/0 (socket says 172.21.15.5:45930) 2024-05-14T15:13:47.695 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.450+0000 7f320bfff700 1 -- 172.21.15.5:0/1910473792 learned_addr learned my addr 172.21.15.5:0/1910473792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.450+0000 7f320bfff700 1 -- 172.21.15.5:0/1910473792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c07cee0 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.450+0000 7f320bfff700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c07c9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f31fc003e70 tx=0x7f31fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.450+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31fc003aa0 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.451+0000 7f3212412700 1 -- 172.21.15.5:0/1910473792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c1a0e20 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.451+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc01a8e0 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.451+0000 7f3212412700 1 -- 172.21.15.5:0/1910473792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c1a1300 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.452+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31fc024870 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.452+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0005330 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.452+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f31fc022020 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.453+0000 7f3209ffb700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f31f803e4c0 0x7f31f80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.453+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f31fc0539c0 con 0x7f320c108bf0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.453+0000 7f3203fff700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f31f803e4c0 0x7f31f80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:47.696 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.453+0000 7f3203fff700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f31f803e4c0 0x7f31f80409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f31f4006fd0 tx=0x7f31f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.458+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31fc055e20 con 0x7f320c108bf0 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.691+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f31f0005620 con 0x7f320c108bf0 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.691+0000 7f3209ffb700 1 -- 172.21.15.5:0/1910473792 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f31fc0212c0 con 0x7f320c108bf0 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f31f803e4c0 msgr2=0x7f31f80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f31f803e4c0 0x7f31f80409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f31f4006fd0 tx=0x7f31f4006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 msgr2=0x7f320c07c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c07c9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f31fc003e70 tx=0x7f31fc011aa0 comp rx=0 tx=0).stop 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 shutdown_connections 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f31f803e4c0 0x7f31f80409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 --2- 172.21.15.5:0/1910473792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320c108bf0 0x7f320c07c9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.693+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 >> 172.21.15.5:0/1910473792 conn(0x7f320c104290 msgr2=0x7f320c104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.694+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 shutdown_connections 2024-05-14T15:13:47.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:46.694+0000 7f3202ffd700 1 -- 172.21.15.5:0/1910473792 wait complete. 2024-05-14T15:13:47.703 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T15:13:47.704 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-05-14T15:13:47.712 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T15:13:47.712 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-05-14T15:13:47.712 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-05-14T15:13:47.712 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: Password: mo7csth5nq 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:47.715 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config directory 2024-05-14T15:13:47.716 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling autotune for osd_memory_target 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54fe499700 1 Processor -- start 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54fe499700 1 -- start start 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54fe499700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f810c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54fe499700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54f810ce70 con 0x7f54f807d550 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f810c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45934/0 (socket says 172.21.15.5:45934) 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.157+0000 7f54f7fff700 1 -- 172.21.15.5:0/399810295 learned_addr learned my addr 172.21.15.5:0/399810295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.158+0000 7f54f7fff700 1 -- 172.21.15.5:0/399810295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f810cff0 con 0x7f54f807d550 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.158+0000 7f54f7fff700 1 --2- 172.21.15.5:0/399810295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f810c860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54e80098a0 tx=0x7f54e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c70853c6ee997 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.158+0000 7f54f77fe700 1 -- 172.21.15.5:0/399810295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e8004160 con 0x7f54f807d550 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.158+0000 7f54f77fe700 1 -- 172.21.15.5:0/399810295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e8004300 con 0x7f54f807d550 2024-05-14T15:13:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.158+0000 7f54f77fe700 1 -- 172.21.15.5:0/399810295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e8011a20 con 0x7f54f807d550 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 -- 172.21.15.5:0/399810295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 msgr2=0x7f54f810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 --2- 172.21.15.5:0/399810295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f810c860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f54e80098a0 tx=0x7f54e8009bb0 comp rx=0 tx=0).stop 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 -- 172.21.15.5:0/399810295 shutdown_connections 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 --2- 172.21.15.5:0/399810295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f810c860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 -- 172.21.15.5:0/399810295 >> 172.21.15.5:0/399810295 conn(0x7f54f8104290 msgr2=0x7f54f8106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 -- 172.21.15.5:0/399810295 shutdown_connections 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.159+0000 7f54fe499700 1 -- 172.21.15.5:0/399810295 wait complete. 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54fe499700 1 Processor -- start 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54fe499700 1 -- start start 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54fe499700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54fe499700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54f810e0b0 con 0x7f54f807d550 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45938/0 (socket says 172.21.15.5:45938) 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54f7fff700 1 -- 172.21.15.5:0/31698930 learned_addr learned my addr 172.21.15.5:0/31698930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.160+0000 7f54f7fff700 1 -- 172.21.15.5:0/31698930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f819e560 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.161+0000 7f54f7fff700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f819e020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.161+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e8003aa0 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.161+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54e801b8f0 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.161+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e8023890 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.161+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f819e7f0 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.161+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f81a1400 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.162+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f54e802c3f0 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.162+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f8064e20 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.162+0000 7f54f5ffb700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f54e403e5c0 0x7f54e4040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.162+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f54e8053fd0 con 0x7f54f807d550 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.162+0000 7f54effff700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f54e403e5c0 0x7f54e4040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.163+0000 7f54effff700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f54e403e5c0 0x7f54e4040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f54e0006fd0 tx=0x7f54e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.168+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54e801f030 con 0x7f54f807d550 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.384+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f54f81a15d0 con 0x7f54f807d550 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.384+0000 7f54f5ffb700 1 -- 172.21.15.5:0/31698930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f54e8017e90 con 0x7f54f807d550 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f54e403e5c0 msgr2=0x7f54e4040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f54e403e5c0 0x7f54e4040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f54e0006fd0 tx=0x7f54e0006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 msgr2=0x7f54f819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f819e020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e8011aa0 comp rx=0 tx=0).stop 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 shutdown_connections 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f54e403e5c0 0x7f54e4040ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 --2- 172.21.15.5:0/31698930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f807d550 0x7f54f819e020 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.386+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 >> 172.21.15.5:0/31698930 conn(0x7f54f8104290 msgr2=0x7f54f8104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.387+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 shutdown_connections 2024-05-14T15:13:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:48.387+0000 7f54fe499700 1 -- 172.21.15.5:0/31698930 wait complete. 2024-05-14T15:13:49.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:49 smithi005 ceph-mon[31559]: mgrmap e13: x(active, since 6s) 2024-05-14T15:13:50.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.407+0000 7f2cbda3d700 1 Processor -- start 2024-05-14T15:13:50.288 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.407+0000 7f2cbda3d700 1 -- start start 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cbda3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb8109010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cbda3d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb8109600 con 0x7f2cb8108c00 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb8109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb8109010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45942/0 (socket says 172.21.15.5:45942) 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cb6ffd700 1 -- 172.21.15.5:0/1190695477 learned_addr learned my addr 172.21.15.5:0/1190695477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cb6ffd700 1 -- 172.21.15.5:0/1190695477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb8109d30 con 0x7f2cb8108c00 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.408+0000 7f2cb6ffd700 1 --2- 172.21.15.5:0/1190695477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb8109010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009e80 tx=0x7f2ca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b536abcf5f9ac3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cb5ffb700 1 -- 172.21.15.5:0/1190695477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8017070 con 0x7f2cb8108c00 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cb5ffb700 1 -- 172.21.15.5:0/1190695477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca800b8c0 con 0x7f2cb8108c00 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cb5ffb700 1 -- 172.21.15.5:0/1190695477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8004490 con 0x7f2cb8108c00 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 -- 172.21.15.5:0/1190695477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 msgr2=0x7f2cb8109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 --2- 172.21.15.5:0/1190695477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb8109010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009e80 tx=0x7f2ca800b330 comp rx=0 tx=0).stop 2024-05-14T15:13:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 -- 172.21.15.5:0/1190695477 shutdown_connections 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 --2- 172.21.15.5:0/1190695477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb8109010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 -- 172.21.15.5:0/1190695477 >> 172.21.15.5:0/1190695477 conn(0x7f2cb8104280 msgr2=0x7f2cb81066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 -- 172.21.15.5:0/1190695477 shutdown_connections 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.409+0000 7f2cbda3d700 1 -- 172.21.15.5:0/1190695477 wait complete. 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.410+0000 7f2cbda3d700 1 Processor -- start 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.410+0000 7f2cbda3d700 1 -- start start 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbda3d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbda3d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810d0e0 con 0x7f2cb8108c00 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45950/0 (socket says 172.21.15.5:45950) 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cb6ffd700 1 -- 172.21.15.5:0/4249227084 learned_addr learned my addr 172.21.15.5:0/4249227084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cb6ffd700 1 -- 172.21.15.5:0/4249227084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb819e5f0 con 0x7f2cb8108c00 2024-05-14T15:13:50.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cb6ffd700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb819e0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800bc80 tx=0x7f2ca800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8020020 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb819e880 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca8003b50 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8017040 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.411+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb81a1490 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.412+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f2ca8023020 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.413+0000 7f2cbca3b700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2ca403e560 0x7f2ca4040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.413+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb8064e20 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.413+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ca800f050 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.413+0000 7f2cb67fc700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2ca403e560 0x7f2ca4040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.413+0000 7f2cb67fc700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2ca403e560 0x7f2ca4040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0006fd0 tx=0x7f2ca0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.418+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ca802a020 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.759+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2cb802e900 con 0x7f2cb8108c00 2024-05-14T15:13:50.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.762+0000 7f2cbca3b700 1 -- 172.21.15.5:0/4249227084 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f2ca802a200 con 0x7f2cb8108c00 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2ca403e560 msgr2=0x7f2ca4040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2ca403e560 0x7f2ca4040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0006fd0 tx=0x7f2ca0006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 msgr2=0x7f2cb819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb819e0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800bc80 tx=0x7f2ca800bd60 comp rx=0 tx=0).stop 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 shutdown_connections 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2ca403e560 0x7f2ca4040a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 --2- 172.21.15.5:0/4249227084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb8108c00 0x7f2cb819e0b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.764+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 >> 172.21.15.5:0/4249227084 conn(0x7f2cb8104280 msgr2=0x7f2cb8194690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.765+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 shutdown_connections 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-05-14T15:13:49.765+0000 7f2cbda3d700 1 -- 172.21.15.5:0/4249227084 wait complete. 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-05-14T15:13:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-14T15:13:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:13:50.294 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-05-14T15:13:50.323 INFO:tasks.cephadm:Fetching config... 2024-05-14T15:13:50.323 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:13:50.323 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-14T15:13:50.340 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-14T15:13:50.340 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:13:50.340 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-14T15:13:50.396 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-14T15:13:50.396 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:13:50.396 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-14T15:13:50.462 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-14T15:13:50.462 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:13:50.463 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-14T15:13:50.519 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-14T15:13:50.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T15:13:50.595 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:13:50.607 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T15:13:50.645 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:13:50.657 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T15:13:50.697 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhlfzV1KkCAGn73Yi4/iNiVU7X09OZnBKFLkl7J9ERLiTEleEWP1Hwkv/QED7uT+2fu+AOmviGP0LwfmjhOM1hKVmcJQ/y/EfCCCHrxibyF9FblB8F0DfbTPH5EDMSQ8f8C1LwWBjUIO3WC1nfFDMLwCmMsho6flnRMxuhj1TBgfWdwtylfaXFtVS9hKiEa0pJId/xMjzoSNzGdmhA8GeffQ1YgDKu1MbhlWHkaaiTpQWOgXPgLd5vkw+qSkEtAKeIzqA8U7/4WHFuKp1KY0CgwwbeeKOH/nDkPv8mIwKNIVniFgIp/+tRT8upg7A6P811cJUHJeyew2MPrw6+XLrQjw1plGnU1ZW2O017L0VGHexLZ72US0ysrke3fR3+x+mgQcgaaFY1dD/YM1of+KbRAbHvUD8cIRWI5D3JHCFlTKXkB3fL++I9knOmfrJS2I8SZnXTH4gTpT7nu0q7VN8YfxKVlWYJvJHuh6IUJrXI1qD/hJLYXLUpQ4zjzE0EwFk= ceph-4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:13:50.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-14T15:13:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:13:51.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:50 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4249227084' entity='client.admin' 2024-05-14T15:13:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 -- 172.21.15.5:0/1125854068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 msgr2=0x7f8a54103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:53.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 --2- 172.21.15.5:0/1125854068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54103ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009910 tx=0x7f8a44009c20 comp rx=0 tx=0).stop 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 -- 172.21.15.5:0/1125854068 shutdown_connections 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 --2- 172.21.15.5:0/1125854068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54103ff0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 -- 172.21.15.5:0/1125854068 >> 172.21.15.5:0/1125854068 conn(0x7f8a540ff220 msgr2=0x7f8a541016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 -- 172.21.15.5:0/1125854068 shutdown_connections 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 -- 172.21.15.5:0/1125854068 wait complete. 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.342+0000 7f8a5a612700 1 Processor -- start 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a5a612700 1 -- start start 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a5a612700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a5a612700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54109ba0 con 0x7f8a54103be0 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a53fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45972/0 (socket says 172.21.15.5:45972) 2024-05-14T15:13:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a53fff700 1 -- 172.21.15.5:0/4284833220 learned_addr learned my addr 172.21.15.5:0/4284833220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.343+0000 7f8a53fff700 1 -- 172.21.15.5:0/4284833220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a54199c60 con 0x7f8a54103be0 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.344+0000 7f8a53fff700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54199720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a440057d0 tx=0x7f8a44005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.344+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a44027070 con 0x7f8a54103be0 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.344+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4400be10 con 0x7f8a54103be0 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.344+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a54199ef0 con 0x7f8a54103be0 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.344+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a44022dd0 con 0x7f8a54103be0 2024-05-14T15:13:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.344+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5419cb00 con 0x7f8a54103be0 2024-05-14T15:13:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.345+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f8a4402a070 con 0x7f8a54103be0 2024-05-14T15:13:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.345+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5410a320 con 0x7f8a54103be0 2024-05-14T15:13:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.345+0000 7f8a517fa700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8a3c03e4c0 0x7f8a3c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.345+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a4401e030 con 0x7f8a54103be0 2024-05-14T15:13:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.345+0000 7f8a537fe700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8a3c03e4c0 0x7f8a3c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.346+0000 7f8a537fe700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8a3c03e4c0 0x7f8a3c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006fd0 tx=0x7f8a48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:53.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.351+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a44013070 con 0x7f8a54103be0 2024-05-14T15:13:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.569+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8a5402e700 con 0x7f8a54103be0 2024-05-14T15:13:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.572+0000 7f8a517fa700 1 -- 172.21.15.5:0/4284833220 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8a4400c2d0 con 0x7f8a54103be0 2024-05-14T15:13:53.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.574+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8a3c03e4c0 msgr2=0x7f8a3c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8a3c03e4c0 0x7f8a3c0409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006fd0 tx=0x7f8a48006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 msgr2=0x7f8a54199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54199720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a440057d0 tx=0x7f8a44005800 comp rx=0 tx=0).stop 2024-05-14T15:13:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 shutdown_connections 2024-05-14T15:13:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8a3c03e4c0 0x7f8a3c0409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 --2- 172.21.15.5:0/4284833220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a54103be0 0x7f8a54199720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 >> 172.21.15.5:0/4284833220 conn(0x7f8a540ff220 msgr2=0x7f8a540fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 shutdown_connections 2024-05-14T15:13:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:53.575+0000 7f8a5a612700 1 -- 172.21.15.5:0/4284833220 wait complete. 2024-05-14T15:13:54.112 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-14T15:13:54.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-14T15:13:54.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:54 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4284833220' entity='client.admin' 2024-05-14T15:13:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:13:55.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:55.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:55.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:13:55.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:55.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:55.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 -- 172.21.15.5:0/1781167800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 msgr2=0x7f1728103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 --2- 172.21.15.5:0/1781167800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728103ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1724009910 tx=0x7f1724009c20 comp rx=0 tx=0).stop 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 -- 172.21.15.5:0/1781167800 shutdown_connections 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 --2- 172.21.15.5:0/1781167800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728103ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 -- 172.21.15.5:0/1781167800 >> 172.21.15.5:0/1781167800 conn(0x7f17280ff220 msgr2=0x7f17281016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 -- 172.21.15.5:0/1781167800 shutdown_connections 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.337+0000 7f173079b700 1 -- 172.21.15.5:0/1781167800 wait complete. 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f173079b700 1 Processor -- start 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f173079b700 1 -- start start 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f173079b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728077380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f173079b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1728109ba0 con 0x7f1728103be0 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f172e537700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728077380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f172e537700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728077380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57994/0 (socket says 172.21.15.5:57994) 2024-05-14T15:13:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f172e537700 1 -- 172.21.15.5:0/2379361781 learned_addr learned my addr 172.21.15.5:0/2379361781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.338+0000 7f172e537700 1 -- 172.21.15.5:0/2379361781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17280778c0 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.339+0000 7f172e537700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728077380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17240057d0 tx=0x7f1724005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.339+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1724027070 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.339+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1728075b10 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.339+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f172400be10 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.339+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1724022dd0 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.339+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1728075ff0 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.340+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f172402a070 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.340+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172810a320 con 0x7f1728103be0 2024-05-14T15:13:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.340+0000 7f171f7fe700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f171403e4c0 0x7f17140409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.340+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f172401e030 con 0x7f1728103be0 2024-05-14T15:13:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.340+0000 7f172dd36700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f171403e4c0 0x7f17140409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.341+0000 7f172dd36700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f171403e4c0 0x7f17140409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1718006fd0 tx=0x7f1718006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.346+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1724013070 con 0x7f1728103be0 2024-05-14T15:13:56.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.562+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f1728192540 con 0x7f171403e4c0 2024-05-14T15:13:56.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.566+0000 7f171f7fe700 1 -- 172.21.15.5:0/2379361781 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1728192540 con 0x7f171403e4c0 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f171403e4c0 msgr2=0x7f17140409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f171403e4c0 0x7f17140409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1718006fd0 tx=0x7f1718006e40 comp rx=0 tx=0).stop 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 msgr2=0x7f1728077380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728077380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17240057d0 tx=0x7f1724005800 comp rx=0 tx=0).stop 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 shutdown_connections 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f171403e4c0 0x7f17140409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 --2- 172.21.15.5:0/2379361781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1728103be0 0x7f1728077380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 >> 172.21.15.5:0/2379361781 conn(0x7f17280ff220 msgr2=0x7f17280fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 shutdown_connections 2024-05-14T15:13:56.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:56.568+0000 7f173079b700 1 -- 172.21.15.5:0/2379361781 wait complete. 2024-05-14T15:13:57.256 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-05-14T15:13:57.256 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:13:57.256 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-05-14T15:13:57.276 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:13:57.276 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:13:57.333 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-05-14T15:13:57.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch host add smithi144 2024-05-14T15:13:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:13:57.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:57 smithi005 ceph-mon[31559]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:13:57.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:57.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:13:57.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:13:57.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:13:57.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:57 smithi005 ceph-mon[31559]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:13:58.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:58 smithi005 ceph-mon[31559]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:13:58.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:58 smithi005 ceph-mon[31559]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:13:58.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:58 smithi005 ceph-mon[31559]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:13:58.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:58.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:58.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:13:59.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.086+0000 7f0a64705700 1 -- 172.21.15.5:0/1375107276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 msgr2=0x7f0a5c108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:13:59.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.086+0000 7f0a64705700 1 --2- 172.21.15.5:0/1375107276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c108ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009910 tx=0x7f0a58009c20 comp rx=0 tx=0).stop 2024-05-14T15:13:59.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.086+0000 7f0a64705700 1 -- 172.21.15.5:0/1375107276 shutdown_connections 2024-05-14T15:13:59.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.086+0000 7f0a64705700 1 --2- 172.21.15.5:0/1375107276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c108ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:13:59.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.086+0000 7f0a64705700 1 -- 172.21.15.5:0/1375107276 >> 172.21.15.5:0/1375107276 conn(0x7f0a5c0785b0 msgr2=0x7f0a5c078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:13:59.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.087+0000 7f0a64705700 1 -- 172.21.15.5:0/1375107276 shutdown_connections 2024-05-14T15:13:59.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.087+0000 7f0a64705700 1 -- 172.21.15.5:0/1375107276 wait complete. 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.087+0000 7f0a64705700 1 Processor -- start 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.087+0000 7f0a64705700 1 -- start start 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a64705700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a64705700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c10cfe0 con 0x7f0a5c108a90 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a624a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a624a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c19f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58012/0 (socket says 172.21.15.5:58012) 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a624a1700 1 -- 172.21.15.5:0/1624012977 learned_addr learned my addr 172.21.15.5:0/1624012977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:13:59.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a624a1700 1 -- 172.21.15.5:0/1624012977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c19fb30 con 0x7f0a5c108a90 2024-05-14T15:13:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a624a1700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c19f5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a580056d0 tx=0x7f0a580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58027070 con 0x7f0a5c108a90 2024-05-14T15:13:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5800be10 con 0x7f0a5c108a90 2024-05-14T15:13:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c19fe20 con 0x7f0a5c108a90 2024-05-14T15:13:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.088+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58022d20 con 0x7f0a5c108a90 2024-05-14T15:13:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.089+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c199950 con 0x7f0a5c108a90 2024-05-14T15:13:59.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.089+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f0a5802a070 con 0x7f0a5c108a90 2024-05-14T15:13:59.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.090+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c1041a0 con 0x7f0a5c108a90 2024-05-14T15:13:59.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.090+0000 7f0a537fe700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a4803e510 0x7f0a48040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:13:59.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.090+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a5801e030 con 0x7f0a5c108a90 2024-05-14T15:13:59.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.090+0000 7f0a61ca0700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a4803e510 0x7f0a48040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:13:59.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.090+0000 7f0a61ca0700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a4803e510 0x7f0a48040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006fd0 tx=0x7f0a4c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:13:59.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.095+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a58013070 con 0x7f0a5c108a90 2024-05-14T15:13:59.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:13:59.314+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7f0a5c19ad00 con 0x7f0a4803e510 2024-05-14T15:14:00.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:13:59 smithi005 ceph-mon[31559]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:01.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:00 smithi005 ceph-mon[31559]: Deploying cephadm binary to smithi144 2024-05-14T15:14:02.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:01 smithi005 ceph-mon[31559]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.643+0000 7f0a537fe700 1 -- 172.21.15.5:0/1624012977 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0a5c19ad00 con 0x7f0a4803e510 2024-05-14T15:14:03.642 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a4803e510 msgr2=0x7f0a48040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a4803e510 0x7f0a48040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006fd0 tx=0x7f0a4c006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 msgr2=0x7f0a5c19f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c19f5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a580056d0 tx=0x7f0a580057b0 comp rx=0 tx=0).stop 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 shutdown_connections 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a4803e510 0x7f0a48040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 --2- 172.21.15.5:0/1624012977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a5c108a90 0x7f0a5c19f5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 >> 172.21.15.5:0/1624012977 conn(0x7f0a5c0785b0 msgr2=0x7f0a5c106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 shutdown_connections 2024-05-14T15:14:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:03.646+0000 7f0a64705700 1 -- 172.21.15.5:0/1624012977 wait complete. 2024-05-14T15:14:04.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T15:14:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:04 smithi005 ceph-mon[31559]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:04 smithi005 ceph-mon[31559]: Added host smithi144 2024-05-14T15:14:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:04.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:05.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.026+0000 7f41c9292700 1 -- 172.21.15.5:0/765187202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 msgr2=0x7f41c410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.026+0000 7f41c9292700 1 --2- 172.21.15.5:0/765187202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c410a040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f41b400b0a0 tx=0x7f41b400b3b0 comp rx=0 tx=0).stop 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.026+0000 7f41c9292700 1 -- 172.21.15.5:0/765187202 shutdown_connections 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.026+0000 7f41c9292700 1 --2- 172.21.15.5:0/765187202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c410a040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.026+0000 7f41c9292700 1 -- 172.21.15.5:0/765187202 >> 172.21.15.5:0/765187202 conn(0x7f41c40ff670 msgr2=0x7f41c4101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.027+0000 7f41c9292700 1 -- 172.21.15.5:0/765187202 shutdown_connections 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.027+0000 7f41c9292700 1 -- 172.21.15.5:0/765187202 wait complete. 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.027+0000 7f41c9292700 1 Processor -- start 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.027+0000 7f41c9292700 1 -- start start 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c9292700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c9292700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410e110 con 0x7f41c4109c30 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42484/0 (socket says 172.21.15.5:42484) 2024-05-14T15:14:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c2ffd700 1 -- 172.21.15.5:0/3711391769 learned_addr learned my addr 172.21.15.5:0/3711391769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:14:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c2ffd700 1 -- 172.21.15.5:0/3711391769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c419fac0 con 0x7f41c4109c30 2024-05-14T15:14:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41c2ffd700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c419f580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f41b400b830 tx=0x7f41b4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41b4021020 con 0x7f41c4109c30 2024-05-14T15:14:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.028+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b4007a80 con 0x7f41c4109c30 2024-05-14T15:14:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.029+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c419fd50 con 0x7f41c4109c30 2024-05-14T15:14:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.029+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41b4017040 con 0x7f41c4109c30 2024-05-14T15:14:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.029+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c4199710 con 0x7f41c4109c30 2024-05-14T15:14:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.029+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4105200 con 0x7f41c4109c30 2024-05-14T15:14:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.029+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f41b4024020 con 0x7f41c4109c30 2024-05-14T15:14:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.030+0000 7f41abfff700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f41ac03e570 0x7f41ac040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.030+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f41b400f050 con 0x7f41c4109c30 2024-05-14T15:14:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.030+0000 7f41c27fc700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f41ac03e570 0x7f41ac040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:06.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.030+0000 7f41c27fc700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f41ac03e570 0x7f41ac040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.035+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41b402b020 con 0x7f41c4109c30 2024-05-14T15:14:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.258+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41c4064e50 con 0x7f41ac03e570 2024-05-14T15:14:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.259+0000 7f41abfff700 1 -- 172.21.15.5:0/3711391769 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f41c4064e50 con 0x7f41ac03e570 2024-05-14T15:14:06.257 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:14:06.257 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-05-14T15:14:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f41ac03e570 msgr2=0x7f41ac040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f41ac03e570 0x7f41ac040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 msgr2=0x7f41c419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c419f580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f41b400b830 tx=0x7f41b4009790 comp rx=0 tx=0).stop 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 shutdown_connections 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f41ac03e570 0x7f41ac040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 --2- 172.21.15.5:0/3711391769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41c4109c30 0x7f41c419f580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 >> 172.21.15.5:0/3711391769 conn(0x7f41c40ff670 msgr2=0x7f41c4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 shutdown_connections 2024-05-14T15:14:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:06.262+0000 7f41c9292700 1 -- 172.21.15.5:0/3711391769 wait complete. 2024-05-14T15:14:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:06 smithi005 ceph-mon[31559]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:06 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:06.807 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-05-14T15:14:06.807 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:14:06.807 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-05-14T15:14:06.826 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:14:06.826 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:06.884 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-05-14T15:14:06.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch host add smithi193 2024-05-14T15:14:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:07.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:07 smithi005 ceph-mon[31559]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 -- 172.21.15.5:0/3282746507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 msgr2=0x7f6d90108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 --2- 172.21.15.5:0/3282746507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d90108e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009910 tx=0x7f6d8c009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 -- 172.21.15.5:0/3282746507 shutdown_connections 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 --2- 172.21.15.5:0/3282746507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d90108e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 -- 172.21.15.5:0/3282746507 >> 172.21.15.5:0/3282746507 conn(0x7f6d900fe5c0 msgr2=0x7f6d90100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 -- 172.21.15.5:0/3282746507 shutdown_connections 2024-05-14T15:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.515+0000 7f6d983f5700 1 -- 172.21.15.5:0/3282746507 wait complete. 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d983f5700 1 Processor -- start 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d983f5700 1 -- start start 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d983f5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d9019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d983f5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9010cfc0 con 0x7f6d90108a70 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d96191700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d9019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d96191700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d9019f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42494/0 (socket says 172.21.15.5:42494) 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.516+0000 7f6d96191700 1 -- 172.21.15.5:0/4143500062 learned_addr learned my addr 172.21.15.5:0/4143500062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d96191700 1 -- 172.21.15.5:0/4143500062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9019fac0 con 0x7f6d90108a70 2024-05-14T15:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d96191700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d9019f580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0056d0 tx=0x7f6d8c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d8c027070 con 0x7f6d90108a70 2024-05-14T15:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d8c00be10 con 0x7f6d90108a70 2024-05-14T15:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9019fdb0 con 0x7f6d90108a70 2024-05-14T15:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d8c022d20 con 0x7f6d90108a70 2024-05-14T15:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.517+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d901997c0 con 0x7f6d90108a70 2024-05-14T15:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.518+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d90104180 con 0x7f6d90108a70 2024-05-14T15:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.518+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f6d8c02a070 con 0x7f6d90108a70 2024-05-14T15:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.518+0000 7f6d877fe700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6d7c03e570 0x7f6d7c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.518+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d8c01e030 con 0x7f6d90108a70 2024-05-14T15:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.518+0000 7f6d95990700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6d7c03e570 0x7f6d7c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.519+0000 7f6d95990700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6d7c03e570 0x7f6d7c040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006fd0 tx=0x7f6d80006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:08.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.524+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d8c013070 con 0x7f6d90108a70 2024-05-14T15:14:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:08.743+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7f6d9019acd0 con 0x7f6d7c03e570 2024-05-14T15:14:09.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:08 smithi005 ceph-mon[31559]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:10.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:14:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:10.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:09 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:11.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:10 smithi005 ceph-mon[31559]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:11.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:10 smithi005 ceph-mon[31559]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:14:11.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:10 smithi005 ceph-mon[31559]: Deploying cephadm binary to smithi193 2024-05-14T15:14:11.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:10 smithi005 ceph-mon[31559]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:12.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:11 smithi005 ceph-mon[31559]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:13.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:12 smithi005 ceph-mon[31559]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:13.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.131+0000 7f6d877fe700 1 -- 172.21.15.5:0/4143500062 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6d9019acd0 con 0x7f6d7c03e570 2024-05-14T15:14:13.130 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-05-14T15:14:13.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6d7c03e570 msgr2=0x7f6d7c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:13.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6d7c03e570 0x7f6d7c040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006fd0 tx=0x7f6d80006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 msgr2=0x7f6d9019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d9019f580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0056d0 tx=0x7f6d8c0057b0 comp rx=0 tx=0).stop 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 shutdown_connections 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6d7c03e570 0x7f6d7c040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 --2- 172.21.15.5:0/4143500062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d90108a70 0x7f6d9019f580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 >> 172.21.15.5:0/4143500062 conn(0x7f6d900fe5c0 msgr2=0x7f6d90106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 shutdown_connections 2024-05-14T15:14:13.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:13.134+0000 7f6d983f5700 1 -- 172.21.15.5:0/4143500062 wait complete. 2024-05-14T15:14:13.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T15:14:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:14.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:14 smithi005 ceph-mon[31559]: Added host smithi193 2024-05-14T15:14:14.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:14 smithi005 ceph-mon[31559]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:14.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:14 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 -- 172.21.15.5:0/186908033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 msgr2=0x7fd1401041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 --2- 172.21.15.5:0/186908033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd1401041e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd130009910 tx=0x7fd130009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 -- 172.21.15.5:0/186908033 shutdown_connections 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 --2- 172.21.15.5:0/186908033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd1401041e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 -- 172.21.15.5:0/186908033 >> 172.21.15.5:0/186908033 conn(0x7fd1400ff660 msgr2=0x7fd140101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 -- 172.21.15.5:0/186908033 shutdown_connections 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 -- 172.21.15.5:0/186908033 wait complete. 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 Processor -- start 2024-05-14T15:14:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.344+0000 7fd146f0f700 1 -- start start 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd146f0f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd14019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd146f0f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd14010e0c0 con 0x7fd140103dd0 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd144cab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd14019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd144cab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd14019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42514/0 (socket says 172.21.15.5:42514) 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd144cab700 1 -- 172.21.15.5:0/3737050145 learned_addr learned my addr 172.21.15.5:0/3737050145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd144cab700 1 -- 172.21.15.5:0/3737050145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14019fb40 con 0x7fd140103dd0 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd144cab700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd14019f600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1300056d0 tx=0x7fd1300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd130026070 con 0x7fd140103dd0 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd13000bba0 con 0x7fd140103dd0 2024-05-14T15:14:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.345+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14019fe30 con 0x7fd140103dd0 2024-05-14T15:14:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.346+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd130003920 con 0x7fd140103dd0 2024-05-14T15:14:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.346+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd140199960 con 0x7fd140103dd0 2024-05-14T15:14:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.347+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd140105280 con 0x7fd140103dd0 2024-05-14T15:14:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.347+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fd130029070 con 0x7fd140103dd0 2024-05-14T15:14:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.347+0000 7fd13dffb700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd12803e570 0x7fd128040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.347+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd13001e070 con 0x7fd140103dd0 2024-05-14T15:14:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.347+0000 7fd13ffff700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd12803e570 0x7fd128040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.348+0000 7fd13ffff700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd12803e570 0x7fd128040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd134006fd0 tx=0x7fd134006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:15.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.352+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd130013070 con 0x7fd140103dd0 2024-05-14T15:14:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.572+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd14019aca0 con 0x7fd12803e570 2024-05-14T15:14:15.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.573+0000 7fd13dffb700 1 -- 172.21.15.5:0/3737050145 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fd14019aca0 con 0x7fd12803e570 2024-05-14T15:14:15.571 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:14:15.571 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd12803e570 msgr2=0x7fd128040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd12803e570 0x7fd128040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd134006fd0 tx=0x7fd134006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 msgr2=0x7fd14019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd14019f600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1300056d0 tx=0x7fd1300057b0 comp rx=0 tx=0).stop 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 shutdown_connections 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd12803e570 0x7fd128040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 --2- 172.21.15.5:0/3737050145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140103dd0 0x7fd14019f600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 >> 172.21.15.5:0/3737050145 conn(0x7fd1400ff660 msgr2=0x7fd140109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 shutdown_connections 2024-05-14T15:14:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:15.575+0000 7fd146f0f700 1 -- 172.21.15.5:0/3737050145 wait complete. 2024-05-14T15:14:16.176 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-14T15:14:16.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-14T15:14:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:16.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:16 smithi005 ceph-mon[31559]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:16.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:16 smithi005 ceph-mon[31559]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:14:16.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/4280933865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 msgr2=0x7fbeac1050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 --2- 172.21.15.5:0/4280933865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac1050e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009910 tx=0x7fbea0009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/4280933865 shutdown_connections 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 --2- 172.21.15.5:0/4280933865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac1050e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/4280933865 >> 172.21.15.5:0/4280933865 conn(0x7fbeac0fe5c0 msgr2=0x7fbeac100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/4280933865 shutdown_connections 2024-05-14T15:14:17.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.896+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/4280933865 wait complete. 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb2cfd700 1 Processor -- start 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb2cfd700 1 -- start start 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb2cfd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb2cfd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10e130 con 0x7fbeac104cd0 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb0a99700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb0a99700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52596/0 (socket says 172.21.15.5:52596) 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb0a99700 1 -- 172.21.15.5:0/2725733835 learned_addr learned my addr 172.21.15.5:0/2725733835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.897+0000 7fbeb0a99700 1 -- 172.21.15.5:0/2725733835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac19fac0 con 0x7fbeac104cd0 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.898+0000 7fbeb0a99700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac19f580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbea0006010 tx=0x7fbea0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.898+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea0026070 con 0x7fbeac104cd0 2024-05-14T15:14:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.898+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea000bc00 con 0x7fbeac104cd0 2024-05-14T15:14:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.898+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac19fd50 con 0x7fbeac104cd0 2024-05-14T15:14:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.898+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea0003910 con 0x7fbeac104cd0 2024-05-14T15:14:17.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.898+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac1997c0 con 0x7fbeac104cd0 2024-05-14T15:14:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.899+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fbea0029070 con 0x7fbeac104cd0 2024-05-14T15:14:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.899+0000 7fbea9ffb700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe9c03e570 0x7fbe9c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.899+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac06c3e0 con 0x7fbeac104cd0 2024-05-14T15:14:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.899+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbea001e070 con 0x7fbeac104cd0 2024-05-14T15:14:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.899+0000 7fbeabfff700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe9c03e570 0x7fbe9c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:17.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.900+0000 7fbeabfff700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe9c03e570 0x7fbe9c040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:17.905+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbea0013070 con 0x7fbeac104cd0 2024-05-14T15:14:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.122+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbeac064e20 con 0x7fbeac104cd0 2024-05-14T15:14:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.734+0000 7fbea9ffb700 1 -- 172.21.15.5:0/2725733835 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fbea0015350 con 0x7fbeac104cd0 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe9c03e570 msgr2=0x7fbe9c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe9c03e570 0x7fbe9c040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 msgr2=0x7fbeac19f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac19f580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbea0006010 tx=0x7fbea0005780 comp rx=0 tx=0).stop 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 shutdown_connections 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe9c03e570 0x7fbe9c040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 --2- 172.21.15.5:0/2725733835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeac104cd0 0x7fbeac19f580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 >> 172.21.15.5:0/2725733835 conn(0x7fbeac0fe5c0 msgr2=0x7fbeac105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 shutdown_connections 2024-05-14T15:14:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:18.738+0000 7fbeb2cfd700 1 -- 172.21.15.5:0/2725733835 wait complete. 2024-05-14T15:14:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-05-14T15:14:19.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:18 smithi005 ceph-mon[31559]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:19.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:18 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2725733835' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T15:14:19.340 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-05-14T15:14:19.340 INFO:tasks.cephadm:Adding mon.b on smithi144 2024-05-14T15:14:19.340 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-05-14T15:14:19.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c' 2024-05-14T15:14:19.577 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:20.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2725733835' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T15:14:20.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:20.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:20.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:21.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:20 smithi005 ceph-mon[31559]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:14:21.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:20 smithi005 ceph-mon[31559]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:21.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:20 smithi005 ceph-mon[31559]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:21.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:20 smithi005 ceph-mon[31559]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:21.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:20 smithi005 ceph-mon[31559]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:14:21.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/1779504870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d34103dd0 msgr2=0x7f7d341041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 --2- 172.21.15.193:0/1779504870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d34103dd0 0x7f7d341041e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009910 tx=0x7f7d28009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/1779504870 shutdown_connections 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 --2- 172.21.15.193:0/1779504870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d34103dd0 0x7f7d341041e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/1779504870 >> 172.21.15.193:0/1779504870 conn(0x7f7d340ff660 msgr2=0x7f7d34101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/1779504870 shutdown_connections 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.131+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/1779504870 wait complete. 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.132+0000 7f7d3a7c1700 1 Processor -- start 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.132+0000 7f7d3a7c1700 1 -- start start 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.132+0000 7f7d3a7c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 0x7f7d341a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:21.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.132+0000 7f7d3a7c1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3410e0c0 con 0x7f7d341a5ed0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 0x7f7d341a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 0x7f7d341a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.193:41562/0 (socket says 172.21.15.193:41562) 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d33fff700 1 -- 172.21.15.193:0/236972216 learned_addr learned my addr 172.21.15.193:0/236972216 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d33fff700 1 -- 172.21.15.193:0/236972216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d341a6820 con 0x7f7d341a5ed0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d33fff700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 0x7f7d341a62e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b5c0 tx=0x7f7d28005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d28026070 con 0x7f7d341a5ed0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341a0170 con 0x7f7d341a5ed0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2800b6f0 con 0x7f7d341a5ed0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d28003810 con 0x7f7d341a5ed0 2024-05-14T15:14:21.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.133+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341a0630 con 0x7f7d341a5ed0 2024-05-14T15:14:21.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.134+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34105280 con 0x7f7d341a5ed0 2024-05-14T15:14:21.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.134+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f7d28029070 con 0x7f7d341a5ed0 2024-05-14T15:14:21.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.135+0000 7f7d317fa700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7d1c03e4c0 0x7f7d1c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:21.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.135+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d2801e070 con 0x7f7d341a5ed0 2024-05-14T15:14:21.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.135+0000 7f7d337fe700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7d1c03e4c0 0x7f7d1c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:21.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.135+0000 7f7d337fe700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7d1c03e4c0 0x7f7d1c0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006fd0 tx=0x7f7d24006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:21.138 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.141+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d28013070 con 0x7f7d341a5ed0 2024-05-14T15:14:21.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.358+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) -- 0x7f7d34064e20 con 0x7f7d1c03e4c0 2024-05-14T15:14:21.363 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.365+0000 7f7d317fa700 1 -- 172.21.15.193:0/236972216 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7d34064e20 con 0x7f7d1c03e4c0 2024-05-14T15:14:21.363 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-05-14T15:14:21.365 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.367+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7d1c03e4c0 msgr2=0x7f7d1c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:21.365 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.367+0000 7f7d3a7c1700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7d1c03e4c0 0x7f7d1c0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d24006fd0 tx=0x7f7d24006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 msgr2=0x7f7d341a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 0x7f7d341a62e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b5c0 tx=0x7f7d28005780 comp rx=0 tx=0).stop 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 shutdown_connections 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7d1c03e4c0 0x7f7d1c0409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 --2- 172.21.15.193:0/236972216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d341a5ed0 0x7f7d341a62e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 >> 172.21.15.193:0/236972216 conn(0x7f7d340ff660 msgr2=0x7f7d34075c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 shutdown_connections 2024-05-14T15:14:21.366 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:21.368+0000 7f7d3a7c1700 1 -- 172.21.15.193:0/236972216 wait complete. 2024-05-14T15:14:21.968 DEBUG:teuthology.orchestra.run.smithi144:mon.b> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.b.service 2024-05-14T15:14:21.970 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.c.service 2024-05-14T15:14:21.973 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T15:14:21.973 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T15:14:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c;count:3 2024-05-14T15:14:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:22 smithi005 ceph-mon[31559]: Deploying daemon mon.c on smithi193 2024-05-14T15:14:23.582 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:24.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:24 smithi005 ceph-mon[31559]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:25.175 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 systemd[1]: Starting Ceph mon.c for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:14:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 -- 172.21.15.193:0/1094206864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 msgr2=0x7fde80108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 --2- 172.21.15.193:0/1094206864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde80108e80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009910 tx=0x7fde7c009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 -- 172.21.15.193:0/1094206864 shutdown_connections 2024-05-14T15:14:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 --2- 172.21.15.193:0/1094206864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde80108e80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 -- 172.21.15.193:0/1094206864 >> 172.21.15.193:0/1094206864 conn(0x7fde800fe560 msgr2=0x7fde801009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 -- 172.21.15.193:0/1094206864 shutdown_connections 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.620+0000 7fde881c5700 1 -- 172.21.15.193:0/1094206864 wait complete. 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.621+0000 7fde881c5700 1 Processor -- start 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.621+0000 7fde881c5700 1 -- start start 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde881c5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde8019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde881c5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde8010cfc0 con 0x7fde80108a70 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde85f61700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde8019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde85f61700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde8019e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.193:41570/0 (socket says 172.21.15.193:41570) 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde85f61700 1 -- 172.21.15.193:0/2641992333 learned_addr learned my addr 172.21.15.193:0/2641992333 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde85f61700 1 -- 172.21.15.193:0/2641992333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8019ea70 con 0x7fde80108a70 2024-05-14T15:14:25.621 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.622+0000 7fde85f61700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde8019e530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0056d0 tx=0x7fde7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.623+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde7c027070 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.623+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8019ed60 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.623+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c00be10 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.623+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde80198890 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.623+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde7c022d10 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.624+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde80104180 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.624+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fde7c02a070 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.624+0000 7fde76ffd700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fde6c03e510 0x7fde6c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.624+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fde7c01e030 con 0x7fde80108a70 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.624+0000 7fde85760700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fde6c03e510 0x7fde6c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.625+0000 7fde85760700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fde6c03e510 0x7fde6c040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde70006fd0 tx=0x7fde70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:25.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.630+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde7c013070 con 0x7fde80108a70 2024-05-14T15:14:25.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.923+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fde8019ef90 con 0x7fde80108a70 2024-05-14T15:14:25.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.924+0000 7fde76ffd700 1 -- 172.21.15.193:0/2641992333 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fde7c0292c0 con 0x7fde80108a70 2024-05-14T15:14:25.922 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:14:25.922 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","modified":"2024-05-14T15:12:35.137977Z","created":"2024-05-14T15:12:35.137977Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-14T15:14:25.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fde6c03e510 msgr2=0x7fde6c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:25.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fde6c03e510 0x7fde6c040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde70006fd0 tx=0x7fde70006e40 comp rx=0 tx=0).stop 2024-05-14T15:14:25.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 msgr2=0x7fde8019e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:25.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde8019e530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0056d0 tx=0x7fde7c0057b0 comp rx=0 tx=0).stop 2024-05-14T15:14:25.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 shutdown_connections 2024-05-14T15:14:25.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fde6c03e510 0x7fde6c040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:25.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 --2- 172.21.15.193:0/2641992333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108a70 0x7fde8019e530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:25.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 >> 172.21.15.193:0/2641992333 conn(0x7fde800fe560 msgr2=0x7fde800ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:25.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.926+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 shutdown_connections 2024-05-14T15:14:25.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:25.927+0000 7fde881c5700 1 -- 172.21.15.193:0/2641992333 wait complete. 2024-05-14T15:14:25.925 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-05-14T15:14:25.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 podman[35960]: 2024-05-14 15:14:25.58950042 +0000 UTC m=+0.072943512 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:14:25.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 podman[35960]: 2024-05-14 15:14:25.689593053 +0000 UTC m=+0.173036120 container create 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-05-14T15:14:25.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 podman[35960]: 2024-05-14 15:14:25.925526203 +0000 UTC m=+0.408969275 container init 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-14T15:14:25.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 podman[35960]: 2024-05-14 15:14:25.929436076 +0000 UTC m=+0.412879149 container start 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-14T15:14:26.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T15:14:26.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev), process ceph-mon, pid 2 2024-05-14T15:14:26.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pidfile_write: ignore empty --pid-file 2024-05-14T15:14:26.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: load: jerasure load: lrc 2024-05-14T15:14:26.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: RocksDB version: 7.9.2 2024-05-14T15:14:26.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Git sha 0 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Compile date 2024-05-09 05:50:40 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: DB SUMMARY 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: DB Session ID: W2CEAY7AYTJ8QUFT9T5D 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: CURRENT file: CURRENT 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.error_if_exists: 0 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.create_if_missing: 0 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.paranoid_checks: 1 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T15:14:26.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.env: 0x55ebc1363500 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.info_log: 0x55ebc233a020 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.statistics: (nil) 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.use_fsync: 0 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_log_file_size: 0 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.allow_fallocate: 1 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T15:14:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.use_direct_reads: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.db_log_dir: 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.wal_dir: 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T15:14:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.write_buffer_manager: 0x55ebc22048c0 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.unordered_write: 0 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.row_cache: None 2024-05-14T15:14:26.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.wal_filter: None 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.two_write_queues: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.wal_compression: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.atomic_flush: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.log_readahead_size: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T15:14:26.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T15:14:26.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T15:14:26.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T15:14:26.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T15:14:26.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T15:14:26.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_background_jobs: 2 2024-05-14T15:14:26.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_background_compactions: -1 2024-05-14T15:14:26.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_subcompactions: 1 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_open_files: -1 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T15:14:26.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_background_flushes: -1 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Compression algorithms supported: 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kZSTD supported: 0 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kXpressCompression supported: 0 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kZlibCompression supported: 1 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kSnappyCompression supported: 1 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kLZ4Compression supported: 1 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: kBZip2Compression supported: 0 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T15:14:26.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.merge_operator: 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_filter: None 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ebc233a1e0) 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-05-14T15:14:26.311 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x55ebc22371f0 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-05-14T15:14:26.312 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-05-14T15:14:26.313 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression: NoCompression 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.num_levels: 7 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T15:14:26.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T15:14:26.315 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T15:14:26.316 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T15:14:26.317 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.inplace_update_support: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.bloom_locality: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.max_successive_merges: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T15:14:26.318 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.ttl: 2592000 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.enable_blob_files: false 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.min_blob_size: 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-14T15:14:26.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2bf42b23-46b6-4041-a58e-600cd7a4cb50 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715699665972451, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715699665973065, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715699665, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2bf42b23-46b6-4041-a58e-600cd7a4cb50", "db_session_id": "W2CEAY7AYTJ8QUFT9T5D", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715699665973170, "job": 1, "event": "recovery_finished"} 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ebc221bc00 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: DB pointer 0x55ebc700e000 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T15:14:26.320 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T15:14:26.321 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T15:14:26.322 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x55ebc22371f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(???) e0 preinit fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).mds e1 new map 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-05-14T15:14:26.323 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.324 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Deploying cephadm binary to smithi193 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Added host smithi193 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2725733835' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2725733835' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c;count:3 2024-05-14T15:14:26.327 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: Deploying daemon mon.c on smithi193 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:25 smithi193 ceph-mon[36001]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:26 smithi193 bash[35960]: 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e 2024-05-14T15:14:26.328 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:26 smithi193 systemd[1]: Started Ceph mon.c for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:14:27.738 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T15:14:27.738 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T15:14:27.982 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:14:31.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: mon.a calling monitor election 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: mon.c calling monitor election 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: monmap epoch 2 2024-05-14T15:14:31.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: last_changed 2024-05-14T15:14:25.999457+0000 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: min_mon_release 19 (squid) 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: election_strategy: 1 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: fsmap 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: mgrmap e13: x(active, since 49s) 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: overall HEALTH_OK 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:31.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:31.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: mon.a calling monitor election 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: mon.c calling monitor election 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:31.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: monmap epoch 2 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: last_changed 2024-05-14T15:14:25.999457+0000 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: min_mon_release 19 (squid) 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: election_strategy: 1 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: fsmap 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: mgrmap e13: x(active, since 49s) 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: overall HEALTH_OK 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:31.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:32.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:32.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:32 smithi193 ceph-mon[36001]: Deploying daemon mon.b on smithi144 2024-05-14T15:14:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:32 smithi193 ceph-mon[36001]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:32.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:32.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:32.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:32.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:32 smithi005 ceph-mon[31559]: Deploying daemon mon.b on smithi144 2024-05-14T15:14:32.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:32 smithi005 ceph-mon[31559]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:32.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.549+0000 7f2752cec700 1 -- 172.21.15.193:0/442846412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27380035c0 msgr2=0x7f2738005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.549+0000 7f2752cec700 1 --2- 172.21.15.193:0/442846412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27380035c0 0x7f2738005aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f274c06ab00 tx=0x7f2744009e80 comp rx=0 tx=0).stop 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.550+0000 7f2752cec700 1 -- 172.21.15.193:0/442846412 shutdown_connections 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.550+0000 7f2752cec700 1 --2- 172.21.15.193:0/442846412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27380035c0 0x7f2738005aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.550+0000 7f2752cec700 1 -- 172.21.15.193:0/442846412 >> 172.21.15.193:0/442846412 conn(0x7f274c0fdfa0 msgr2=0x7f274c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.550+0000 7f2752cec700 1 -- 172.21.15.193:0/442846412 shutdown_connections 2024-05-14T15:14:32.548 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.550+0000 7f2752cec700 1 -- 172.21.15.193:0/442846412 wait complete. 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2752cec700 1 Processor -- start 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2752cec700 1 -- start start 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2752cec700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2752cec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f274c19eb20 0x7f274c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2752cec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10aae0 con 0x7f274c19eb20 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2752cec700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f274c10a960 con 0x7f27380035c0 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.551+0000 7f2750a88700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2750a88700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:38370/0 (socket says 172.21.15.193:38370) 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2750a88700 1 -- 172.21.15.193:0/3243808164 learned_addr learned my addr 172.21.15.193:0/3243808164 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f274bfff700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f274c19eb20 0x7f274c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2750a88700 1 -- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 msgr2=0x7f274c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T15:14:32.550 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2750a88700 1 -- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 msgr2=0x7f274c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T15:14:32.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2750a88700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T15:14:32.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2750a88700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:14:32.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f274bfff700 1 -- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 msgr2=0x7f274c19e5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:14:32.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f274bfff700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:32.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f274bfff700 1 -- 172.21.15.193:0/3243808164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c19f0c0 con 0x7f274c19eb20 2024-05-14T15:14:32.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f274bfff700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f274c19eb20 0x7f274c198770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f273400b4d0 tx=0x7f273400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:32.552 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2734011630 con 0x7f274c19eb20 2024-05-14T15:14:32.552 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2734011c80 con 0x7f274c19eb20 2024-05-14T15:14:32.552 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.552+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c198d70 con 0x7f274c19eb20 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.553+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2734010da0 con 0x7f274c19eb20 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.553+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c199380 con 0x7f274c19eb20 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.554+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f2734018020 con 0x7f274c19eb20 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.554+0000 7f2749ffb700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f272c03e320 0x7f272c040830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.554+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2734053090 con 0x7f274c19eb20 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.554+0000 7f2750a88700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f272c03e320 0x7f272c040830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.554+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c104400 con 0x7f274c19eb20 2024-05-14T15:14:32.553 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.555+0000 7f2750a88700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f272c03e320 0x7f272c040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f274401c040 tx=0x7f27440091d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:32.557 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.560+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f273401f020 con 0x7f274c19eb20 2024-05-14T15:14:32.862 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.865+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f274c064e20 con 0x7f274c19eb20 2024-05-14T15:14:32.863 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.866+0000 7f2749ffb700 1 -- 172.21.15.193:0/3243808164 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f2734010410 con 0x7f274c19eb20 2024-05-14T15:14:32.864 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:14:32.864 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","modified":"2024-05-14T15:14:25.999457Z","created":"2024-05-14T15:12:35.137977Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T15:14:32.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f272c03e320 msgr2=0x7f272c040830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:32.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f272c03e320 0x7f272c040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f274401c040 tx=0x7f27440091d0 comp rx=0 tx=0).stop 2024-05-14T15:14:32.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f274c19eb20 msgr2=0x7f274c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:32.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f274c19eb20 0x7f274c198770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f273400b4d0 tx=0x7f273400b890 comp rx=0 tx=0).stop 2024-05-14T15:14:32.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 shutdown_connections 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f272c03e320 0x7f272c040830 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27380035c0 0x7f274c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 --2- 172.21.15.193:0/3243808164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f274c19eb20 0x7f274c198770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 >> 172.21.15.193:0/3243808164 conn(0x7f274c0fdfa0 msgr2=0x7f274c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 shutdown_connections 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:32.868+0000 7f2752cec700 1 -- 172.21.15.193:0/3243808164 wait complete. 2024-05-14T15:14:32.867 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-05-14T15:14:33.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:33 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/3243808164' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T15:14:33.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:33 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/3243808164' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T15:14:34.450 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T15:14:34.450 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T15:14:34.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:34 smithi193 ceph-mon[36001]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:34.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:34 smithi005 ceph-mon[31559]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:34.695 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:14:35.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:34 smithi144 systemd[1]: Started Ceph mon.b for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: mon.a calling monitor election 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: mon.c calling monitor election 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: mon.b calling monitor election 2024-05-14T15:14:40.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: monmap epoch 3 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: last_changed 2024-05-14T15:14:34.984053+0000 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: min_mon_release 19 (squid) 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: election_strategy: 1 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:14:40.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: fsmap 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: mgrmap e13: x(active, since 58s) 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: overall HEALTH_OK 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:14:40.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: mon.a calling monitor election 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: mon.c calling monitor election 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: mon.b calling monitor election 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: monmap epoch 3 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:40.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: last_changed 2024-05-14T15:14:34.984053+0000 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: min_mon_release 19 (squid) 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: election_strategy: 1 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: fsmap 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: mgrmap e13: x(active, since 58s) 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: overall HEALTH_OK 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.544 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: mon.a calling monitor election 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: mon.c calling monitor election 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: mon.b calling monitor election 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: monmap epoch 3 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: last_changed 2024-05-14T15:14:34.984053+0000 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: created 2024-05-14T15:12:35.137977+0000 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: min_mon_release 19 (squid) 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: election_strategy: 1 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: fsmap 2024-05-14T15:14:40.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T15:14:40.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: mgrmap e13: x(active, since 58s) 2024-05-14T15:14:40.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: overall HEALTH_OK 2024-05-14T15:14:40.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:40.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 -- 172.21.15.193:0/1041325507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d40035c0 msgr2=0x7f77d4005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:40.624 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 --2- 172.21.15.193:0/1041325507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d40035c0 0x7f77d4005aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f77e8102a70 tx=0x7f77e400fe60 comp rx=0 tx=0).stop 2024-05-14T15:14:40.625 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 -- 172.21.15.193:0/1041325507 shutdown_connections 2024-05-14T15:14:40.625 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 --2- 172.21.15.193:0/1041325507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d40035c0 0x7f77d4005aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.625 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 -- 172.21.15.193:0/1041325507 >> 172.21.15.193:0/1041325507 conn(0x7f77e80797e0 msgr2=0x7f77e8079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:40.625 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 -- 172.21.15.193:0/1041325507 shutdown_connections 2024-05-14T15:14:40.625 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.626+0000 7f77efdfb700 1 -- 172.21.15.193:0/1041325507 wait complete. 2024-05-14T15:14:40.626 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 Processor -- start 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 -- start start 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 0x7f77e8105660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 0x7f77e8105fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 0x7f77e81a8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810aaa0 con 0x7f77e8105ba0 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810a920 con 0x7f77e81a5e30 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.627+0000 7f77efdfb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810ac20 con 0x7f77d40035c0 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 0x7f77e8105fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 0x7f77e8105fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.193:49464/0 (socket says 172.21.15.193:49464) 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 -- 172.21.15.193:0/867094076 learned_addr learned my addr 172.21.15.193:0/867094076 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77edb97700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 0x7f77e8105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ee398700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 0x7f77e81a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:40.627 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77edb97700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 msgr2=0x7f77e8105660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77edb97700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 msgr2=0x7f77e8105660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77edb97700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 0x7f77e8105660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ee398700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 msgr2=0x7f77e81a8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ee398700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 msgr2=0x7f77e81a8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ee398700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 0x7f77e81a8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77edb97700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 0x7f77e8105660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ee398700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 0x7f77e81a8250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 msgr2=0x7f77e8105660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 0x7f77e8105660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 msgr2=0x7f77e81a8250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 0x7f77e81a8250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 -- 172.21.15.193:0/867094076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81a88d0 con 0x7f77e8105ba0 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77ed396700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 0x7f77e8105fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77d800b780 tx=0x7f77d800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:40.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.628+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004230 con 0x7f77e8105ba0 2024-05-14T15:14:40.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.629+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d80043d0 con 0x7f77e8105ba0 2024-05-14T15:14:40.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.629+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e81a8bc0 con 0x7f77e8105ba0 2024-05-14T15:14:40.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.629+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8011bd0 con 0x7f77e8105ba0 2024-05-14T15:14:40.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.629+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81a9100 con 0x7f77e8105ba0 2024-05-14T15:14:40.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.630+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e8064e20 con 0x7f77e8105ba0 2024-05-14T15:14:40.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.630+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f77d801a400 con 0x7f77e8105ba0 2024-05-14T15:14:40.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.630+0000 7f77deffd700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f77cc03e9a0 0x7f77cc040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:40.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.630+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f77d80523b0 con 0x7f77e8105ba0 2024-05-14T15:14:40.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.630+0000 7f77edb97700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f77cc03e9a0 0x7f77cc040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:40.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.631+0000 7f77edb97700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f77cc03e9a0 0x7f77cc040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77e400fa10 tx=0x7f77e400f4e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:40.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.636+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77d8020740 con 0x7f77e8105ba0 2024-05-14T15:14:40.932 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.934+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f77e81a94b0 con 0x7f77e8105ba0 2024-05-14T15:14:40.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.935+0000 7f77deffd700 1 -- 172.21.15.193:0/867094076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f77d801a840 con 0x7f77e8105ba0 2024-05-14T15:14:40.933 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-05-14T15:14:40.933 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","modified":"2024-05-14T15:14:34.984053Z","created":"2024-05-14T15:12:35.137977Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-14T15:14:40.935 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f77cc03e9a0 msgr2=0x7f77cc040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:40.935 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f77cc03e9a0 0x7f77cc040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77e400fa10 tx=0x7f77e400f4e0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.935 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 msgr2=0x7f77e8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:40.935 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 0x7f77e8105fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77d800b780 tx=0x7f77d800bb40 comp rx=0 tx=0).stop 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 shutdown_connections 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f77cc03e9a0 0x7f77cc040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81a5e30 0x7f77e81a8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77d40035c0 0x7f77e8105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 --2- 172.21.15.193:0/867094076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8105ba0 0x7f77e8105fb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 >> 172.21.15.193:0/867094076 conn(0x7f77e80797e0 msgr2=0x7f77e8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 shutdown_connections 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:40.937+0000 7f77efdfb700 1 -- 172.21.15.193:0/867094076 wait complete. 2024-05-14T15:14:40.936 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-05-14T15:14:41.272 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:41.272 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:41 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/867094076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T15:14:41.272 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:41.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:41.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:41 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/867094076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T15:14:41.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:41.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:41.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:41 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/867094076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T15:14:41.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:14:41.564 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-14T15:14:41.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-14T15:14:41.820 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:42.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:42.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:42.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:42.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:42.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:42.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:42.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:42.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:42.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 -- 172.21.15.5:0/2393608799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb918108a60 msgr2=0x7fb918108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 --2- 172.21.15.5:0/2393608799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb918108a60 0x7fb918108e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb908009910 tx=0x7fb908009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 -- 172.21.15.5:0/2393608799 shutdown_connections 2024-05-14T15:14:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 --2- 172.21.15.5:0/2393608799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb918108a60 0x7fb918108e70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 -- 172.21.15.5:0/2393608799 >> 172.21.15.5:0/2393608799 conn(0x7fb9180fe570 msgr2=0x7fb9181009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 -- 172.21.15.5:0/2393608799 shutdown_connections 2024-05-14T15:14:43.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.481+0000 7fb91e492700 1 -- 172.21.15.5:0/2393608799 wait complete. 2024-05-14T15:14:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.482+0000 7fb91e492700 1 Processor -- start 2024-05-14T15:14:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.482+0000 7fb91e492700 1 -- start start 2024-05-14T15:14:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.482+0000 7fb91e492700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb918108a60 0x7fb918075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:43.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.482+0000 7fb91e492700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 0x7fb91806bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb91e492700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb91806c530 0x7fb9181b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb91e492700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810cfb0 con 0x7fb91806bb40 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb91e492700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810ce30 con 0x7fb91806c530 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb91e492700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810d130 con 0x7fb918108a60 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 0x7fb91806bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 0x7fb91806bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48252/0 (socket says 172.21.15.5:48252) 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 -- 172.21.15.5:0/3243657154 learned_addr learned my addr 172.21.15.5:0/3243657154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb917fff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb918108a60 0x7fb918075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb91ca2f700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb91806c530 0x7fb9181b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:43.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 -- 172.21.15.5:0/3243657154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb918108a60 msgr2=0x7fb918075490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb918108a60 0x7fb918075490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 -- 172.21.15.5:0/3243657154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb91806c530 msgr2=0x7fb9181b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb91806c530 0x7fb9181b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 -- 172.21.15.5:0/3243657154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9181b1040 con 0x7fb91806bb40 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb917fff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb918108a60 0x7fb918075490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.483+0000 7fb90ffff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 0x7fb91806bf50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b780 tx=0x7fb90000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.484+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900016070 con 0x7fb91806bb40 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.484+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9000036a0 con 0x7fb91806bb40 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.484+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000119c0 con 0x7fb91806bb40 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.484+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181b1330 con 0x7fb91806bb40 2024-05-14T15:14:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.484+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181b17c0 con 0x7fb91806bb40 2024-05-14T15:14:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.485+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918104170 con 0x7fb91806bb40 2024-05-14T15:14:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.485+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fb900019070 con 0x7fb91806bb40 2024-05-14T15:14:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.485+0000 7fb915ffb700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb90403e9a0 0x7fb904040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.485+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb900051260 con 0x7fb91806bb40 2024-05-14T15:14:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.486+0000 7fb917fff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb90403e9a0 0x7fb904040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:43.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.486+0000 7fb917fff700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb90403e9a0 0x7fb904040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb908009750 tx=0x7fb90801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:43.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.491+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb900019360 con 0x7fb91806bb40 2024-05-14T15:14:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.706+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb9181b1bf0 con 0x7fb91806bb40 2024-05-14T15:14:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.706+0000 7fb915ffb700 1 -- 172.21.15.5:0/3243657154 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fb90001d030 con 0x7fb91806bb40 2024-05-14T15:14:43.705 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:43.705 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-14T15:14:43.705 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:14:43.705 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-05-14T15:14:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb90403e9a0 msgr2=0x7fb904040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb90403e9a0 0x7fb904040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb908009750 tx=0x7fb90801a040 comp rx=0 tx=0).stop 2024-05-14T15:14:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 msgr2=0x7fb91806bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 0x7fb91806bf50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b780 tx=0x7fb90000ba90 comp rx=0 tx=0).stop 2024-05-14T15:14:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 shutdown_connections 2024-05-14T15:14:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb90403e9a0 0x7fb904040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb91806c530 0x7fb9181b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb918108a60 0x7fb918075490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 --2- 172.21.15.5:0/3243657154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb91806bb40 0x7fb91806bf50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.708+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 >> 172.21.15.5:0/3243657154 conn(0x7fb9180fe570 msgr2=0x7fb9180ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.709+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 shutdown_connections 2024-05-14T15:14:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:43.709+0000 7fb91e492700 1 -- 172.21.15.5:0/3243657154 wait complete. 2024-05-14T15:14:43.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:14:43.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:43.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T15:14:43.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:43.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:43.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:43.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:43 smithi005 ceph-mon[31559]: Reconfiguring daemon mon.a on smithi005 2024-05-14T15:14:44.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:14:44.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:14:44.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T15:14:44.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:44.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:44.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:44.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:43 smithi144 ceph-mon[35650]: Reconfiguring daemon mon.a on smithi005 2024-05-14T15:14:44.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:14:44.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:14:44.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:44.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T15:14:44.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:44.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:44.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:44.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:43 smithi193 ceph-mon[36001]: Reconfiguring daemon mon.a on smithi005 2024-05-14T15:14:44.705 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-14T15:14:44.706 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:14:44.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T15:14:44.736 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:14:44.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:44.804 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:14:44.804 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T15:14:44.836 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:14:44.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:44.900 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:44 smithi144 ceph-mon[35650]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:44.900 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:44 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3243657154' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:44.904 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:14:44.904 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T15:14:44.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:44 smithi193 ceph-mon[36001]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:44.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:44 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3243657154' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:44.936 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:14:44.936 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:14:44.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:44 smithi005 ceph-mon[31559]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:44.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:44 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3243657154' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:45.003 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-05-14T15:14:45.003 INFO:tasks.cephadm:Adding mgr.y on smithi144 2024-05-14T15:14:45.004 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-05-14T15:14:45.004 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi144=y;smithi193=z' 2024-05-14T15:14:45.279 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:14:46.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:46.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:46.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: Reconfiguring mon.b (monmap changed)... 2024-05-14T15:14:46.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:46.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:46.493 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:46.494 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: Reconfiguring daemon mon.b on smithi144 2024-05-14T15:14:46.494 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:46 smithi193 ceph-mon[36001]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:46.544 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: Reconfiguring mon.b (monmap changed)... 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: Reconfiguring daemon mon.b on smithi144 2024-05-14T15:14:46.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:46 smithi144 ceph-mon[35650]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:46.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:46.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:46.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: Reconfiguring mon.b (monmap changed)... 2024-05-14T15:14:46.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:46.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:46.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:46.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: Reconfiguring daemon mon.b on smithi144 2024-05-14T15:14:46.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:46 smithi005 ceph-mon[31559]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 -- 172.21.15.193:0/79607633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 msgr2=0x7facb4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 --2- 172.21.15.193:0/79607633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 0x7facb4103000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faca8009910 tx=0x7faca8009c20 comp rx=0 tx=0).stop 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 -- 172.21.15.193:0/79607633 shutdown_connections 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 --2- 172.21.15.193:0/79607633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 0x7facb4103000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 -- 172.21.15.193:0/79607633 >> 172.21.15.193:0/79607633 conn(0x7facb40fe250 msgr2=0x7facb41006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 -- 172.21.15.193:0/79607633 shutdown_connections 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.867+0000 7facbb069700 1 -- 172.21.15.193:0/79607633 wait complete. 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 Processor -- start 2024-05-14T15:14:46.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 -- start start 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 0x7facb406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 0x7facb419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb419f0a0 0x7facb41a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facb4108bb0 con 0x7facb4102bf0 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facb4108a30 con 0x7facb419f0a0 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facbb069700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7facb4108d30 con 0x7facb419c740 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facb3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 0x7facb419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facb3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 0x7facb419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:36522/0 (socket says 172.21.15.193:36522) 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facb3fff700 1 -- 172.21.15.193:0/3492014011 learned_addr learned my addr 172.21.15.193:0/3492014011 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facb8e05700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 0x7facb406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.868+0000 7facb9606700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb419f0a0 0x7facb41a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb3fff700 1 -- 172.21.15.193:0/3492014011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb419f0a0 msgr2=0x7facb41a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb3fff700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb419f0a0 0x7facb41a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:46.868 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb3fff700 1 -- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 msgr2=0x7facb406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb3fff700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 0x7facb406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb3fff700 1 -- 172.21.15.193:0/3492014011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb41a1d10 con 0x7facb419c740 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb3fff700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 0x7facb419eb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faca000b780 tx=0x7faca000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0016070 con 0x7facb419c740 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca00036a0 con 0x7facb419c740 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb41a1fa0 con 0x7facb419c740 2024-05-14T15:14:46.869 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca00119c0 con 0x7facb419c740 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.869+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a24d0 con 0x7facb419c740 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.870+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7faca0019070 con 0x7facb419c740 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.870+0000 7facb1ffb700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7faca403e9a0 0x7faca4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.870+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4064e50 con 0x7facb419c740 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.870+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faca0003850 con 0x7facb419c740 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.870+0000 7facb8e05700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7faca403e9a0 0x7faca4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:46.870 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.871+0000 7facb8e05700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7faca403e9a0 0x7faca4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faca8006010 tx=0x7faca801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:46.873 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:46.876+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7facb4064e50 con 0x7facb419c740 2024-05-14T15:14:47.097 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.099+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi144=y;smithi193=z", "target": ["mon-mgr", ""]}) -- 0x7facb41000a0 con 0x7faca403e9a0 2024-05-14T15:14:47.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.106+0000 7facb1ffb700 1 -- 172.21.15.193:0/3492014011 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7facb41000a0 con 0x7faca403e9a0 2024-05-14T15:14:47.104 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-05-14T15:14:47.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7faca403e9a0 msgr2=0x7faca4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:47.106 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7faca403e9a0 0x7faca4040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faca8006010 tx=0x7faca801a040 comp rx=0 tx=0).stop 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 msgr2=0x7facb419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 0x7facb419eb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faca000b780 tx=0x7faca000ba90 comp rx=0 tx=0).stop 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 shutdown_connections 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7faca403e9a0 0x7faca4040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb419f0a0 0x7facb41a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facb4102bf0 0x7facb406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 --2- 172.21.15.193:0/3492014011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facb419c740 0x7facb419eb60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.108+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 >> 172.21.15.193:0/3492014011 conn(0x7facb40fe250 msgr2=0x7facb40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.109+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 shutdown_connections 2024-05-14T15:14:47.107 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:14:47.109+0000 7facbb069700 1 -- 172.21.15.193:0/3492014011 wait complete. 2024-05-14T15:14:47.730 DEBUG:teuthology.orchestra.run.smithi144:mgr.y> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.y.service 2024-05-14T15:14:47.733 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.z.service 2024-05-14T15:14:47.735 INFO:tasks.cephadm:Deploying OSDs... 2024-05-14T15:14:47.735 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:14:47.735 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T15:14:47.756 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T15:14:47.756 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-05-14T15:14:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T15:14:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-14 15:13:53.563339574 +0000 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-14 15:08:53.549280231 +0000 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-14 15:08:53.549280231 +0000 2024-05-14T15:14:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-14 15:08:53.549280231 +0000 2024-05-14T15:14:47.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T15:14:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-14T15:14:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-14T15:14:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000131266 s, 3.9 MB/s 2024-05-14T15:14:47.889 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T15:14:47.948 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: Reconfiguring mon.c (monmap changed)... 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: Reconfiguring daemon mon.c on smithi193 2024-05-14T15:14:47.978 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-14 15:13:53.607338687 +0000 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-14 15:08:53.895273079 +0000 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-14 15:08:53.895273079 +0000 2024-05-14T15:14:48.007 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-14 15:08:53.895273079 +0000 2024-05-14T15:14:48.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T15:14:48.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: Reconfiguring mon.c (monmap changed)... 2024-05-14T15:14:48.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:48.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:48.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:48.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: Reconfiguring daemon mon.c on smithi193 2024-05-14T15:14:48.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: Reconfiguring mon.c (monmap changed)... 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:48.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: Reconfiguring daemon mon.c on smithi193 2024-05-14T15:14:48.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:48.073 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-14T15:14:48.073 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-14T15:14:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000122443 s, 4.2 MB/s 2024-05-14T15:14:48.075 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T15:14:48.135 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-14 15:13:53.643337961 +0000 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-14 15:08:54.236266031 +0000 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-14 15:08:54.236266031 +0000 2024-05-14T15:14:48.193 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-14 15:08:54.236266031 +0000 2024-05-14T15:14:48.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T15:14:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-14T15:14:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-14T15:14:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00011919 s, 4.3 MB/s 2024-05-14T15:14:48.261 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T15:14:48.319 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-05-14T15:14:48.376 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T15:14:48.376 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:48.376 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-14T15:14:48.377 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:48.377 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:48.377 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-14 15:13:53.678337255 +0000 2024-05-14T15:14:48.377 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-14 15:08:54.566259210 +0000 2024-05-14T15:14:48.377 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-14 15:08:54.566259210 +0000 2024-05-14T15:14:48.377 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-14 15:08:54.566259210 +0000 2024-05-14T15:14:48.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T15:14:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-14T15:14:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-14T15:14:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000110011 s, 4.7 MB/s 2024-05-14T15:14:48.444 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T15:14:48.502 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-05-14T15:14:48.502 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T15:14:48.521 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T15:14:48.522 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-05-14T15:14:48.579 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T15:14:48.579 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:48.579 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-14T15:14:48.579 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:48.579 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:48.580 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-14 15:14:34.400575832 +0000 2024-05-14T15:14:48.580 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-14 15:09:00.872408630 +0000 2024-05-14T15:14:48.580 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-14 15:09:00.872408630 +0000 2024-05-14T15:14:48.580 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-14 15:09:00.872408630 +0000 2024-05-14T15:14:48.581 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T15:14:48.644 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-14T15:14:48.644 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-14T15:14:48.645 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000282277 s, 1.8 MB/s 2024-05-14T15:14:48.646 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T15:14:48.704 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-05-14T15:14:48.760 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-14 15:14:34.401575816 +0000 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-14 15:09:01.225402876 +0000 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-14 15:09:01.225402876 +0000 2024-05-14T15:14:48.761 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-14 15:09:01.225402876 +0000 2024-05-14T15:14:48.761 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T15:14:48.825 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-14T15:14:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-14T15:14:48.826 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000204965 s, 2.5 MB/s 2024-05-14T15:14:48.827 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T15:14:48.883 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-14 15:14:34.401575816 +0000 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-14 15:09:01.561397399 +0000 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-14 15:09:01.561397399 +0000 2024-05-14T15:14:48.940 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-14 15:09:01.561397399 +0000 2024-05-14T15:14:48.941 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T15:14:49.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:48 smithi144 ceph-mon[35650]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi144=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:49.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:48 smithi144 ceph-mon[35650]: Saving service mgr spec with placement smithi005=x;smithi144=y;smithi193=z;count:3 2024-05-14T15:14:49.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:48 smithi144 ceph-mon[35650]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:49.003 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-14T15:14:49.003 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-14T15:14:49.003 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000208738 s, 2.5 MB/s 2024-05-14T15:14:49.004 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T15:14:49.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:48 smithi193 ceph-mon[36001]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi144=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:49.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:48 smithi193 ceph-mon[36001]: Saving service mgr spec with placement smithi005=x;smithi144=y;smithi193=z;count:3 2024-05-14T15:14:49.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:48 smithi193 ceph-mon[36001]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:49.060 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-05-14T15:14:49.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:48 smithi005 ceph-mon[31559]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi144=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:49.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:48 smithi005 ceph-mon[31559]: Saving service mgr spec with placement smithi005=x;smithi144=y;smithi193=z;count:3 2024-05-14T15:14:49.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:48 smithi005 ceph-mon[31559]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-05-14 15:14:34.401575816 +0000 2024-05-14T15:14:49.115 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-05-14 15:09:01.896391938 +0000 2024-05-14T15:14:49.116 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-05-14 15:09:01.896391938 +0000 2024-05-14T15:14:49.116 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-05-14 15:09:01.896391938 +0000 2024-05-14T15:14:49.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T15:14:49.177 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-05-14T15:14:49.177 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-05-14T15:14:49.178 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000194516 s, 2.6 MB/s 2024-05-14T15:14:49.179 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T15:14:49.235 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-05-14T15:14:49.236 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T15:14:49.253 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T15:14:49.253 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-05-14T15:14:49.309 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T15:14:49.309 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:49.309 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-14T15:14:49.310 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:49.310 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:49.310 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 15:14:24.994816842 +0000 2024-05-14T15:14:49.310 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 15:09:02.128074610 +0000 2024-05-14T15:14:49.310 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 15:09:02.128074610 +0000 2024-05-14T15:14:49.310 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 15:09:02.128074610 +0000 2024-05-14T15:14:49.311 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T15:14:49.373 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T15:14:49.373 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T15:14:49.373 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000278134 s, 1.8 MB/s 2024-05-14T15:14:49.374 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T15:14:49.431 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-05-14T15:14:49.486 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 15:14:24.995816826 +0000 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 15:09:02.523068441 +0000 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 15:09:02.523068441 +0000 2024-05-14T15:14:49.487 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 15:09:02.523068441 +0000 2024-05-14T15:14:49.487 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T15:14:49.550 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T15:14:49.550 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T15:14:49.550 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000218328 s, 2.3 MB/s 2024-05-14T15:14:49.552 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T15:14:49.608 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-05-14T15:14:49.662 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 15:14:24.995816826 +0000 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 15:09:02.887062756 +0000 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 15:09:02.887062756 +0000 2024-05-14T15:14:49.663 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 15:09:02.887062756 +0000 2024-05-14T15:14:49.663 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T15:14:49.725 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T15:14:49.725 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T15:14:49.725 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000195469 s, 2.6 MB/s 2024-05-14T15:14:49.726 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T15:14:49.782 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-05-14T15:14:49.839 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T15:14:49.839 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-05-14 15:14:24.996816811 +0000 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-05-14 15:09:03.215057634 +0000 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-05-14 15:09:03.215057634 +0000 2024-05-14T15:14:49.840 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-05-14 15:09:03.215057634 +0000 2024-05-14T15:14:49.841 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T15:14:49.909 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-05-14T15:14:49.909 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-05-14T15:14:49.909 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000179402 s, 2.9 MB/s 2024-05-14T15:14:49.910 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T15:14:49.971 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-05-14T15:14:49.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T15:14:50.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:50.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:50.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:50.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:50.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:50.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:50.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T15:14:50.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:50.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:50.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:50.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:51.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:50 smithi193 ceph-mon[36001]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:51.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:50 smithi193 ceph-mon[36001]: Deploying daemon mgr.z on smithi193 2024-05-14T15:14:51.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:50 smithi005 ceph-mon[31559]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:51.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:50 smithi005 ceph-mon[31559]: Deploying daemon mgr.z on smithi193 2024-05-14T15:14:51.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:50 smithi144 ceph-mon[35650]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:51.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:50 smithi144 ceph-mon[35650]: Deploying daemon mgr.z on smithi193 2024-05-14T15:14:51.995 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:51 smithi193 systemd[1]: Started Ceph mgr.z for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:14:52.276 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:51.997+0000 7fd149501200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:14:52.277 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:52.179+0000 7fd149501200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:14:52.277 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:52.278+0000 7fd149501200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:14:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T15:14:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T15:14:52.326 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-14T15:14:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-14T15:14:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00861731 s, 1.2 GB/s 2024-05-14T15:14:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-14T15:14:52.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:52.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:52.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T15:14:52.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:52.876 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:52.897 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:52.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-05-14T15:14:52.973 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:52.582+0000 7fd149501200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:14:52.973 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:52.665+0000 7fd149501200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:14:52.973 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:52.973 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.973 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:52.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:52.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T15:14:52.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:52.974 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:14:53.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:53 smithi005 ceph-mon[31559]: Deploying daemon mgr.y on smithi144 2024-05-14T15:14:53.979 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:53 smithi144 ceph-mon[35650]: Deploying daemon mgr.y on smithi144 2024-05-14T15:14:54.201 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:53 smithi193 ceph-mon[36001]: Deploying daemon mgr.y on smithi144 2024-05-14T15:14:54.202 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:53.879+0000 7fd149501200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:14:54.202 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:54.050+0000 7fd149501200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:14:54.294 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 systemd[1]: Starting Ceph mgr.y for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:14:54.511 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:54.231+0000 7fd149501200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:14:54.512 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:54.513+0000 7fd149501200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:14:54.654 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 podman[37022]: 2024-05-14 15:14:54.277180748 +0000 UTC m=+0.016846520 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:14:54.654 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 podman[37022]: 2024-05-14 15:14:54.415516839 +0000 UTC m=+0.155182641 container create 03589df7a821feba698be56207ea00af434302c8d17610774ff95e287ac66255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T15:14:54.654 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 podman[37022]: 2024-05-14 15:14:54.617997946 +0000 UTC m=+0.357663714 container init 03589df7a821feba698be56207ea00af434302c8d17610774ff95e287ac66255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-14T15:14:54.654 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 podman[37022]: 2024-05-14 15:14:54.620653035 +0000 UTC m=+0.360318799 container start 03589df7a821feba698be56207ea00af434302c8d17610774ff95e287ac66255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-14T15:14:54.654 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 ceph-mgr[37041]: -- 172.21.15.144:0/279799251 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ee7e4a5a0 con 0x557eecbd4480 2024-05-14T15:14:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 -- 172.21.15.5:0/1403647627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105e40 msgr2=0x7f1d8c10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 --2- 172.21.15.5:0/1403647627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105e40 0x7f1d8c10c6f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400b0a0 tx=0x7f1d8400b3b0 comp rx=0 tx=0).stop 2024-05-14T15:14:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 -- 172.21.15.5:0/1403647627 shutdown_connections 2024-05-14T15:14:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 --2- 172.21.15.5:0/1403647627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105e40 0x7f1d8c10c6f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 --2- 172.21.15.5:0/1403647627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d8c104370 0x7f1d8c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 --2- 172.21.15.5:0/1403647627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d8c1054a0 0x7f1d8c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 -- 172.21.15.5:0/1403647627 >> 172.21.15.5:0/1403647627 conn(0x7f1d8c0ff9d0 msgr2=0x7f1d8c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:14:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 -- 172.21.15.5:0/1403647627 shutdown_connections 2024-05-14T15:14:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.690+0000 7f1d946d6700 1 -- 172.21.15.5:0/1403647627 wait complete. 2024-05-14T15:14:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 Processor -- start 2024-05-14T15:14:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 -- start start 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 0x7f1d8c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d8c1054a0 0x7f1d8c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d8c105e40 0x7f1d8c1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10fde0 con 0x7f1d8c104370 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10fc60 con 0x7f1d8c105e40 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d946d6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8c10ff60 con 0x7f1d8c1054a0 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 0x7f1d8c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 0x7f1d8c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55618/0 (socket says 172.21.15.5:55618) 2024-05-14T15:14:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 -- 172.21.15.5:0/2010942727 learned_addr learned my addr 172.21.15.5:0/2010942727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d91c71700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d8c1054a0 0x7f1d8c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92c73700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d8c105e40 0x7f1d8c1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 -- 172.21.15.5:0/2010942727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d8c1054a0 msgr2=0x7f1d8c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d8c1054a0 0x7f1d8c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 -- 172.21.15.5:0/2010942727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d8c105e40 msgr2=0x7f1d8c1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d8c105e40 0x7f1d8c1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.691+0000 7f1d92472700 1 -- 172.21.15.5:0/2010942727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8c1a23d0 con 0x7f1d8c104370 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.692+0000 7f1d92472700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 0x7f1d8c19a870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800e940 tx=0x7f1d8800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.692+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88004220 con 0x7f1d8c104370 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.692+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d880043c0 con 0x7f1d8c104370 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.692+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88003c60 con 0x7f1d8c104370 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.692+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8c1a26c0 con 0x7f1d8c104370 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.692+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8c1a2bb0 con 0x7f1d8c104370 2024-05-14T15:14:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.693+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f1d88018430 con 0x7f1d8c104370 2024-05-14T15:14:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.693+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d8c0512c0 con 0x7f1d8c104370 2024-05-14T15:14:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.693+0000 7f1d837fe700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1d7803e750 0x7f1d78040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:14:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.693+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d88052c50 con 0x7f1d8c104370 2024-05-14T15:14:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.693+0000 7f1d91c71700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1d7803e750 0x7f1d78040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:14:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.694+0000 7f1d91c71700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1d7803e750 0x7f1d78040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c19b920 tx=0x7f1d7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:14:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.699+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d88003e40 con 0x7f1d8c104370 2024-05-14T15:14:54.800 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:54.605+0000 7fd149501200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:14:54.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:54.931+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1d8c063c00 con 0x7f1d7803e750 2024-05-14T15:14:54.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:54 smithi005 ceph-mon[31559]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:54.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.972 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:54 smithi144 ceph-mon[35650]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:54.972 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.972 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.972 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.972 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:54.972 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 bash[37022]: 03589df7a821feba698be56207ea00af434302c8d17610774ff95e287ac66255 2024-05-14T15:14:54.973 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 systemd[1]: Started Ceph mgr.y for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:14:54.973 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:54.786+0000 7fbd7738e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:14:54.973 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:54.974+0000 7fbd7738e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:14:55.294 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:55.084+0000 7fbd7738e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:14:55.300 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:54.808+0000 7fd149501200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:14:55.300 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:55.005+0000 7fd149501200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:14:55.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:54 smithi193 ceph-mon[36001]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:55.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:55.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:55.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:55.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:55.670 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:55.412+0000 7fbd7738e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:14:55.671 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:55.514+0000 7fbd7738e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:14:55.999 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:55.634+0000 7fd149501200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:14:56.000 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:55.857+0000 7fd149501200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:14:56.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:55 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:56.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:55 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:14:56.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:55 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:14:56.000 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:55 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:56.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:55 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:56.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:55 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:14:56.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:55 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:14:56.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:55 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:56.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:14:56.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:14:56.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:14:56.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:56.550 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:56.291+0000 7fd149501200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:14:56.550 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:56.395+0000 7fd149501200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:14:57.044 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:56.731+0000 7fbd7738e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:14:57.044 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:56.899+0000 7fbd7738e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:57.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:57.050 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:56.771+0000 7fd149501200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:14:57.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:57.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:57.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:14:57.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:57.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T15:14:57.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:14:57.350 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:57.070+0000 7fbd7738e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:14:57.350 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:57.352+0000 7fbd7738e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:14:57.463 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:57.127+0000 7fd149501200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:14:57.463 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:57.220+0000 7fd149501200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:14:57.645 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:57.443+0000 7fbd7738e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:14:57.645 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:57.647+0000 7fbd7738e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:14:57.800 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:57.465+0000 7fd149501200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:14:57.800 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:57.599+0000 7fd149501200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:14:57.801 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:57.696+0000 7fd149501200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:14:58.044 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:57.837+0000 7fbd7738e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:14:58.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:57 smithi144 ceph-mon[35650]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T15:14:58.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:57 smithi144 ceph-mon[35650]: Reconfiguring daemon mgr.x on smithi005 2024-05-14T15:14:58.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:57 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3541419423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca22e76e-dbda-4a5c-be55-9740b288c6d4"}]: dispatch 2024-05-14T15:14:58.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:57 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3541419423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca22e76e-dbda-4a5c-be55-9740b288c6d4"}]': finished 2024-05-14T15:14:58.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:57 smithi144 ceph-mon[35650]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T15:14:58.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:57 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:14:58.116 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:58.026+0000 7fd149501200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:14:58.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:57 smithi193 ceph-mon[36001]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T15:14:58.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:57 smithi193 ceph-mon[36001]: Reconfiguring daemon mgr.x on smithi005 2024-05-14T15:14:58.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:57 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3541419423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca22e76e-dbda-4a5c-be55-9740b288c6d4"}]: dispatch 2024-05-14T15:14:58.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:57 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3541419423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca22e76e-dbda-4a5c-be55-9740b288c6d4"}]': finished 2024-05-14T15:14:58.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:57 smithi193 ceph-mon[36001]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T15:14:58.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:57 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:14:58.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:57 smithi005 ceph-mon[31559]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T15:14:58.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:57 smithi005 ceph-mon[31559]: Reconfiguring daemon mgr.x on smithi005 2024-05-14T15:14:58.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:57 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3541419423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca22e76e-dbda-4a5c-be55-9740b288c6d4"}]: dispatch 2024-05-14T15:14:58.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:57 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3541419423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca22e76e-dbda-4a5c-be55-9740b288c6d4"}]': finished 2024-05-14T15:14:58.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:57 smithi005 ceph-mon[31559]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T15:14:58.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:14:58.550 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:58.118+0000 7fd149501200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:14:58.794 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:58.436+0000 7fbd7738e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:14:58.794 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:58.654+0000 7fbd7738e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:14:59.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:58 smithi193 ceph-mon[36001]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:59.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:58 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1513363595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:14:59.051 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:58.719+0000 7fd149501200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:14:59.051 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:14:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:14:58.936+0000 7fd149501200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:14:59.163 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:58 smithi144 ceph-mon[35650]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:59.163 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:58 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1513363595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:14:59.163 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:59.070+0000 7fbd7738e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:14:59.163 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:59.165+0000 7fbd7738e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:14:59.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:58 smithi005 ceph-mon[31559]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:14:59.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:58 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1513363595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:14:59.794 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:59.535+0000 7fbd7738e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:14:59.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:14:59.835+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7f1d8801dd20 con 0x7f1d8c104370 2024-05-14T15:15:00.216 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:59.887+0000 7fbd7738e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:15:00.216 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:14:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:14:59.979+0000 7fbd7738e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:15:00.217 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:59 smithi144 ceph-mon[35650]: Standby manager daemon z started 2024-05-14T15:15:00.217 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:59 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T15:15:00.217 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:59 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:15:00.217 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:59 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T15:15:00.217 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:14:59 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:15:00.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:59 smithi193 ceph-mon[36001]: Standby manager daemon z started 2024-05-14T15:15:00.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:59 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T15:15:00.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:59 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:15:00.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:59 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T15:15:00.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:14:59 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:15:00.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:59 smithi005 ceph-mon[31559]: Standby manager daemon z started 2024-05-14T15:15:00.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:59 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T15:15:00.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:59 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:15:00.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:59 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T15:15:00.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:14:59 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/773359922' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:15:00.544 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:00.219+0000 7fbd7738e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:15:00.544 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:00.352+0000 7fbd7738e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:15:00.544 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:00.446+0000 7fbd7738e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:15:01.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:00 smithi144 ceph-mon[35650]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:01.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:00 smithi144 ceph-mon[35650]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-14T15:15:01.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T15:15:01.045 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:00.773+0000 7fbd7738e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:15:01.045 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:00.863+0000 7fbd7738e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:15:01.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:00 smithi005 ceph-mon[31559]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:01.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:00 smithi005 ceph-mon[31559]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-14T15:15:01.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T15:15:01.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:00 smithi193 ceph-mon[36001]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:01.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:00 smithi193 ceph-mon[36001]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-14T15:15:01.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T15:15:01.794 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:01.457+0000 7fbd7738e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:15:01.795 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:15:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:15:01.670+0000 7fbd7738e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:15:01.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:01.843+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f1d88017cd0 con 0x7f1d8c104370 2024-05-14T15:15:02.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:01 smithi005 ceph-mon[31559]: Standby manager daemon y started 2024-05-14T15:15:02.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:01 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T15:15:02.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:01 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:15:02.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:01 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T15:15:02.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:01 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:15:02.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:01 smithi144 ceph-mon[35650]: Standby manager daemon y started 2024-05-14T15:15:02.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:01 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T15:15:02.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:01 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:15:02.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:01 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T15:15:02.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:01 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:01 smithi193 ceph-mon[36001]: Standby manager daemon y started 2024-05-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:01 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:01 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:01 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:01 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/3486030536' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:15:03.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:02 smithi005 ceph-mon[31559]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:03.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:02 smithi005 ceph-mon[31559]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T15:15:03.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:02 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T15:15:03.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:02 smithi193 ceph-mon[36001]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:03.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:02 smithi193 ceph-mon[36001]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T15:15:03.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:02 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T15:15:03.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:02 smithi144 ceph-mon[35650]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:03.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:02 smithi144 ceph-mon[35650]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T15:15:03.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:02 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:04 smithi005 ceph-mon[31559]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:04.544 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:04.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:04.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:04.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:04 smithi144 ceph-mon[35650]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:04.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:04.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:04.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:04.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:04.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:04 smithi193 ceph-mon[36001]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:05.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T15:15:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:05.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T15:15:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:05.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T15:15:05.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:06.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:06 smithi005 ceph-mon[31559]: Deploying daemon osd.0 on smithi005 2024-05-14T15:15:06.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:06 smithi005 ceph-mon[31559]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:06.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:06 smithi144 ceph-mon[35650]: Deploying daemon osd.0 on smithi005 2024-05-14T15:15:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:06 smithi144 ceph-mon[35650]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:06.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:06 smithi193 ceph-mon[36001]: Deploying daemon osd.0 on smithi005 2024-05-14T15:15:06.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:06 smithi193 ceph-mon[36001]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:08.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:08 smithi005 ceph-mon[31559]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:08.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:08 smithi144 ceph-mon[35650]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:08.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:08 smithi193 ceph-mon[36001]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:10.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:10 smithi005 ceph-mon[31559]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:10.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:10 smithi144 ceph-mon[35650]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:10.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:10 smithi193 ceph-mon[36001]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:11.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:11.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:11.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:12.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:12.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:12.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:12.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:12.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:12.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:12.642 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:12 smithi005 ceph-mon[31559]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:13.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:12 smithi144 ceph-mon[35650]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:13.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:12 smithi193 ceph-mon[36001]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:13.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:13.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:13.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:13.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:13.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:14.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:14.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:14.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:14.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.189+0000 7f1d837fe700 1 -- 172.21.15.5:0/2010942727 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d8c063c00 con 0x7f1d7803e750 2024-05-14T15:15:14.188 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1d7803e750 msgr2=0x7f1d78040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1d7803e750 0x7f1d78040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c19b920 tx=0x7f1d7c009260 comp rx=0 tx=0).stop 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 msgr2=0x7f1d8c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 0x7f1d8c19a870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800e940 tx=0x7f1d8800ec50 comp rx=0 tx=0).stop 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 shutdown_connections 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f1d7803e750 0x7f1d78040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d8c105e40 0x7f1d8c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c104370 0x7f1d8c19a870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 --2- 172.21.15.5:0/2010942727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d8c1054a0 0x7f1d8c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 >> 172.21.15.5:0/2010942727 conn(0x7f1d8c0ff9d0 msgr2=0x7f1d8c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:15:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 shutdown_connections 2024-05-14T15:15:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:15:14.192+0000 7f1d946d6700 1 -- 172.21.15.5:0/2010942727 wait complete. 2024-05-14T15:15:14.796 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.0.service 2024-05-14T15:15:14.799 INFO:tasks.cephadm:Deploying osd.1 on smithi144 with /dev/vg_nvme/lv_4... 2024-05-14T15:15:14.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T15:15:15.041 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:15:15.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:14 smithi144 ceph-mon[35650]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:15.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:14 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:15.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:14 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:15.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:14 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:15.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:14 smithi144 ceph-mon[35650]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T15:15:15.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:14 smithi193 ceph-mon[36001]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:15.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:14 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:15.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:14 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:15.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:14 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:15.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:14 smithi193 ceph-mon[36001]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T15:15:15.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:14 smithi005 ceph-mon[31559]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:15.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:14 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:15.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:14 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:15.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:14 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:15.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:14 smithi005 ceph-mon[31559]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T15:15:16.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:16 smithi144 ceph-mon[35650]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T15:15:16.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:16 smithi144 ceph-mon[35650]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T15:15:16.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:16 smithi144 ceph-mon[35650]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:15:16.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:16.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:16 smithi144 ceph-mon[35650]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:16.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:16 smithi193 ceph-mon[36001]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T15:15:16.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:16 smithi193 ceph-mon[36001]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T15:15:16.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:16 smithi193 ceph-mon[36001]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:15:16.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:16.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:16 smithi193 ceph-mon[36001]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:16 smithi005 ceph-mon[31559]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T15:15:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:16 smithi005 ceph-mon[31559]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T15:15:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:16 smithi005 ceph-mon[31559]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:15:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:16 smithi005 ceph-mon[31559]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:16.871 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T15:15:16.871 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T15:15:16.871 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-14T15:15:16.871 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-14T15:15:16.872 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113821 s, 921 MB/s 2024-05-14T15:15:16.872 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-14T15:15:17.059 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:15:16 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0[46433]: 2024-05-14T15:15:16.750+0000 7f37d71db700 -1 osd.0 0 waiting for initial osdmap 2024-05-14T15:15:17.059 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:15:16 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0[46433]: 2024-05-14T15:15:16.754+0000 7f37cef88700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:15:17.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:17 smithi005 ceph-mon[31559]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:15:17.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:17 smithi005 ceph-mon[31559]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T15:15:17.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:17.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:17.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:17 smithi005 ceph-mon[31559]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' 2024-05-14T15:15:17.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:17 smithi144 ceph-mon[35650]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:15:17.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:17 smithi144 ceph-mon[35650]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T15:15:17.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:17.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:17.447 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:17 smithi144 ceph-mon[35650]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' 2024-05-14T15:15:17.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-05-14T15:15:17.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:17 smithi193 ceph-mon[36001]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:15:17.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:17 smithi193 ceph-mon[36001]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T15:15:17.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:17.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:17.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:17 smithi193 ceph-mon[36001]: from='osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370]' entity='osd.0' 2024-05-14T15:15:17.730 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:15:18.294 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] boot 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi005 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi005 to 17093M 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:18.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:18.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] boot 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi005 2024-05-14T15:15:18.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi005 to 17093M 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:18.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:15:18.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: osd.0 [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] boot 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi005 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi005 to 17093M 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:18.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 -- 172.21.15.144:0/1170838027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c10c3e0 msgr2=0x7f886c10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 --2- 172.21.15.144:0/1170838027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c10c3e0 0x7f886c10e800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f886400b0a0 tx=0x7f886400b3b0 comp rx=0 tx=0).stop 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 -- 172.21.15.144:0/1170838027 shutdown_connections 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 --2- 172.21.15.144:0/1170838027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c10c3e0 0x7f886c10e800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 --2- 172.21.15.144:0/1170838027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c104830 0x7f886c104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 --2- 172.21.15.144:0/1170838027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f886c109a80 0x7f886c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 -- 172.21.15.144:0/1170838027 >> 172.21.15.144:0/1170838027 conn(0x7f886c0fe020 msgr2=0x7f886c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 -- 172.21.15.144:0/1170838027 shutdown_connections 2024-05-14T15:15:19.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.153+0000 7f88709ba700 1 -- 172.21.15.144:0/1170838027 wait complete. 2024-05-14T15:15:19.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 Processor -- start 2024-05-14T15:15:19.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 -- start start 2024-05-14T15:15:19.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 0x7f886c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:19.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c109a80 0x7f886c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f886c10c3e0 0x7f886c1a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f886c112010 con 0x7f886c10c3e0 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f886c111e90 con 0x7f886c104830 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f88709ba700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f886c112190 con 0x7f886c109a80 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f886a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 0x7f886c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f886a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 0x7f886c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41150/0 (socket says 172.21.15.144:41150) 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f886a59c700 1 -- 172.21.15.144:0/2926410427 learned_addr learned my addr 172.21.15.144:0/2926410427 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f8869d9b700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c109a80 0x7f886c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.154+0000 7f886ad9d700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f886c10c3e0 0x7f886c1a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f886a59c700 1 -- 172.21.15.144:0/2926410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c109a80 msgr2=0x7f886c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f886a59c700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c109a80 0x7f886c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f886a59c700 1 -- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f886c10c3e0 msgr2=0x7f886c1a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f886a59c700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f886c10c3e0 0x7f886c1a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:19.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f886a59c700 1 -- 172.21.15.144:0/2926410427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c1a45e0 con 0x7f886c104830 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f886a59c700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 0x7f886c19ca80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f885c00d980 tx=0x7f885c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c017070 con 0x7f886c104830 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f885c005020 con 0x7f886c104830 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c00fa80 con 0x7f886c104830 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c1a48d0 con 0x7f886c104830 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.155+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c1a4dc0 con 0x7f886c104830 2024-05-14T15:15:19.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.156+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c0512c0 con 0x7f886c104830 2024-05-14T15:15:19.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.157+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f885c005510 con 0x7f886c104830 2024-05-14T15:15:19.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.158+0000 7f885b7fe700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f88540b27f0 0x7f88540b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:19.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.158+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f885c0e0360 con 0x7f886c104830 2024-05-14T15:15:19.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.158+0000 7f8869d9b700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f88540b27f0 0x7f88540b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:19.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.159+0000 7f8869d9b700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f88540b27f0 0x7f88540b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f886c19db30 tx=0x7f886000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:15:19.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.162+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f885c0a9af0 con 0x7f886c104830 2024-05-14T15:15:19.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:19.371+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f886c063c00 con 0x7f88540b27f0 2024-05-14T15:15:20.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:20 smithi144 ceph-mon[35650]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T15:15:20.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:20 smithi144 ceph-mon[35650]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:15:20.276 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:15:20.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:15:20.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:20.277 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:20 smithi144 ceph-mon[35650]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:20.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:20 smithi193 ceph-mon[36001]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T15:15:20.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:20 smithi193 ceph-mon[36001]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:15:20.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:15:20.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:15:20.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:20.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:20 smithi193 ceph-mon[36001]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:20.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:20 smithi005 ceph-mon[31559]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T15:15:20.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:20 smithi005 ceph-mon[31559]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:15:20.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:15:20.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:15:20.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:20.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:20 smithi005 ceph-mon[31559]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:22 smithi144 ceph-mon[35650]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:22 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/2682093865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]: dispatch 2024-05-14T15:15:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:22 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]: dispatch 2024-05-14T15:15:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:22 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]': finished 2024-05-14T15:15:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:22 smithi144 ceph-mon[35650]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T15:15:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:22.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:22 smithi193 ceph-mon[36001]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:22.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:22 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/2682093865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]: dispatch 2024-05-14T15:15:22.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:22 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]: dispatch 2024-05-14T15:15:22.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:22 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]': finished 2024-05-14T15:15:22.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:22 smithi193 ceph-mon[36001]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T15:15:22.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:22 smithi005 ceph-mon[31559]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:22 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/2682093865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]: dispatch 2024-05-14T15:15:22.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:22 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]: dispatch 2024-05-14T15:15:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:22 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e57534-cfcb-479f-bcfc-64cadc8937cd"}]': finished 2024-05-14T15:15:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:22 smithi005 ceph-mon[31559]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T15:15:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:23.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:23 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/15694181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:15:23.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:23 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/15694181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:15:23.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:23 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/15694181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:15:24.715 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:24 smithi005 ceph-mon[31559]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:24.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:24 smithi144 ceph-mon[35650]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:24.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:24 smithi193 ceph-mon[36001]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:26.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:26 smithi144 ceph-mon[35650]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:26.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:26 smithi193 ceph-mon[36001]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:26.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:26 smithi005 ceph-mon[31559]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:28.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:28 smithi144 ceph-mon[35650]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:28.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:28 smithi193 ceph-mon[36001]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:28.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:28 smithi005 ceph-mon[31559]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:29.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:29 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T15:15:29.752 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:29 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:29.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:29 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T15:15:29.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:29 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:29.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:29 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T15:15:29.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:29 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:30.739 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:30 smithi144 ceph-mon[35650]: Deploying daemon osd.1 on smithi144 2024-05-14T15:15:30.740 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:30 smithi144 ceph-mon[35650]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:30.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:30 smithi193 ceph-mon[36001]: Deploying daemon osd.1 on smithi144 2024-05-14T15:15:30.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:30 smithi193 ceph-mon[36001]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:30.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:30 smithi005 ceph-mon[31559]: Deploying daemon osd.1 on smithi144 2024-05-14T15:15:30.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:30 smithi005 ceph-mon[31559]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:32.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:32 smithi144 ceph-mon[35650]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:32.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:32 smithi193 ceph-mon[36001]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:32.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:32 smithi005 ceph-mon[31559]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:34.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:34 smithi144 ceph-mon[35650]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:34.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:34.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:34.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:34.800 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:34 smithi193 ceph-mon[36001]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:34.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:34.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:34.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:34.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:34 smithi005 ceph-mon[31559]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:34.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:34.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:34.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:36.551 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:36 smithi144 ceph-mon[35650]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:36.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:36 smithi193 ceph-mon[36001]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:36.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:36 smithi005 ceph-mon[31559]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:37.690 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.691+0000 7f885b7fe700 1 -- 172.21.15.144:0/2926410427 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f886c063c00 con 0x7f88540b27f0 2024-05-14T15:15:37.690 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 1 on host 'smithi144' 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f88540b27f0 msgr2=0x7f88540b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f88540b27f0 0x7f88540b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f886c19db30 tx=0x7f886000b410 comp rx=0 tx=0).stop 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 msgr2=0x7f886c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 0x7f886c19ca80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f885c00d980 tx=0x7f885c00dc90 comp rx=0 tx=0).stop 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 shutdown_connections 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f88540b27f0 0x7f88540b4d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f886c10c3e0 0x7f886c1a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:37.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f886c104830 0x7f886c19ca80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:37.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 --2- 172.21.15.144:0/2926410427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c109a80 0x7f886c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:37.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 >> 172.21.15.144:0/2926410427 conn(0x7f886c0fe020 msgr2=0x7f886c10a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:15:37.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 shutdown_connections 2024-05-14T15:15:37.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:15:37.693+0000 7f88709ba700 1 -- 172.21.15.144:0/2926410427 wait complete. 2024-05-14T15:15:38.305 DEBUG:teuthology.orchestra.run.smithi144:osd.1> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.1.service 2024-05-14T15:15:38.308 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-05-14T15:15:38.309 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:38.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T15:15:38.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:38 smithi144 ceph-mon[35650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T15:15:38.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.550 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T15:15:38.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:38 smithi193 ceph-mon[36001]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T15:15:38.554 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T15:15:38.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:38 smithi005 ceph-mon[31559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T15:15:39.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:39 smithi193 ceph-mon[36001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T15:15:39.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:39 smithi193 ceph-mon[36001]: from='osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:15:39.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:39 smithi193 ceph-mon[36001]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T15:15:39.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:39 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:39.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:39 smithi193 ceph-mon[36001]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:15:40.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:39 smithi144 ceph-mon[35650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T15:15:40.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:39 smithi144 ceph-mon[35650]: from='osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:15:40.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:39 smithi144 ceph-mon[35650]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T15:15:40.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:39 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:40.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:39 smithi144 ceph-mon[35650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:15:40.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:39 smithi005 ceph-mon[31559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T15:15:40.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:39 smithi005 ceph-mon[31559]: from='osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:15:40.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:39 smithi005 ceph-mon[31559]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T15:15:40.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:39 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:40.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:39 smithi005 ceph-mon[31559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:15:40.338 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T15:15:40.339 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T15:15:40.339 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T15:15:40.339 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T15:15:40.339 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115367 s, 909 MB/s 2024-05-14T15:15:40.339 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T15:15:40.451 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:15:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1[40230]: 2024-05-14T15:15:40.167+0000 7fbab8ce7700 -1 osd.1 0 waiting for initial osdmap 2024-05-14T15:15:40.451 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:15:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1[40230]: 2024-05-14T15:15:40.173+0000 7fbab0a94700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:15:40.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:40 smithi144 ceph-mon[35650]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:40.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:40 smithi144 ceph-mon[35650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:15:40.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:40 smithi144 ceph-mon[35650]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T15:15:40.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:40.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:40.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:40 smithi144 ceph-mon[35650]: from='osd.1 ' entity='osd.1' 2024-05-14T15:15:40.909 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:40 smithi193 ceph-mon[36001]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:40.910 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:40 smithi193 ceph-mon[36001]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:15:40.910 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:40 smithi193 ceph-mon[36001]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T15:15:40.910 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:40.910 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:40.910 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:40 smithi193 ceph-mon[36001]: from='osd.1 ' entity='osd.1' 2024-05-14T15:15:40.938 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-05-14T15:15:41.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:40 smithi005 ceph-mon[31559]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:41.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:40 smithi005 ceph-mon[31559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:15:41.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:40 smithi005 ceph-mon[31559]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T15:15:41.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:41.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:41.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:40 smithi005 ceph-mon[31559]: from='osd.1 ' entity='osd.1' 2024-05-14T15:15:41.180 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:15:41.948 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:15:41.948 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:15:41.948 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] boot 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T15:15:41.949 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:42.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] boot 2024-05-14T15:15:42.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T15:15:42.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: osd.1 [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] boot 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T15:15:42.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:15:42.683 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.682+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/98381765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 msgr2=0x7fb8b4102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:42.683 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.682+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/98381765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b4102760 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009990 tx=0x7fb8a8009ca0 comp rx=0 tx=0).stop 2024-05-14T15:15:42.683 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/98381765 shutdown_connections 2024-05-14T15:15:42.683 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/98381765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4102ca0 0x7fb8b410c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:42.683 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/98381765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b4105900 0x7fb8b4105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:42.683 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/98381765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b4102760 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/98381765 >> 172.21.15.193:0/98381765 conn(0x7fb8b40fe020 msgr2=0x7fb8b41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/98381765 shutdown_connections 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/98381765 wait complete. 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 Processor -- start 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.683+0000 7fb8ba9b4700 1 -- start start 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8ba9b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8ba9b4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b4102ca0 0x7fb8b419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8ba9b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4105900 0x7fb8b41a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8ba9b4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410fdc0 con 0x7fb8b4102ca0 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8ba9b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410fc40 con 0x7fb8b4105900 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8ba9b4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410ff40 con 0x7fb8b4102350 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b419a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:45782/0 (socket says 172.21.15.193:45782) 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 -- 172.21.15.193:0/2350766741 learned_addr learned my addr 172.21.15.193:0/2350766741 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:15:42.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b37fe700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b4102ca0 0x7fb8b419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b8f51700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4105900 0x7fb8b41a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 -- 172.21.15.193:0/2350766741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4105900 msgr2=0x7fb8b41a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4105900 0x7fb8b41a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 -- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b4102ca0 msgr2=0x7fb8b419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b4102ca0 0x7fb8b419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 -- 172.21.15.193:0/2350766741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b41a2440 con 0x7fb8b4102350 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b8f51700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4105900 0x7fb8b41a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.684+0000 7fb8b3fff700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b419a8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400d950 tx=0x7fb8a400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.685+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4017070 con 0x7fb8b4102350 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.685+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a4009730 con 0x7fb8b4102350 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.685+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b41a2730 con 0x7fb8b4102350 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.685+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4003a50 con 0x7fb8b4102350 2024-05-14T15:15:42.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.685+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a2c70 con 0x7fb8b4102350 2024-05-14T15:15:42.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.686+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b40512c0 con 0x7fb8b4102350 2024-05-14T15:15:42.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.687+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fb8a4003bf0 con 0x7fb8b4102350 2024-05-14T15:15:42.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.688+0000 7fb8b17fa700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb89c0b27f0 0x7fb89c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:15:42.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.688+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fb8a40e0c70 con 0x7fb8b4102350 2024-05-14T15:15:42.686 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.688+0000 7fb8b37fe700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb89c0b27f0 0x7fb89c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:15:42.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.689+0000 7fb8b37fe700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb89c0b27f0 0x7fb89c0b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800b5c0 tx=0x7fb8a801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:15:42.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.692+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb8a4020030 con 0x7fb8b4102350 2024-05-14T15:15:42.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:15:42.905+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb8b4063c00 con 0x7fb89c0b27f0 2024-05-14T15:15:42.903 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:42 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi144 2024-05-14T15:15:42.904 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:42 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi144 to 17088M 2024-05-14T15:15:42.904 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:42 smithi193 ceph-mon[36001]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:42.904 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:42 smithi193 ceph-mon[36001]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T15:15:43.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:42 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi144 2024-05-14T15:15:43.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:42 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi144 to 17088M 2024-05-14T15:15:43.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:42 smithi144 ceph-mon[35650]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:43.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:42 smithi144 ceph-mon[35650]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T15:15:43.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:42 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi144 2024-05-14T15:15:43.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:42 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi144 to 17088M 2024-05-14T15:15:43.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:42 smithi005 ceph-mon[31559]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-14T15:15:43.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:42 smithi005 ceph-mon[31559]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T15:15:43.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:15:43.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:15:43.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:43.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:15:43.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:15:43.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:15:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:15:44.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:45.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:44 smithi144 ceph-mon[35650]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:15:45.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:44 smithi144 ceph-mon[35650]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:45.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:44 smithi193 ceph-mon[36001]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:15:45.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:44 smithi193 ceph-mon[36001]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:45.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:44 smithi005 ceph-mon[31559]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:15:45.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:44 smithi005 ceph-mon[31559]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:46.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:45 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/364856858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]: dispatch 2024-05-14T15:15:46.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:45 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]: dispatch 2024-05-14T15:15:46.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:45 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]': finished 2024-05-14T15:15:46.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:45 smithi144 ceph-mon[35650]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T15:15:46.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:45 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:15:46.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:45 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/364856858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]: dispatch 2024-05-14T15:15:46.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:45 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]: dispatch 2024-05-14T15:15:46.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:45 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]': finished 2024-05-14T15:15:46.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:45 smithi193 ceph-mon[36001]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T15:15:46.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:45 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:15:46.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:45 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/364856858' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]: dispatch 2024-05-14T15:15:46.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:45 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]: dispatch 2024-05-14T15:15:46.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:45 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58e105d1-e047-41db-a978-c3baed2b8d25"}]': finished 2024-05-14T15:15:46.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:45 smithi005 ceph-mon[31559]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T15:15:46.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:45 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:15:47.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:46 smithi144 ceph-mon[35650]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:47.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:46 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/2054101337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:15:47.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:46 smithi193 ceph-mon[36001]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:47.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:46 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/2054101337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:15:47.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:46 smithi005 ceph-mon[31559]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:47.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:46 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/2054101337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:15:49.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:48 smithi144 ceph-mon[35650]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:49.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:48 smithi193 ceph-mon[36001]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:49.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:48 smithi005 ceph-mon[31559]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:50.905 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:50 smithi193 ceph-mon[36001]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:51.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:50 smithi144 ceph-mon[35650]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:51.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:50 smithi005 ceph-mon[31559]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:52.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:52 smithi193 ceph-mon[36001]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:52.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T15:15:52.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:53.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:52 smithi144 ceph-mon[35650]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:53.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T15:15:53.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:53.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:52 smithi005 ceph-mon[31559]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:53.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T15:15:53.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:15:53.976 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:53 smithi193 ceph-mon[36001]: Deploying daemon osd.2 on smithi193 2024-05-14T15:15:54.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:53 smithi144 ceph-mon[35650]: Deploying daemon osd.2 on smithi193 2024-05-14T15:15:54.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:53 smithi005 ceph-mon[31559]: Deploying daemon osd.2 on smithi193 2024-05-14T15:15:55.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:54 smithi144 ceph-mon[35650]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:55.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:54 smithi193 ceph-mon[36001]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:55.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:54 smithi005 ceph-mon[31559]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:57.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:56 smithi144 ceph-mon[35650]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:57.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:56 smithi193 ceph-mon[36001]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:57.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:56 smithi005 ceph-mon[31559]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:59.183 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:58 smithi193 ceph-mon[36001]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:59.183 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:59.183 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:59.183 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:15:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:59.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:58 smithi144 ceph-mon[35650]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:59.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:59.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:59.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:15:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:15:59.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:58 smithi005 ceph-mon[31559]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:15:59.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:59.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:15:59.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:15:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:01.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:00 smithi193 ceph-mon[36001]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:01.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:01.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:01.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:00 smithi144 ceph-mon[35650]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:01.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:01.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:01.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:00 smithi005 ceph-mon[31559]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:01.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:01.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:01.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:01.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.423+0000 7fb8b17fa700 1 -- 172.21.15.193:0/2350766741 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8b4063c00 con 0x7fb89c0b27f0 2024-05-14T15:16:01.422 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-05-14T15:16:01.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.425+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb89c0b27f0 msgr2=0x7fb89c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:01.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.425+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb89c0b27f0 0x7fb89c0b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800b5c0 tx=0x7fb8a801a040 comp rx=0 tx=0).stop 2024-05-14T15:16:01.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.425+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 msgr2=0x7fb8b419a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:01.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.425+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b419a8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400d950 tx=0x7fb8a400dc60 comp rx=0 tx=0).stop 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 shutdown_connections 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb89c0b27f0 0x7fb89c0b4d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b4105900 0x7fb8b41a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102350 0x7fb8b419a8e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 --2- 172.21.15.193:0/2350766741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b4102ca0 0x7fb8b419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 >> 172.21.15.193:0/2350766741 conn(0x7fb8b40fe020 msgr2=0x7fb8b41088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 shutdown_connections 2024-05-14T15:16:01.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:01.426+0000 7fb8ba9b4700 1 -- 172.21.15.193:0/2350766741 wait complete. 2024-05-14T15:16:02.005 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.2.service 2024-05-14T15:16:02.007 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-05-14T15:16:02.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T15:16:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:16:02.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:02 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:02.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:02 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:02.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:02 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:02.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:02 smithi193 ceph-mon[36001]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:02.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:02 smithi193 ceph-mon[36001]: from='osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T15:16:02.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:02 smithi193 ceph-mon[36001]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T15:16:02.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:02 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:02.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:02 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:02.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:02 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:02.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:02 smithi005 ceph-mon[31559]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:02.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:02 smithi005 ceph-mon[31559]: from='osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T15:16:02.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:02 smithi005 ceph-mon[31559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T15:16:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:02 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:02 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:02 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:02 smithi144 ceph-mon[35650]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:02 smithi144 ceph-mon[35650]: from='osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T15:16:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:02 smithi144 ceph-mon[35650]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T15:16:03.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:03 smithi193 ceph-mon[36001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T15:16:03.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:03 smithi193 ceph-mon[36001]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T15:16:03.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:03.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:03 smithi193 ceph-mon[36001]: from='osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:16:03.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:03 smithi193 ceph-mon[36001]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:16:03.399 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2[41149]: 2024-05-14T15:16:03.395+0000 7fa87c5de700 -1 osd.2 0 waiting for initial osdmap 2024-05-14T15:16:03.399 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2[41149]: 2024-05-14T15:16:03.399+0000 7fa878393700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:16:03.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:03 smithi005 ceph-mon[31559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T15:16:03.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:03 smithi005 ceph-mon[31559]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T15:16:03.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:03.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:03 smithi005 ceph-mon[31559]: from='osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:16:03.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:03 smithi005 ceph-mon[31559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:16:03.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:03 smithi144 ceph-mon[35650]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T15:16:03.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:03 smithi144 ceph-mon[35650]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T15:16:03.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:03.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:03 smithi144 ceph-mon[35650]: from='osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:16:03.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:03 smithi144 ceph-mon[35650]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:16:04.120 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T15:16:04.120 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T15:16:04.120 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-14T15:16:04.120 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-14T15:16:04.120 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00837437 s, 1.3 GB/s 2024-05-14T15:16:04.120 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-14T15:16:04.542 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:04 smithi193 ceph-mon[36001]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:16:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:04 smithi193 ceph-mon[36001]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T15:16:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:04.543 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:04 smithi193 ceph-mon[36001]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:04 smithi144 ceph-mon[35650]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:16:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:04 smithi144 ceph-mon[35650]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T15:16:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:04 smithi144 ceph-mon[35650]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:04.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:04 smithi005 ceph-mon[31559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:16:04.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:04 smithi005 ceph-mon[31559]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T15:16:04.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:04.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:04.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:04 smithi005 ceph-mon[31559]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-14T15:16:04.951 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-05-14T15:16:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] boot 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:05.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:05.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:05.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:05 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:16:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:16:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] boot 2024-05-14T15:16:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T15:16:05.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:05 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: osd.2 [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] boot 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:05.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:05.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:05.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:05 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:06.550 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:06 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi193 2024-05-14T15:16:06.550 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:06 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi193 to 17088M 2024-05-14T15:16:06.550 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:06 smithi005 ceph-mon[31559]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:06.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:06 smithi005 ceph-mon[31559]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T15:16:06.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:06 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:16:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:06 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi193 2024-05-14T15:16:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:06 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi193 to 17088M 2024-05-14T15:16:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:06 smithi144 ceph-mon[35650]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:06 smithi144 ceph-mon[35650]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T15:16:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:06 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:16:06.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:06 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi193 2024-05-14T15:16:06.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:06 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi193 to 17088M 2024-05-14T15:16:06.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:06 smithi193 ceph-mon[36001]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:06.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:06 smithi193 ceph-mon[36001]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T15:16:06.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:06 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:16:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 -- 172.21.15.5:0/225489647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 msgr2=0x7febbc106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 --2- 172.21.15.5:0/225489647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc106ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7febac00b0a0 tx=0x7febac00b3b0 comp rx=0 tx=0).stop 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 -- 172.21.15.5:0/225489647 shutdown_connections 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 --2- 172.21.15.5:0/225489647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 0x7febbc10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 --2- 172.21.15.5:0/225489647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc106ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 --2- 172.21.15.5:0/225489647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 0x7febbc109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 -- 172.21.15.5:0/225489647 >> 172.21.15.5:0/225489647 conn(0x7febbc0fe880 msgr2=0x7febbc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.813+0000 7febc202b700 1 -- 172.21.15.5:0/225489647 shutdown_connections 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.814+0000 7febc202b700 1 -- 172.21.15.5:0/225489647 wait complete. 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.814+0000 7febc202b700 1 Processor -- start 2024-05-14T15:16:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.814+0000 7febc202b700 1 -- start start 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febc202b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febc202b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 0x7febbc102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febc202b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 0x7febbc103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febc202b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7febbc10fd70 con 0x7febbc06c2b0 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febc202b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febbc10fbf0 con 0x7febbc1074a0 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febc202b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febbc10fef0 con 0x7febbc109ff0 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc104720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60452/0 (socket says 172.21.15.5:60452) 2024-05-14T15:16:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 -- 172.21.15.5:0/2082120386 learned_addr learned my addr 172.21.15.5:0/2082120386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbbfff700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 0x7febbc103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbaffd700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 0x7febbc102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 -- 172.21.15.5:0/2082120386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 msgr2=0x7febbc103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 0x7febbc103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 -- 172.21.15.5:0/2082120386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 msgr2=0x7febbc102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 0x7febbc102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 -- 172.21.15.5:0/2082120386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc103a70 con 0x7febbc06c2b0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbaffd700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 0x7febbc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbbfff700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 0x7febbc103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.815+0000 7febbb7fe700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc104720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febac00b070 tx=0x7febac004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.816+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac021070 con 0x7febbc06c2b0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.816+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac007d40 con 0x7febbc06c2b0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.816+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac0048a0 con 0x7febbc06c2b0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.816+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc1a49c0 con 0x7febbc06c2b0 2024-05-14T15:16:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.816+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a4e20 con 0x7febbc06c2b0 2024-05-14T15:16:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.817+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc1a4ff0 con 0x7febbc06c2b0 2024-05-14T15:16:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.819+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7febac004000 con 0x7febbc06c2b0 2024-05-14T15:16:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.819+0000 7febb8ff9700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feba80b2a10 0x7feba80b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.820+0000 7febbaffd700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feba80b2a10 0x7feba80b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.820+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7febac00f020 con 0x7febbc06c2b0 2024-05-14T15:16:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.820+0000 7febbaffd700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feba80b2a10 0x7feba80b4f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feba4005fd0 tx=0x7feba4005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:16:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:06.823+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febac02a020 con 0x7febbc06c2b0 2024-05-14T15:16:07.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:07.047+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7febbc106b10 con 0x7feba80b2a10 2024-05-14T15:16:07.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T15:16:07.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 ceph-mon[31559]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T15:16:07.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:16:07.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:07.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:07.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50757]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50757]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50757]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50757]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.682 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50770]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.683 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50770]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.721 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45197]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T15:16:07.721 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.721 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45197]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.721 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45197]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.721 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T15:16:07.722 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.722 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45200]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.722 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45200]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 ceph-mon[36001]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:07.722 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T15:16:07.759 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T15:16:07.760 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 ceph-mon[35650]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T15:16:07.760 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:16:07.760 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:07.760 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:07.760 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:07.760 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44222]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T15:16:07.760 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44222]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.760 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44222]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.760 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44222]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.760 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44225]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T15:16:07.760 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44225]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.761 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44225]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.761 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44225]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50817]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50817]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50820]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:07.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:07 smithi005 sudo[50820]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44228]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44228]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44228]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44228]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44231]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:08.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:07 smithi144 sudo[44231]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45203]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45203]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45206]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T15:16:08.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:07 smithi193 sudo[45206]: pam_unix(sudo:session): session closed for user root 2024-05-14T15:16:08.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:08.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:08.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:16:08.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.652 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.652 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.652 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.652 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.652 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:08 smithi005 ceph-mon[31559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:08.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:08 smithi144 ceph-mon[35650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:08.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:16:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:08 smithi193 ceph-mon[36001]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T15:16:09.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:09.439+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7febac0a7480 con 0x7febbc06c2b0 2024-05-14T15:16:09.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:09 smithi005 ceph-mon[31559]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T15:16:09.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:09 smithi144 ceph-mon[35650]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T15:16:09.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:09 smithi193 ceph-mon[36001]: osdmap e22: 3 total, 3 up, 3 in 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/623005046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7530ff13-b342-47a6-9bf9-fa376975ab8f"}]: dispatch 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/623005046' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7530ff13-b342-47a6-9bf9-fa376975ab8f"}]': finished 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T15:16:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:10 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/623005046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7530ff13-b342-47a6-9bf9-fa376975ab8f"}]: dispatch 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/623005046' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7530ff13-b342-47a6-9bf9-fa376975ab8f"}]': finished 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T15:16:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:10 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/623005046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7530ff13-b342-47a6-9bf9-fa376975ab8f"}]: dispatch 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/623005046' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7530ff13-b342-47a6-9bf9-fa376975ab8f"}]': finished 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T15:16:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:10 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:11.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:11 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/980918799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:16:11.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:11 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/980918799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:16:11.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:11 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/980918799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:16:12.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:12 smithi144 ceph-mon[35650]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:12.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:12 smithi193 ceph-mon[36001]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:12.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:12 smithi005 ceph-mon[31559]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:14.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:14 smithi144 ceph-mon[35650]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:14.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:14 smithi193 ceph-mon[36001]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:14.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:14 smithi005 ceph-mon[31559]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:16.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:16 smithi005 ceph-mon[31559]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:16.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:16 smithi144 ceph-mon[35650]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:16.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:16 smithi193 ceph-mon[36001]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:17.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T15:16:17.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:17.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T15:16:17.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:17.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T15:16:17.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:18.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:18 smithi005 ceph-mon[31559]: Deploying daemon osd.3 on smithi005 2024-05-14T15:16:18.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:18 smithi005 ceph-mon[31559]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:18.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:18 smithi144 ceph-mon[35650]: Deploying daemon osd.3 on smithi005 2024-05-14T15:16:18.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:18 smithi144 ceph-mon[35650]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:18.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:18 smithi193 ceph-mon[36001]: Deploying daemon osd.3 on smithi005 2024-05-14T15:16:18.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:18 smithi193 ceph-mon[36001]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:20.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:20 smithi005 ceph-mon[31559]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:20.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:20 smithi144 ceph-mon[35650]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:20.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:20 smithi193 ceph-mon[36001]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:22.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:22 smithi005 ceph-mon[31559]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:22.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:22 smithi144 ceph-mon[35650]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:22.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:22 smithi193 ceph-mon[36001]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:24.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:24.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:24.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:24.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:24 smithi144 ceph-mon[35650]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:24.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:24.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:24.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:24.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:24 smithi193 ceph-mon[36001]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:24.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:24.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:24.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:24.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:24 smithi005 ceph-mon[31559]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.763+0000 7febb8ff9700 1 -- 172.21.15.5:0/2082120386 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febbc106b10 con 0x7feba80b2a10 2024-05-14T15:16:26.762 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.765+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feba80b2a10 msgr2=0x7feba80b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.765+0000 7febc202b700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feba80b2a10 0x7feba80b4f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feba4005fd0 tx=0x7feba4005f00 comp rx=0 tx=0).stop 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.765+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 msgr2=0x7febbc104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.765+0000 7febc202b700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc104720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febac00b070 tx=0x7febac004e00 comp rx=0 tx=0).stop 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 shutdown_connections 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feba80b2a10 0x7feba80b4f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc109ff0 0x7febbc103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc06c2b0 0x7febbc104720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 --2- 172.21.15.5:0/2082120386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febbc1074a0 0x7febbc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 >> 172.21.15.5:0/2082120386 conn(0x7febbc0fe880 msgr2=0x7febbc10c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:16:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 shutdown_connections 2024-05-14T15:16:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:16:26.766+0000 7febc202b700 1 -- 172.21.15.5:0/2082120386 wait complete. 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:27.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:27.563 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.3.service 2024-05-14T15:16:27.565 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_3... 2024-05-14T15:16:27.566 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T15:16:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:16:28.172 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:27 smithi144 ceph-mon[35650]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T15:16:28.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:27 smithi193 ceph-mon[36001]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T15:16:28.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:27 smithi005 ceph-mon[31559]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T15:16:29.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:28 smithi005 ceph-mon[31559]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:29.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:28 smithi005 ceph-mon[31559]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T15:16:29.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:28 smithi005 ceph-mon[31559]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T15:16:29.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:28 smithi005 ceph-mon[31559]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:16:29.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:29.126 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:28 smithi144 ceph-mon[35650]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:29.126 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:28 smithi144 ceph-mon[35650]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T15:16:29.126 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:28 smithi144 ceph-mon[35650]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T15:16:29.126 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:28 smithi144 ceph-mon[35650]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:16:29.126 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:29.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:28 smithi193 ceph-mon[36001]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:29.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:28 smithi193 ceph-mon[36001]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T15:16:29.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:28 smithi193 ceph-mon[36001]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T15:16:29.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:28 smithi193 ceph-mon[36001]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:16:29.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:29.813 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:16:29 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:16:29.350+0000 7f0728b48700 -1 osd.3 0 waiting for initial osdmap 2024-05-14T15:16:29.813 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:16:29 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:16:29.358+0000 7f07208f5700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:16:29.816 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T15:16:29.816 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T15:16:29.817 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-14T15:16:29.817 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-14T15:16:29.817 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129762 s, 808 MB/s 2024-05-14T15:16:29.817 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-14T15:16:30.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:29 smithi144 ceph-mon[35650]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:16:30.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:29 smithi144 ceph-mon[35650]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T15:16:30.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:29 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:30.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:29 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:30.133 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:29 smithi144 ceph-mon[35650]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' 2024-05-14T15:16:30.133 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:29 smithi144 ceph-mon[35650]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:30.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:29 smithi193 ceph-mon[36001]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:16:30.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:29 smithi193 ceph-mon[36001]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T15:16:30.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:29 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:30.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:29 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:30.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:29 smithi193 ceph-mon[36001]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' 2024-05-14T15:16:30.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:29 smithi193 ceph-mon[36001]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:30.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:29 smithi005 ceph-mon[31559]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:16:30.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:29 smithi005 ceph-mon[31559]: osdmap e25: 4 total, 3 up, 4 in 2024-05-14T15:16:30.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:29 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:30.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:29 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:30.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:29 smithi005 ceph-mon[31559]: from='osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675]' entity='osd.3' 2024-05-14T15:16:30.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:29 smithi005 ceph-mon[31559]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T15:16:30.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-05-14T15:16:30.700 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:16:31.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:30 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:16:31.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:30 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:16:31.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:30 smithi144 ceph-mon[35650]: osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] boot 2024-05-14T15:16:31.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:30 smithi144 ceph-mon[35650]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T15:16:31.119 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:31.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:30 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:16:31.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:30 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:16:31.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:30 smithi193 ceph-mon[36001]: osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] boot 2024-05-14T15:16:31.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:30 smithi193 ceph-mon[36001]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T15:16:31.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:31.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:30 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:16:31.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:30 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:16:31.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:30 smithi005 ceph-mon[31559]: osd.3 [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] boot 2024-05-14T15:16:31.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:30 smithi005 ceph-mon[31559]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T15:16:31.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi005 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:32.078 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi005 to 8546M 2024-05-14T15:16:32.079 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.079 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:32.079 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:32.079 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.079 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:31 smithi144 ceph-mon[35650]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:32.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 -- 172.21.15.144:0/4134589533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c103250 msgr2=0x7f905c103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:32.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4134589533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c103250 0x7f905c103660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9058009990 tx=0x7f9058009ca0 comp rx=0 tx=0).stop 2024-05-14T15:16:32.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 -- 172.21.15.144:0/4134589533 shutdown_connections 2024-05-14T15:16:32.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4134589533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905c104d20 0x7f905c10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:32.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4134589533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c103250 0x7f905c103660 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:32.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4134589533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c104380 0x7f905c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:32.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 -- 172.21.15.144:0/4134589533 >> 172.21.15.144:0/4134589533 conn(0x7f905c0fe890 msgr2=0x7f905c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:16:32.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 -- 172.21.15.144:0/4134589533 shutdown_connections 2024-05-14T15:16:32.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.191+0000 7f9062bc2700 1 -- 172.21.15.144:0/4134589533 wait complete. 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 Processor -- start 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 -- start start 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c103250 0x7f905c107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 0x7f905c1056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905c104d20 0x7f905c105c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f905c10ed10 con 0x7f905c103250 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f905c10eb90 con 0x7f905c104380 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9062bc2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f905c10ee90 con 0x7f905c104d20 2024-05-14T15:16:32.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 0x7f905c1056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 0x7f905c1056c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34708/0 (socket says 172.21.15.144:34708) 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 -- 172.21.15.144:0/4064321404 learned_addr learned my addr 172.21.15.144:0/4064321404 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f906095e700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c103250 0x7f905c107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f906115f700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905c104d20 0x7f905c105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 -- 172.21.15.144:0/4064321404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905c104d20 msgr2=0x7f905c105c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905c104d20 0x7f905c105c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 -- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c103250 msgr2=0x7f905c107010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c103250 0x7f905c107010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.192+0000 7f9053fff700 1 -- 172.21.15.144:0/4064321404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c1062d0 con 0x7f905c104380 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.193+0000 7f9053fff700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 0x7f905c1056c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c9b0 tx=0x7f904c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.193+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c007b00 con 0x7f905c104380 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.193+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c007ca0 con 0x7f905c104380 2024-05-14T15:16:32.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.193+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c1a4a70 con 0x7f905c104380 2024-05-14T15:16:32.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.193+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c018660 con 0x7f905c104380 2024-05-14T15:16:32.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.193+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1a4f40 con 0x7f905c104380 2024-05-14T15:16:32.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.194+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c0512c0 con 0x7f905c104380 2024-05-14T15:16:32.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.195+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f904c00f450 con 0x7f905c104380 2024-05-14T15:16:32.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.196+0000 7f9051ffb700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f90440b2880 0x7f90440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:32.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.196+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f904c0e1290 con 0x7f905c104380 2024-05-14T15:16:32.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.196+0000 7f906095e700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f90440b2880 0x7f90440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:32.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.197+0000 7f906095e700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f90440b2880 0x7f90440b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90580038a0 tx=0x7f905801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:16:32.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.200+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f904c014030 con 0x7f905c104380 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi005 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi005 to 8546M 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:32.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:32.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:31 smithi005 ceph-mon[31559]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T15:16:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi005 2024-05-14T15:16:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi005 to 8546M 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:31 smithi193 ceph-mon[36001]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:32.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:32.408+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f905c1a51d0 con 0x7f90440b2880 2024-05-14T15:16:33.153 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:32 smithi144 ceph-mon[35650]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T15:16:33.153 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:32 smithi144 ceph-mon[35650]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:33.153 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:33.154 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:33.154 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:33.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:32 smithi005 ceph-mon[31559]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T15:16:33.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:32 smithi005 ceph-mon[31559]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:33.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:33.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:33.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:33.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:32 smithi193 ceph-mon[36001]: osdmap e28: 4 total, 4 up, 4 in 2024-05-14T15:16:33.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:32 smithi193 ceph-mon[36001]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:33.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:33.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:33.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:33.973 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:33 smithi144 ceph-mon[35650]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:34.300 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:33 smithi193 ceph-mon[36001]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:34.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:33 smithi005 ceph-mon[31559]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:35.432 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:35 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/2418653996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]: dispatch 2024-05-14T15:16:35.432 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:35 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]: dispatch 2024-05-14T15:16:35.432 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:35 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]': finished 2024-05-14T15:16:35.432 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:35 smithi144 ceph-mon[35650]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T15:16:35.432 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:35 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/2418653996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]: dispatch 2024-05-14T15:16:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:35 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]: dispatch 2024-05-14T15:16:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:35 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]': finished 2024-05-14T15:16:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:35 smithi193 ceph-mon[36001]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T15:16:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:35.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/2418653996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]: dispatch 2024-05-14T15:16:35.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:35 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]: dispatch 2024-05-14T15:16:35.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:35 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8db2181-5e0b-407f-9ce9-3a23a0898b57"}]': finished 2024-05-14T15:16:35.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:35 smithi005 ceph-mon[31559]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T15:16:35.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:36.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:36 smithi144 ceph-mon[35650]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:36.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:36 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/1951165151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:16:36.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:36 smithi193 ceph-mon[36001]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:36.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:36 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/1951165151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:16:36.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:36 smithi005 ceph-mon[31559]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:36.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:36 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/1951165151' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:16:38.771 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:38 smithi144 ceph-mon[35650]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-14T15:16:38.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:38 smithi193 ceph-mon[36001]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-14T15:16:38.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:38 smithi005 ceph-mon[31559]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-14T15:16:40.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:40 smithi144 ceph-mon[35650]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:16:40.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:40 smithi193 ceph-mon[36001]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:16:40.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:40 smithi005 ceph-mon[31559]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:16:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:42 smithi144 ceph-mon[35650]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-14T15:16:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T15:16:42.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:42.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:42 smithi193 ceph-mon[36001]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-14T15:16:42.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T15:16:42.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:42.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:42 smithi005 ceph-mon[31559]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-14T15:16:42.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T15:16:42.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:43.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:43 smithi144 ceph-mon[35650]: Deploying daemon osd.4 on smithi144 2024-05-14T15:16:43.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:43 smithi193 ceph-mon[36001]: Deploying daemon osd.4 on smithi144 2024-05-14T15:16:43.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:43 smithi005 ceph-mon[31559]: Deploying daemon osd.4 on smithi144 2024-05-14T15:16:44.774 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:44 smithi144 ceph-mon[35650]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:16:44.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:44 smithi193 ceph-mon[36001]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:16:44.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:44 smithi005 ceph-mon[31559]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:16:46.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:46 smithi144 ceph-mon[35650]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-14T15:16:46.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:46 smithi193 ceph-mon[36001]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-14T15:16:46.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:46 smithi005 ceph-mon[31559]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-14T15:16:48.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:48 smithi144 ceph-mon[35650]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T15:16:48.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:48.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:48.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:48.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:48 smithi193 ceph-mon[36001]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T15:16:48.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:48.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:48.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:48.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:48 smithi005 ceph-mon[31559]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T15:16:48.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:48.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:48.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:50.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:50 smithi144 ceph-mon[35650]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:50.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:50 smithi193 ceph-mon[36001]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:50.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:50 smithi005 ceph-mon[31559]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:52.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.077+0000 7f9051ffb700 1 -- 172.21.15.144:0/4064321404 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f905c1a51d0 con 0x7f90440b2880 2024-05-14T15:16:52.077 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-05-14T15:16:52.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f90440b2880 msgr2=0x7f90440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:52.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f90440b2880 0x7f90440b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90580038a0 tx=0x7f905801a040 comp rx=0 tx=0).stop 2024-05-14T15:16:52.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 msgr2=0x7f905c1056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:52.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 0x7f905c1056c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c9b0 tx=0x7f904c00ccc0 comp rx=0 tx=0).stop 2024-05-14T15:16:52.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 shutdown_connections 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f90440b2880 0x7f90440b4d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f905c104d20 0x7f905c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c103250 0x7f905c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 --2- 172.21.15.144:0/4064321404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c104380 0x7f905c1056c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 >> 172.21.15.144:0/4064321404 conn(0x7f905c0fe890 msgr2=0x7f905c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 shutdown_connections 2024-05-14T15:16:52.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:16:52.080+0000 7f9062bc2700 1 -- 172.21.15.144:0/4064321404 wait complete. 2024-05-14T15:16:52.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:52.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:52.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:52 smithi144 ceph-mon[35650]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:52.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:52.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:52.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:52 smithi193 ceph-mon[36001]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:52.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:52.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:52.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:52.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:52 smithi005 ceph-mon[31559]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:52.732 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.4.service 2024-05-14T15:16:52.734 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-05-14T15:16:52.734 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T15:16:52.980 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:16:53.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:53.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:53.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:53.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:53 smithi144 ceph-mon[35650]: from='osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T15:16:53.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:53 smithi144 ceph-mon[35650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T15:16:53.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:53.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:53.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:53.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:53 smithi005 ceph-mon[31559]: from='osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T15:16:53.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:53 smithi005 ceph-mon[31559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T15:16:53.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:53.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:53.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:16:53.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:53 smithi193 ceph-mon[36001]: from='osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T15:16:53.360 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:53 smithi193 ceph-mon[36001]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T15:16:54.166 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:16:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:16:54.077+0000 7f68d4db0700 -1 osd.4 0 waiting for initial osdmap 2024-05-14T15:16:54.166 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:16:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:16:54.082+0000 7f68d0b65700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:16:54.166 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:54 smithi144 ceph-mon[35650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T15:16:54.166 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:54 smithi144 ceph-mon[35650]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T15:16:54.166 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:54.166 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:54 smithi144 ceph-mon[35650]: from='osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:16:54.166 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:54 smithi144 ceph-mon[35650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:16:54.166 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:54 smithi144 ceph-mon[35650]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:54.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:54 smithi193 ceph-mon[36001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T15:16:54.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:54 smithi193 ceph-mon[36001]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T15:16:54.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:54.383 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:54 smithi193 ceph-mon[36001]: from='osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:16:54.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:54 smithi193 ceph-mon[36001]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:16:54.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:54 smithi193 ceph-mon[36001]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:54.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:54 smithi005 ceph-mon[31559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T15:16:54.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:54 smithi005 ceph-mon[31559]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T15:16:54.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:54.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:54 smithi005 ceph-mon[31559]: from='osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:16:54.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:54 smithi005 ceph-mon[31559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:16:54.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:54 smithi005 ceph-mon[31559]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:54.926 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T15:16:54.926 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T15:16:54.926 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T15:16:54.926 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T15:16:54.926 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117524 s, 892 MB/s 2024-05-14T15:16:54.926 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T15:16:55.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:55 smithi144 ceph-mon[35650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:16:55.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:55 smithi144 ceph-mon[35650]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T15:16:55.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:55 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:55.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:55 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:55.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:55 smithi193 ceph-mon[36001]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:16:55.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:55 smithi193 ceph-mon[36001]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T15:16:55.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:55 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:55.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:55 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:55.524 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-05-14T15:16:55.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:55 smithi005 ceph-mon[31559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:16:55.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:55 smithi005 ceph-mon[31559]: osdmap e31: 5 total, 4 up, 5 in 2024-05-14T15:16:55.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:55.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:55 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:55.770 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:16:56.468 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] boot 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi144 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi144 to 8544M 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:56.469 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:56.470 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:16:56.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:16:56.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] boot 2024-05-14T15:16:56.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T15:16:56.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi144 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi144 to 8544M 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:16:56.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:16:56.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: osd.4 [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] boot 2024-05-14T15:16:56.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T15:16:56.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:16:56.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi144 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi144 to 8544M 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:16:56.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 -- 172.21.15.193:0/6645820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 msgr2=0x7fcca81058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 --2- 172.21.15.193:0/6645820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca81058b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009990 tx=0x7fcc98009ca0 comp rx=0 tx=0).stop 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 -- 172.21.15.193:0/6645820 shutdown_connections 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 --2- 172.21.15.193:0/6645820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8105e40 0x7fcca810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 --2- 172.21.15.193:0/6645820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8104370 0x7fcca8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 --2- 172.21.15.193:0/6645820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca81058b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 -- 172.21.15.193:0/6645820 >> 172.21.15.193:0/6645820 conn(0x7fcca80ff9d0 msgr2=0x7fcca8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 -- 172.21.15.193:0/6645820 shutdown_connections 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.279+0000 7fccafbe9700 1 -- 172.21.15.193:0/6645820 wait complete. 2024-05-14T15:16:57.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 Processor -- start 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 -- start start 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8104370 0x7fcca819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8105e40 0x7fcca81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810fea0 con 0x7fcca8105e40 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810fd20 con 0x7fcca8104370 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccafbe9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8110020 con 0x7fcca81054a0 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccad184700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:57.282 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccad184700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:36888/0 (socket says 172.21.15.193:36888) 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccad184700 1 -- 172.21.15.193:0/3498189241 learned_addr learned my addr 172.21.15.193:0/3498189241 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccad985700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8104370 0x7fcca819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.280+0000 7fccae186700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8105e40 0x7fcca81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccad184700 1 -- 172.21.15.193:0/3498189241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8104370 msgr2=0x7fcca819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccad184700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8104370 0x7fcca819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccad184700 1 -- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8105e40 msgr2=0x7fcca81a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccad184700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8105e40 0x7fcca81a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccad184700 1 -- 172.21.15.193:0/3498189241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81a24f0 con 0x7fcca81054a0 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccae186700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8105e40 0x7fcca81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccad184700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca819aed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006010 tx=0x7fcc980047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98027070 con 0x7fcca81054a0 2024-05-14T15:16:57.283 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81a2780 con 0x7fcca81054a0 2024-05-14T15:16:57.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc98022470 con 0x7fcca81054a0 2024-05-14T15:16:57.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800f6c0 con 0x7fcca81054a0 2024-05-14T15:16:57.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.281+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81a2cb0 con 0x7fcca81054a0 2024-05-14T15:16:57.284 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.282+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca80512c0 con 0x7fcca81054a0 2024-05-14T15:16:57.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.284+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fcc9802a070 con 0x7fcca81054a0 2024-05-14T15:16:57.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.285+0000 7fcc9effd700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcc940b2880 0x7fcc940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:16:57.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.285+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fcc980e25c0 con 0x7fcca81054a0 2024-05-14T15:16:57.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.285+0000 7fccad985700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcc940b2880 0x7fcc940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:16:57.285 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.286+0000 7fccad985700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcc940b2880 0x7fcc940b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcca81053a0 tx=0x7fcca400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:16:57.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.289+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc98031080 con 0x7fcca81054a0 2024-05-14T15:16:57.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:57 smithi193 ceph-mon[36001]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T15:16:57.502 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:16:57.503+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcca8063c00 con 0x7fcc940b2880 2024-05-14T15:16:57.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:57 smithi144 ceph-mon[35650]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T15:16:57.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:57 smithi005 ceph-mon[31559]: osdmap e33: 5 total, 5 up, 5 in 2024-05-14T15:16:58.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:58 smithi193 ceph-mon[36001]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:16:58.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:58 smithi193 ceph-mon[36001]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:58.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:58.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:58.393 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:16:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:58.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:58 smithi144 ceph-mon[35650]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:16:58.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:58 smithi144 ceph-mon[35650]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:58.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:58.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:58.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:16:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:16:58.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:58 smithi005 ceph-mon[31559]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:16:58.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:58 smithi005 ceph-mon[31559]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:16:58.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:16:58.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:16:58.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:16:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:00.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:00 smithi144 ceph-mon[35650]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:00.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:00 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/3167017554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]: dispatch 2024-05-14T15:17:00.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:00 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]: dispatch 2024-05-14T15:17:00.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:00 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]': finished 2024-05-14T15:17:00.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:00 smithi144 ceph-mon[35650]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T15:17:00.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:00 smithi193 ceph-mon[36001]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:00 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/3167017554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]: dispatch 2024-05-14T15:17:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:00 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]: dispatch 2024-05-14T15:17:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:00 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]': finished 2024-05-14T15:17:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:00 smithi193 ceph-mon[36001]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T15:17:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:00 smithi005 ceph-mon[31559]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:00 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/3167017554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]: dispatch 2024-05-14T15:17:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:00 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]: dispatch 2024-05-14T15:17:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:00 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3301b223-bb64-4c9e-a532-508e7e2e2e8f"}]': finished 2024-05-14T15:17:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:00 smithi005 ceph-mon[31559]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T15:17:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:01.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:01 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/4203014186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:01.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:01 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/4203014186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:01.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:01 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/4203014186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:02.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:02 smithi144 ceph-mon[35650]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:02.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:02 smithi193 ceph-mon[36001]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:02.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:02 smithi005 ceph-mon[31559]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:04 smithi144 ceph-mon[35650]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:04.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:04 smithi193 ceph-mon[36001]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:04.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:04 smithi005 ceph-mon[31559]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:06.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:06 smithi193 ceph-mon[36001]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:06.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:06 smithi144 ceph-mon[35650]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:06.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:06 smithi005 ceph-mon[31559]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:07.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T15:17:07.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:07.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T15:17:07.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:07.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T15:17:07.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:08.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:08 smithi193 ceph-mon[36001]: Deploying daemon osd.5 on smithi193 2024-05-14T15:17:08.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:08 smithi193 ceph-mon[36001]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:08.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:08 smithi144 ceph-mon[35650]: Deploying daemon osd.5 on smithi193 2024-05-14T15:17:08.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:08 smithi144 ceph-mon[35650]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:08.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:08 smithi005 ceph-mon[31559]: Deploying daemon osd.5 on smithi193 2024-05-14T15:17:08.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:08 smithi005 ceph-mon[31559]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:10.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:10 smithi144 ceph-mon[35650]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:10.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:10 smithi193 ceph-mon[36001]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:10 smithi005 ceph-mon[31559]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:12.760 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:12 smithi193 ceph-mon[36001]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:12.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:12 smithi144 ceph-mon[35650]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:12.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:12 smithi005 ceph-mon[31559]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:13.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:13.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:13.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:13.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:13.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:13.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:13.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:13.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:15.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:14 smithi193 ceph-mon[36001]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:15.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:14 smithi144 ceph-mon[35650]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:15.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:14 smithi005 ceph-mon[31559]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:16.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:15 smithi193 ceph-mon[36001]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:16.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:15 smithi144 ceph-mon[35650]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:16.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:15 smithi005 ceph-mon[31559]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:16.471 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-05-14T15:17:16.471 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.470+0000 7fcc9effd700 1 -- 172.21.15.193:0/3498189241 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcca8063c00 con 0x7fcc940b2880 2024-05-14T15:17:16.472 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcc940b2880 msgr2=0x7fcc940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:16.472 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcc940b2880 0x7fcc940b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcca81053a0 tx=0x7fcca400c040 comp rx=0 tx=0).stop 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 msgr2=0x7fcca819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca819aed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006010 tx=0x7fcc980047b0 comp rx=0 tx=0).stop 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 shutdown_connections 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcc940b2880 0x7fcc940b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcca8105e40 0x7fcca81a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8104370 0x7fcca819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 --2- 172.21.15.193:0/3498189241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcca81054a0 0x7fcca819aed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:16.473 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 >> 172.21.15.193:0/3498189241 conn(0x7fcca80ff9d0 msgr2=0x7fcca81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:17:16.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 shutdown_connections 2024-05-14T15:17:16.474 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:17:16.473+0000 7fccafbe9700 1 -- 172.21.15.193:0/3498189241 wait complete. 2024-05-14T15:17:17.123 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.5.service 2024-05-14T15:17:17.125 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-05-14T15:17:17.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T15:17:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:17:17.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:17 smithi005 ceph-mon[31559]: from='osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T15:17:17.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:17 smithi005 ceph-mon[31559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T15:17:17.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:17 smithi144 ceph-mon[35650]: from='osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T15:17:17.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:17 smithi144 ceph-mon[35650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T15:17:17.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:17.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:17 smithi193 ceph-mon[36001]: from='osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T15:17:17.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:17 smithi193 ceph-mon[36001]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T15:17:18.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:18 smithi005 ceph-mon[31559]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:18.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:18 smithi005 ceph-mon[31559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T15:17:18.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:18 smithi005 ceph-mon[31559]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T15:17:18.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:18 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:18.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:18 smithi005 ceph-mon[31559]: from='osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:17:18.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:18 smithi005 ceph-mon[31559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:17:18.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:18 smithi193 ceph-mon[36001]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:18.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:18 smithi193 ceph-mon[36001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T15:17:18.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:18 smithi193 ceph-mon[36001]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T15:17:18.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:18 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:18.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:18 smithi193 ceph-mon[36001]: from='osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:17:18.820 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:18 smithi193 ceph-mon[36001]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:17:19.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:18 smithi144 ceph-mon[35650]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:19.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:18 smithi144 ceph-mon[35650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T15:17:19.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:18 smithi144 ceph-mon[35650]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T15:17:19.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:18 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:19.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:18 smithi144 ceph-mon[35650]: from='osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:17:19.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:18 smithi144 ceph-mon[35650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:17:19.301 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:17:19 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:17:19.011+0000 7f5dc2fb8700 -1 osd.5 0 waiting for initial osdmap 2024-05-14T15:17:19.302 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:17:19 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:17:19.018+0000 7f5dbad65700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:17:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T15:17:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T15:17:19.356 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-14T15:17:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-14T15:17:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841413 s, 1.2 GB/s 2024-05-14T15:17:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-14T15:17:19.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:19 smithi005 ceph-mon[31559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:17:19.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:19 smithi005 ceph-mon[31559]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T15:17:19.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:19.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:19.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:19 smithi005 ceph-mon[31559]: from='osd.5 ' entity='osd.5' 2024-05-14T15:17:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:19 smithi193 ceph-mon[36001]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:17:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:19 smithi193 ceph-mon[36001]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T15:17:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:19.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:19 smithi193 ceph-mon[36001]: from='osd.5 ' entity='osd.5' 2024-05-14T15:17:19.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-05-14T15:17:20.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:19 smithi144 ceph-mon[35650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:17:20.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:19 smithi144 ceph-mon[35650]: osdmap e36: 6 total, 5 up, 6 in 2024-05-14T15:17:20.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:20.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:20.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:19 smithi144 ceph-mon[35650]: from='osd.5 ' entity='osd.5' 2024-05-14T15:17:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] boot 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:20 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:17:21.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:17:21.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] boot 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:21.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:21.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:21.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:20 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:17:21.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: osd.5 [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] boot 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:21.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:21.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:20 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 -- 172.21.15.5:0/741573387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104370 msgr2=0x7f9d00104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 --2- 172.21.15.5:0/741573387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104370 0x7f9d00104780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009990 tx=0x7f9cf4009ca0 comp rx=0 tx=0).stop 2024-05-14T15:17:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 -- 172.21.15.5:0/741573387 shutdown_connections 2024-05-14T15:17:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 --2- 172.21.15.5:0/741573387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00104d10 0x7f9d0010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 --2- 172.21.15.5:0/741573387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d00103240 0x7f9d00103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 --2- 172.21.15.5:0/741573387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104370 0x7f9d00104780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 -- 172.21.15.5:0/741573387 >> 172.21.15.5:0/741573387 conn(0x7f9d000fe880 msgr2=0x7f9d00100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:17:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 -- 172.21.15.5:0/741573387 shutdown_connections 2024-05-14T15:17:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.592+0000 7f9d053fe700 1 -- 172.21.15.5:0/741573387 wait complete. 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 Processor -- start 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 -- start start 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00103240 0x7f9d0019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d00104370 0x7f9d0019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 0x7f9d001a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010eca0 con 0x7f9d00104d10 2024-05-14T15:17:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010eb20 con 0x7f9d00104370 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9d053fe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0010ee20 con 0x7f9d00103240 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 0x7f9d001a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9cff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 0x7f9d001a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56774/0 (socket says 172.21.15.5:56774) 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.593+0000 7f9cff7fe700 1 -- 172.21.15.5:0/938284049 learned_addr learned my addr 172.21.15.5:0/938284049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cfe7fc700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d00104370 0x7f9d0019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cfeffd700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00103240 0x7f9d0019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cff7fe700 1 -- 172.21.15.5:0/938284049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00103240 msgr2=0x7f9d0019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cff7fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00103240 0x7f9d0019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cff7fe700 1 -- 172.21.15.5:0/938284049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d00104370 msgr2=0x7f9d0019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cff7fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d00104370 0x7f9d0019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cff7fe700 1 -- 172.21.15.5:0/938284049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d001a24e0 con 0x7f9d00104d10 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cfeffd700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00103240 0x7f9d0019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:17:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9cff7fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 0x7f9d001a1e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007ee0 tx=0x7f9cf800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800ddd0 con 0x7f9d00104d10 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf8004020 con 0x7f9d00104d10 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d001a27d0 con 0x7f9d00104d10 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8015660 con 0x7f9d00104d10 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.594+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d001a2cc0 con 0x7f9d00104d10 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.595+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d000512c0 con 0x7f9d00104d10 2024-05-14T15:17:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.598+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9cf80041d0 con 0x7f9d00104d10 2024-05-14T15:17:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.599+0000 7f9ce7fff700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9ce80b2880 0x7f9ce80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.599+0000 7f9cfeffd700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9ce80b2880 0x7f9ce80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.599+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f9cf80e1600 con 0x7f9d00104d10 2024-05-14T15:17:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.599+0000 7f9cfeffd700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9ce80b2880 0x7f9ce80b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9d00104270 tx=0x7f9cf0009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:17:21.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.602+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cf8010070 con 0x7f9d00104d10 2024-05-14T15:17:21.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:21 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi193 2024-05-14T15:17:21.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:21 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi193 to 8544M 2024-05-14T15:17:21.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:21 smithi005 ceph-mon[31559]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T15:17:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:21.830+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9d00100910 con 0x7f9ce80b2880 2024-05-14T15:17:22.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:21 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi193 2024-05-14T15:17:22.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:21 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi193 to 8544M 2024-05-14T15:17:22.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:21 smithi144 ceph-mon[35650]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T15:17:22.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:21 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi193 2024-05-14T15:17:22.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:21 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi193 to 8544M 2024-05-14T15:17:22.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:21 smithi193 ceph-mon[36001]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T15:17:22.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:22 smithi005 ceph-mon[31559]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:22.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:22 smithi005 ceph-mon[31559]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T15:17:22.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:22 smithi005 ceph-mon[31559]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:17:22.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:17:22.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:17:22.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:22.947 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:22 smithi144 ceph-mon[35650]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:22.947 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:22 smithi144 ceph-mon[35650]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T15:17:22.947 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:22 smithi144 ceph-mon[35650]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:17:22.947 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:17:22.947 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:17:22.947 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:22.952 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:22 smithi193 ceph-mon[36001]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:22.953 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:22 smithi193 ceph-mon[36001]: osdmap e39: 6 total, 6 up, 6 in 2024-05-14T15:17:22.953 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:22 smithi193 ceph-mon[36001]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:17:22.953 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:17:22.953 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:17:22.953 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:23.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:23 smithi005 ceph-mon[31559]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-14T15:17:23.969 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:23 smithi193 ceph-mon[36001]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-14T15:17:24.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:23 smithi144 ceph-mon[35650]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-14T15:17:25.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:24 smithi144 ceph-mon[35650]: pgmap v140: 1 pgs: 1 activating+degraded; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-14T15:17:25.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:24 smithi193 ceph-mon[36001]: pgmap v140: 1 pgs: 1 activating+degraded; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-14T15:17:25.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:24 smithi005 ceph-mon[31559]: pgmap v140: 1 pgs: 1 activating+degraded; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-14T15:17:26.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/540742819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "024a549a-dc28-482e-a03d-4e4d5987e94b"}]: dispatch 2024-05-14T15:17:26.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/540742819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "024a549a-dc28-482e-a03d-4e4d5987e94b"}]': finished 2024-05-14T15:17:26.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:25 smithi144 ceph-mon[35650]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T15:17:26.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:25 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:26.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3320708765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:26.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/540742819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "024a549a-dc28-482e-a03d-4e4d5987e94b"}]: dispatch 2024-05-14T15:17:26.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/540742819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "024a549a-dc28-482e-a03d-4e4d5987e94b"}]': finished 2024-05-14T15:17:26.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:25 smithi193 ceph-mon[36001]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T15:17:26.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:26.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3320708765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:26.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/540742819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "024a549a-dc28-482e-a03d-4e4d5987e94b"}]: dispatch 2024-05-14T15:17:26.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/540742819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "024a549a-dc28-482e-a03d-4e4d5987e94b"}]': finished 2024-05-14T15:17:26.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:25 smithi005 ceph-mon[31559]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T15:17:26.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:25 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:26.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3320708765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:27.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:26 smithi144 ceph-mon[35650]: pgmap v142: 1 pgs: 1 activating+degraded; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-14T15:17:27.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:26 smithi193 ceph-mon[36001]: pgmap v142: 1 pgs: 1 activating+degraded; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-14T15:17:27.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:26 smithi005 ceph-mon[31559]: pgmap v142: 1 pgs: 1 activating+degraded; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-14T15:17:28.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:27 smithi144 ceph-mon[35650]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-14T15:17:28.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:27 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:17:28.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:27 smithi193 ceph-mon[36001]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-14T15:17:28.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:27 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:17:28.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:27 smithi005 ceph-mon[31559]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-14T15:17:28.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:27 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:17:29.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:28 smithi005 ceph-mon[31559]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-14T15:17:29.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:28 smithi144 ceph-mon[35650]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-14T15:17:29.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:28 smithi193 ceph-mon[36001]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-14T15:17:31.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:30 smithi005 ceph-mon[31559]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:17:31.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:30 smithi144 ceph-mon[35650]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:17:31.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:30 smithi193 ceph-mon[36001]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:17:33.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:32 smithi005 ceph-mon[31559]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T15:17:33.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T15:17:33.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:33.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:32 smithi144 ceph-mon[35650]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T15:17:33.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T15:17:33.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:33.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:32 smithi193 ceph-mon[36001]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-14T15:17:33.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T15:17:33.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:34.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:33 smithi144 ceph-mon[35650]: Deploying daemon osd.6 on smithi005 2024-05-14T15:17:34.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:33 smithi005 ceph-mon[31559]: Deploying daemon osd.6 on smithi005 2024-05-14T15:17:34.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:33 smithi193 ceph-mon[36001]: Deploying daemon osd.6 on smithi005 2024-05-14T15:17:35.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:34 smithi193 ceph-mon[36001]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:17:35.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:34 smithi005 ceph-mon[31559]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:17:35.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:34 smithi144 ceph-mon[35650]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:17:37.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:36 smithi005 ceph-mon[31559]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-14T15:17:37.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:36 smithi144 ceph-mon[35650]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-14T15:17:37.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:36 smithi193 ceph-mon[36001]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-14T15:17:39.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:38 smithi005 ceph-mon[31559]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T15:17:39.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:39.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:39.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:39.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:38 smithi144 ceph-mon[35650]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T15:17:39.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:39.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:39.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:39.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:38 smithi193 ceph-mon[36001]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T15:17:39.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:39.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:39.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:40.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:40 smithi005 ceph-mon[31559]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:41.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:40 smithi144 ceph-mon[35650]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:41.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:40 smithi193 ceph-mon[36001]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.609+0000 7f9ce7fff700 1 -- 172.21.15.5:0/938284049 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9d00100910 con 0x7f9ce80b2880 2024-05-14T15:17:41.609 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-05-14T15:17:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9ce80b2880 msgr2=0x7f9ce80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9ce80b2880 0x7f9ce80b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9d00104270 tx=0x7f9cf0009670 comp rx=0 tx=0).stop 2024-05-14T15:17:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 msgr2=0x7f9d001a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 0x7f9d001a1e60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007ee0 tx=0x7f9cf800d3b0 comp rx=0 tx=0).stop 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 shutdown_connections 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9ce80b2880 0x7f9ce80b4d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d00104d10 0x7f9d001a1e60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d00103240 0x7f9d0019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 --2- 172.21.15.5:0/938284049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d00104370 0x7f9d0019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.611+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 >> 172.21.15.5:0/938284049 conn(0x7f9d000fe880 msgr2=0x7f9d00100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.612+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 shutdown_connections 2024-05-14T15:17:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:17:41.612+0000 7f9d053fe700 1 -- 172.21.15.5:0/938284049 wait complete. 2024-05-14T15:17:42.123 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.6.service 2024-05-14T15:17:42.125 INFO:tasks.cephadm:Deploying osd.7 on smithi144 with /dev/vg_nvme/lv_2... 2024-05-14T15:17:42.125 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T15:17:42.372 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:17:42.786 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:42 smithi144 ceph-mon[35650]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:42.786 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.787 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.787 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:42 smithi144 ceph-mon[35650]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T15:17:42.787 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.787 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:42 smithi005 ceph-mon[31559]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:42 smithi005 ceph-mon[31559]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T15:17:42.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:42.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:43.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:42 smithi193 ceph-mon[36001]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:43.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:43.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:43.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:42 smithi193 ceph-mon[36001]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T15:17:43.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:43.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:44.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:43 smithi005 ceph-mon[31559]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T15:17:44.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:43 smithi005 ceph-mon[31559]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T15:17:44.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:43 smithi005 ceph-mon[31559]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:17:44.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:44.097 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:17:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:17:43.857+0000 7fe63fa1a700 -1 osd.6 0 waiting for initial osdmap 2024-05-14T15:17:44.097 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:17:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:17:43.865+0000 7fe63b7cf700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:17:44.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:43 smithi144 ceph-mon[35650]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T15:17:44.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:43 smithi144 ceph-mon[35650]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T15:17:44.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:43 smithi144 ceph-mon[35650]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:17:44.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:44.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:43 smithi193 ceph-mon[36001]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T15:17:44.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:43 smithi193 ceph-mon[36001]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T15:17:44.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:43 smithi193 ceph-mon[36001]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:17:44.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:44.448 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T15:17:44.448 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T15:17:44.448 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-14T15:17:44.448 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-14T15:17:44.448 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113555 s, 923 MB/s 2024-05-14T15:17:44.448 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-14T15:17:45.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:44 smithi144 ceph-mon[35650]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:45.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:44 smithi144 ceph-mon[35650]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:17:45.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:44 smithi144 ceph-mon[35650]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T15:17:45.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:44 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:45.115 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:44 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:45.143 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-05-14T15:17:45.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:44 smithi193 ceph-mon[36001]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:45.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:44 smithi193 ceph-mon[36001]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:17:45.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:44 smithi193 ceph-mon[36001]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T15:17:45.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:44 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:45.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:44 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:45.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:44 smithi005 ceph-mon[31559]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-14T15:17:45.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:44 smithi005 ceph-mon[31559]: from='osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:17:45.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:44 smithi005 ceph-mon[31559]: osdmap e42: 7 total, 6 up, 7 in 2024-05-14T15:17:45.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:44 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:45.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:44 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:45.387 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:17:46.188 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:45 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:17:46.291 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:45 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:17:46.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:45 smithi144 ceph-mon[35650]: osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] boot 2024-05-14T15:17:46.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:45 smithi144 ceph-mon[35650]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T15:17:46.292 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:45 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:46.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:45 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:17:46.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:45 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:17:46.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:45 smithi193 ceph-mon[36001]: osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] boot 2024-05-14T15:17:46.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:45 smithi193 ceph-mon[36001]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T15:17:46.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:45 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:46.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:45 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:17:46.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:45 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:17:46.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:45 smithi005 ceph-mon[31559]: osd.6 [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] boot 2024-05-14T15:17:46.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:45 smithi005 ceph-mon[31559]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T15:17:46.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:45 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:17:46.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 -- 172.21.15.144:0/2058263743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8107470 msgr2=0x7f2ba8109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/2058263743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8107470 0x7f2ba8109950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009990 tx=0x7f2b90009ca0 comp rx=0 tx=0).stop 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 -- 172.21.15.144:0/2058263743 shutdown_connections 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/2058263743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ba8109fe0 0x7f2ba810c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/2058263743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ba806cd00 0x7f2ba8106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/2058263743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8107470 0x7f2ba8109950 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 -- 172.21.15.144:0/2058263743 >> 172.21.15.144:0/2058263743 conn(0x7f2ba80fe870 msgr2=0x7f2ba8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 -- 172.21.15.144:0/2058263743 shutdown_connections 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.974+0000 7f2bad2f3700 1 -- 172.21.15.144:0/2058263743 wait complete. 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 Processor -- start 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 -- start start 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ba806cd00 0x7f2ba81046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:46.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ba8107470 0x7f2ba8102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 0x7f2ba81032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba810fe50 con 0x7f2ba806cd00 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba810fcd0 con 0x7f2ba8109fe0 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.975+0000 7f2bad2f3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba810ffd0 con 0x7f2ba8107470 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 0x7f2ba81032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 0x7f2ba81032e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48746/0 (socket says 172.21.15.144:48746) 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 -- 172.21.15.144:0/3012131034 learned_addr learned my addr 172.21.15.144:0/3012131034 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba6ffd700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ba806cd00 0x7f2ba81046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2b9e7fc700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ba8107470 0x7f2ba8102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:46.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 -- 172.21.15.144:0/3012131034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ba8107470 msgr2=0x7f2ba8102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ba8107470 0x7f2ba8102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 -- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ba806cd00 msgr2=0x7f2ba81046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ba806cd00 0x7f2ba81046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 -- 172.21.15.144:0/3012131034 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba8103a40 con 0x7f2ba8109fe0 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba77fe700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 0x7f2ba81032e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b970 tx=0x7f2ba000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000d860 con 0x7f2ba8109fe0 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba81a4ac0 con 0x7f2ba8109fe0 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba000e430 con 0x7f2ba8109fe0 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0017660 con 0x7f2ba8109fe0 2024-05-14T15:17:46.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.976+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba81a4fa0 con 0x7f2ba8109fe0 2024-05-14T15:17:46.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.977+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba80512c0 con 0x7f2ba8109fe0 2024-05-14T15:17:46.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.979+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2ba000da00 con 0x7f2ba8109fe0 2024-05-14T15:17:46.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.979+0000 7f2ba4ff9700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2b940b2880 0x7f2b940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:17:46.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.980+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f2ba00e12c0 con 0x7f2ba8109fe0 2024-05-14T15:17:46.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.980+0000 7f2ba6ffd700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2b940b2880 0x7f2b940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:17:46.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.980+0000 7f2ba6ffd700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2b940b2880 0x7f2b940b4d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009a10 tx=0x7f2b98009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:17:46.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:46.983+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ba0053df0 con 0x7f2ba8109fe0 2024-05-14T15:17:47.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:47.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:17:47.193+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2ba8063c00 con 0x7f2b940b2880 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: osdmap e44: 7 total, 7 up, 7 in 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:17:47.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:47.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:17:47.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:17:48.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:47 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi005 2024-05-14T15:17:48.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:47 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi005 to 5697M 2024-05-14T15:17:48.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:17:48.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:17:48.132 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:48.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:47 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi005 2024-05-14T15:17:48.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:47 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi005 to 5697M 2024-05-14T15:17:48.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:17:48.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:17:48.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:48.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:47 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi005 2024-05-14T15:17:48.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:47 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi005 to 5697M 2024-05-14T15:17:48.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:17:48.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:17:48.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:49.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:48 smithi144 ceph-mon[35650]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:17:49.002 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:48 smithi144 ceph-mon[35650]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:49.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:48 smithi193 ceph-mon[36001]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:17:49.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:48 smithi193 ceph-mon[36001]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:49.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:48 smithi005 ceph-mon[31559]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:17:49.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:48 smithi005 ceph-mon[31559]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:50.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:49 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/1351833518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]: dispatch 2024-05-14T15:17:50.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:49 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]: dispatch 2024-05-14T15:17:50.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:49 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]': finished 2024-05-14T15:17:50.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:49 smithi144 ceph-mon[35650]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T15:17:50.189 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:17:50.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:49 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/1351833518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]: dispatch 2024-05-14T15:17:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:49 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]: dispatch 2024-05-14T15:17:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:49 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]': finished 2024-05-14T15:17:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:49 smithi193 ceph-mon[36001]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T15:17:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:17:50.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:49 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/1351833518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]: dispatch 2024-05-14T15:17:50.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:49 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]: dispatch 2024-05-14T15:17:50.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:49 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aebc627-3769-4770-b617-721385d1cb23"}]': finished 2024-05-14T15:17:50.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:49 smithi005 ceph-mon[31559]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T15:17:50.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:17:51.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:50 smithi144 ceph-mon[35650]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:51.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:50 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/1226000209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:51.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:50 smithi193 ceph-mon[36001]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:51.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:50 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/1226000209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:51.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:50 smithi005 ceph-mon[31559]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:51.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:50 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/1226000209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:17:53.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:52 smithi144 ceph-mon[35650]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:53.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:52 smithi193 ceph-mon[36001]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:53.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:52 smithi005 ceph-mon[31559]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:54.142 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:53 smithi144 ceph-mon[35650]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:54.301 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:53 smithi193 ceph-mon[36001]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:54.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:53 smithi005 ceph-mon[31559]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:56.788 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:56 smithi144 ceph-mon[35650]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:56.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:56 smithi193 ceph-mon[36001]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:56.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:56 smithi005 ceph-mon[31559]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:57.786 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:57 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T15:17:57.786 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:57 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:57.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:57 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T15:17:57.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:57 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:57.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T15:17:57.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:17:58.531 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:58 smithi144 ceph-mon[35650]: Deploying daemon osd.7 on smithi144 2024-05-14T15:17:58.531 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:17:58 smithi144 ceph-mon[35650]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:58.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:58 smithi193 ceph-mon[36001]: Deploying daemon osd.7 on smithi144 2024-05-14T15:17:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:17:58 smithi193 ceph-mon[36001]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:17:58.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:58 smithi005 ceph-mon[31559]: Deploying daemon osd.7 on smithi144 2024-05-14T15:17:58.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:17:58 smithi005 ceph-mon[31559]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:00.760 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:00 smithi144 ceph-mon[35650]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:00.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:00 smithi193 ceph-mon[36001]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:00.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:00 smithi005 ceph-mon[31559]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:02.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:02 smithi144 ceph-mon[35650]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:02.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:02 smithi193 ceph-mon[36001]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:02.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:02 smithi005 ceph-mon[31559]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:03.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:03.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:03.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:04.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:04.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:04.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:04.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:04 smithi144 ceph-mon[35650]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:05.051 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:04 smithi193 ceph-mon[36001]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:05.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:04 smithi005 ceph-mon[31559]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:06.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.015+0000 7f2ba4ff9700 1 -- 172.21.15.144:0/3012131034 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ba8063c00 con 0x7f2b940b2880 2024-05-14T15:18:06.013 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 7 on host 'smithi144' 2024-05-14T15:18:06.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2b940b2880 msgr2=0x7f2b940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2b940b2880 0x7f2b940b4d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009a10 tx=0x7f2b98009190 comp rx=0 tx=0).stop 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 msgr2=0x7f2ba81032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 0x7f2ba81032e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b970 tx=0x7f2ba000bc80 comp rx=0 tx=0).stop 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 shutdown_connections 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2b940b2880 0x7f2b940b4d90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ba8109fe0 0x7f2ba81032e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ba806cd00 0x7f2ba81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 --2- 172.21.15.144:0/3012131034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ba8107470 0x7f2ba8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 >> 172.21.15.144:0/3012131034 conn(0x7f2ba80fe870 msgr2=0x7f2ba81058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 shutdown_connections 2024-05-14T15:18:06.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:18:06.017+0000 7f2bad2f3700 1 -- 172.21.15.144:0/3012131034 wait complete. 2024-05-14T15:18:06.649 DEBUG:teuthology.orchestra.run.smithi144:osd.7> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.7.service 2024-05-14T15:18:06.652 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-05-14T15:18:06.652 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T15:18:06.898 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:18:07.020 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:06 smithi193 ceph-mon[36001]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:07.021 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:06 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:07.021 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:06 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:07.021 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:06 smithi193 ceph-mon[36001]: from='osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T15:18:07.021 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:06 smithi193 ceph-mon[36001]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T15:18:07.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:06 smithi144 ceph-mon[35650]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:07.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:06 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:07.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:06 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:07.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:06 smithi144 ceph-mon[35650]: from='osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T15:18:07.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:06 smithi144 ceph-mon[35650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T15:18:07.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:06 smithi005 ceph-mon[31559]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:07.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:06 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:07.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:06 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:07.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:06 smithi005 ceph-mon[31559]: from='osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T15:18:07.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:06 smithi005 ceph-mon[31559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: from='osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:18:08.041 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:07 smithi193 ceph-mon[36001]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: from='osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:18:08.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:07 smithi144 ceph-mon[35650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:18:08.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T15:18:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T15:18:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: from='osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:18:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:07 smithi005 ceph-mon[31559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:18:08.683 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:18:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:18:08.420+0000 7f9eae527700 -1 osd.7 0 waiting for initial osdmap 2024-05-14T15:18:08.683 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:18:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:18:08.433+0000 7f9ea62d4700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:18:08.969 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T15:18:08.970 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T15:18:08.970 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T15:18:08.970 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T15:18:08.970 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132029 s, 794 MB/s 2024-05-14T15:18:08.970 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T15:18:08.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:08 smithi193 ceph-mon[36001]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:08.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:08 smithi193 ceph-mon[36001]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:18:08.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:08 smithi193 ceph-mon[36001]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T15:18:08.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:08.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:08.970 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:08 smithi193 ceph-mon[36001]: from='osd.7 ' entity='osd.7' 2024-05-14T15:18:09.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:08 smithi144 ceph-mon[35650]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:09.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:08 smithi144 ceph-mon[35650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:18:09.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:08 smithi144 ceph-mon[35650]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T15:18:09.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:09.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:09.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:08 smithi144 ceph-mon[35650]: from='osd.7 ' entity='osd.7' 2024-05-14T15:18:09.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:08 smithi005 ceph-mon[31559]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:09.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:08 smithi005 ceph-mon[31559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:18:09.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:08 smithi005 ceph-mon[31559]: osdmap e47: 8 total, 7 up, 8 in 2024-05-14T15:18:09.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:09.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:09.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:08 smithi005 ceph-mon[31559]: from='osd.7 ' entity='osd.7' 2024-05-14T15:18:09.605 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-05-14T15:18:09.852 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:18:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:10 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:18:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:10 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:18:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:10 smithi193 ceph-mon[36001]: osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] boot 2024-05-14T15:18:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:10 smithi193 ceph-mon[36001]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T15:18:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:10 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:10.230 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:10 smithi193 ceph-mon[36001]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:10.295 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:10 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:18:10.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:10 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:18:10.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:10 smithi144 ceph-mon[35650]: osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] boot 2024-05-14T15:18:10.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:10 smithi144 ceph-mon[35650]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T15:18:10.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:10 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:10.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:10 smithi144 ceph-mon[35650]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:10.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:10 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:18:10.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:10 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:18:10.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:10 smithi005 ceph-mon[31559]: osd.7 [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] boot 2024-05-14T15:18:10.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:10 smithi005 ceph-mon[31559]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T15:18:10.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:10 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:18:10.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:10 smithi005 ceph-mon[31559]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi144 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi144 to 5696M 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.251 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:11.252 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:11.252 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:11.252 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:11 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T15:18:11.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi144 2024-05-14T15:18:11.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi144 to 5696M 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:11 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi144 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi144 to 5696M 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:11.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:11.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:11.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:11 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:11.491 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 -- 172.21.15.193:0/1950000015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102ca0 msgr2=0x7f706010c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:11.491 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 --2- 172.21.15.193:0/1950000015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102ca0 0x7f706010c8c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f705800b0a0 tx=0x7f705800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:18:11.492 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 -- 172.21.15.193:0/1950000015 shutdown_connections 2024-05-14T15:18:11.492 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 --2- 172.21.15.193:0/1950000015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102ca0 0x7f706010c8c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:11.492 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 --2- 172.21.15.193:0/1950000015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060105900 0x7f7060105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:11.492 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 --2- 172.21.15.193:0/1950000015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060102350 0x7f7060102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:11.492 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 -- 172.21.15.193:0/1950000015 >> 172.21.15.193:0/1950000015 conn(0x7f70600fe020 msgr2=0x7f70601004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:18:11.492 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 -- 172.21.15.193:0/1950000015 shutdown_connections 2024-05-14T15:18:11.493 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 -- 172.21.15.193:0/1950000015 wait complete. 2024-05-14T15:18:11.493 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 Processor -- start 2024-05-14T15:18:11.493 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.487+0000 7f7067f70700 1 -- start start 2024-05-14T15:18:11.493 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 0x7f706019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:11.493 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060102ca0 0x7f706019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:11.493 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060105900 0x7f70601a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f706010fdc0 con 0x7f7060105900 2024-05-14T15:18:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f706010fc40 con 0x7f7060102ca0 2024-05-14T15:18:11.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f706010ff40 con 0x7f7060102350 2024-05-14T15:18:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 0x7f706019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 0x7f706019a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:52570/0 (socket says 172.21.15.193:52570) 2024-05-14T15:18:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 -- 172.21.15.193:0/2966520485 learned_addr learned my addr 172.21.15.193:0/2966520485 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:18:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f706650d700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060105900 0x7f70601a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f706550b700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060102ca0 0x7f706019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 -- 172.21.15.193:0/2966520485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060102ca0 msgr2=0x7f706019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:11.496 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060102ca0 0x7f706019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:11.496 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 -- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060105900 msgr2=0x7f70601a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:11.496 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060105900 0x7f70601a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:11.496 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 -- 172.21.15.193:0/2966520485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70601a2310 con 0x7f7060102350 2024-05-14T15:18:11.496 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f706550b700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060102ca0 0x7f706019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:18:11.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7065d0c700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 0x7f706019a840 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f705000d950 tx=0x7f705000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:18:11.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050016070 con 0x7f7060102350 2024-05-14T15:18:11.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f705000be00 con 0x7f7060102350 2024-05-14T15:18:11.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.488+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70601a2600 con 0x7f7060102350 2024-05-14T15:18:11.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.489+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705000f3c0 con 0x7f7060102350 2024-05-14T15:18:11.497 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.489+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70601a2b40 con 0x7f7060102350 2024-05-14T15:18:11.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.489+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70600512c0 con 0x7f7060102350 2024-05-14T15:18:11.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.491+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7050019070 con 0x7f7060102350 2024-05-14T15:18:11.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.491+0000 7f7056ffd700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f704c0b2880 0x7f704c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:11.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.492+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f70500e1a90 con 0x7f7060102350 2024-05-14T15:18:11.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.492+0000 7f706550b700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f704c0b2880 0x7f704c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:11.498 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.492+0000 7f706550b700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f704c0b2880 0x7f704c0b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f706019b8f0 tx=0x7f705c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:18:11.499 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.495+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7050020020 con 0x7f7060102350 2024-05-14T15:18:11.704 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:11.705+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7060063c00 con 0x7f704c0b2880 2024-05-14T15:18:12.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:12 smithi193 ceph-mon[36001]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T15:18:12.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:12 smithi193 ceph-mon[36001]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:12.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:12 smithi193 ceph-mon[36001]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:18:12.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:18:12.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:18:12.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:12 smithi144 ceph-mon[35650]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T15:18:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:12 smithi144 ceph-mon[35650]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:12 smithi144 ceph-mon[35650]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:18:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:18:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:18:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:12.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:12 smithi005 ceph-mon[31559]: osdmap e50: 8 total, 8 up, 8 in 2024-05-14T15:18:12.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:12 smithi005 ceph-mon[31559]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:12.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:12 smithi005 ceph-mon[31559]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:18:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:18:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:18:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:14.755 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:14 smithi193 ceph-mon[36001]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:14.755 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:14 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/1315531524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]: dispatch 2024-05-14T15:18:14.755 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:14 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]: dispatch 2024-05-14T15:18:14.755 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:14 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]': finished 2024-05-14T15:18:14.755 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:14 smithi193 ceph-mon[36001]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T15:18:14.755 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:14 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:14.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:14 smithi144 ceph-mon[35650]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:14 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/1315531524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]: dispatch 2024-05-14T15:18:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:14 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]: dispatch 2024-05-14T15:18:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:14 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]': finished 2024-05-14T15:18:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:14 smithi144 ceph-mon[35650]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T15:18:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:14 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:14 smithi005 ceph-mon[31559]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:14 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/1315531524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]: dispatch 2024-05-14T15:18:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:14 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]: dispatch 2024-05-14T15:18:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:14 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb0f1f87-2738-4488-8c2b-eac7983424ef"}]': finished 2024-05-14T15:18:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:14 smithi005 ceph-mon[31559]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T15:18:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:14 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:15.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:15 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/372983204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:18:15.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:15 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/372983204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:18:15.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:15 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/372983204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:18:16.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:16 smithi144 ceph-mon[35650]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T15:18:16.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:16 smithi193 ceph-mon[36001]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T15:18:16.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:16 smithi005 ceph-mon[31559]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T15:18:18.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:18 smithi144 ceph-mon[35650]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T15:18:18.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:18 smithi193 ceph-mon[36001]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T15:18:18.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:18 smithi005 ceph-mon[31559]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-14T15:18:20.685 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:20 smithi193 ceph-mon[36001]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:18:20.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:20 smithi144 ceph-mon[35650]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:18:20.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:20 smithi005 ceph-mon[31559]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:18:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:22 smithi144 ceph-mon[35650]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T15:18:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:22.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:22 smithi193 ceph-mon[36001]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:22.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T15:18:22.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:22 smithi005 ceph-mon[31559]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T15:18:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:23.775 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:23 smithi193 ceph-mon[36001]: Deploying daemon osd.8 on smithi193 2024-05-14T15:18:23.795 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:23 smithi144 ceph-mon[35650]: Deploying daemon osd.8 on smithi193 2024-05-14T15:18:23.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:23 smithi005 ceph-mon[31559]: Deploying daemon osd.8 on smithi193 2024-05-14T15:18:24.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:24 smithi144 ceph-mon[35650]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:24.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:24 smithi193 ceph-mon[36001]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:24.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:24 smithi005 ceph-mon[31559]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:26.710 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:26 smithi193 ceph-mon[36001]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-14T15:18:26.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:26 smithi144 ceph-mon[35650]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-14T15:18:26.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:26 smithi005 ceph-mon[31559]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-14T15:18:28.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:28 smithi193 ceph-mon[36001]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:28.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:28.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:28.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:29.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:28 smithi144 ceph-mon[35650]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:29.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:29.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:29.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:29.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:28 smithi005 ceph-mon[31559]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:29.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:29.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:29.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:30.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:30 smithi193 ceph-mon[36001]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:31.045 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:30 smithi144 ceph-mon[35650]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:31.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:30 smithi005 ceph-mon[31559]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:31.178 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-05-14T15:18:31.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.178+0000 7f7056ffd700 1 -- 172.21.15.193:0/2966520485 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7060063c00 con 0x7f704c0b2880 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f704c0b2880 msgr2=0x7f704c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f704c0b2880 0x7f704c0b4d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f706019b8f0 tx=0x7f705c00a680 comp rx=0 tx=0).stop 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 msgr2=0x7f706019a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 0x7f706019a840 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f705000d950 tx=0x7f705000dc60 comp rx=0 tx=0).stop 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 shutdown_connections 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f704c0b2880 0x7f704c0b4d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060105900 0x7f70601a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7060102350 0x7f706019a840 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 --2- 172.21.15.193:0/2966520485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060102ca0 0x7f706019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 >> 172.21.15.193:0/2966520485 conn(0x7f70600fe020 msgr2=0x7f70601088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:18:31.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 shutdown_connections 2024-05-14T15:18:31.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:18:31.181+0000 7f7067f70700 1 -- 172.21.15.193:0/2966520485 wait complete. 2024-05-14T15:18:31.863 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.8.service 2024-05-14T15:18:31.866 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-05-14T15:18:31.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T15:18:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: from='osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T15:18:32.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:32 smithi005 ceph-mon[31559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: from='osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T15:18:32.398 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:32 smithi193 ceph-mon[36001]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: from='osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T15:18:32.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:32 smithi144 ceph-mon[35650]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T15:18:33.342 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:33 smithi193 ceph-mon[36001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T15:18:33.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:33 smithi193 ceph-mon[36001]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T15:18:33.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:33 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:33.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:33 smithi193 ceph-mon[36001]: from='osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:18:33.343 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:33 smithi193 ceph-mon[36001]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:18:33.343 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:18:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:18:33.231+0000 7fd0ed2a3700 -1 osd.8 0 waiting for initial osdmap 2024-05-14T15:18:33.343 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:18:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:18:33.240+0000 7fd0e9058700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:18:33.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:33 smithi005 ceph-mon[31559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T15:18:33.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:33 smithi005 ceph-mon[31559]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T15:18:33.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:33 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:33.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:33 smithi005 ceph-mon[31559]: from='osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:18:33.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:33 smithi005 ceph-mon[31559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:18:33.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:33 smithi144 ceph-mon[35650]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T15:18:33.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:33 smithi144 ceph-mon[35650]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T15:18:33.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:33 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:33.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:33 smithi144 ceph-mon[35650]: from='osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:18:33.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:33 smithi144 ceph-mon[35650]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:18:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T15:18:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T15:18:34.240 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-05-14T15:18:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-05-14T15:18:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00937377 s, 1.1 GB/s 2024-05-14T15:18:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-05-14T15:18:34.442 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:34 smithi193 ceph-mon[36001]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:18:34.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:34 smithi193 ceph-mon[36001]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T15:18:34.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:34.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:34.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:34 smithi193 ceph-mon[36001]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:34.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:34 smithi144 ceph-mon[35650]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:18:34.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:34 smithi144 ceph-mon[35650]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T15:18:34.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:34.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:34.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:34 smithi144 ceph-mon[35650]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:34.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:34 smithi005 ceph-mon[31559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:18:34.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:34 smithi005 ceph-mon[31559]: osdmap e53: 9 total, 8 up, 9 in 2024-05-14T15:18:34.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:34.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:34.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:34 smithi005 ceph-mon[31559]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-14T15:18:34.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-05-14T15:18:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:18:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:18:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:18:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] boot 2024-05-14T15:18:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T15:18:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi193 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi193 to 5696M 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:35.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:35.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] boot 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi193 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi193 to 5696M 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:35.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:18:35.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:18:35.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:35.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: osd.8 [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] boot 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi193 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi193 to 5696M 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:35.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:35.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:35.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:36.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:36 smithi005 ceph-mon[31559]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T15:18:36.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:36 smithi005 ceph-mon[31559]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:36.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:36 smithi144 ceph-mon[35650]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T15:18:36.545 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:36 smithi144 ceph-mon[35650]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:36.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:36 smithi193 ceph-mon[36001]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T15:18:36.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:36 smithi193 ceph-mon[36001]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 -- 172.21.15.5:0/1460257944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d541054a0 msgr2=0x7f0d541058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1460257944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d541054a0 0x7f0d541058b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009990 tx=0x7f0d48009ca0 comp rx=0 tx=0).stop 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 -- 172.21.15.5:0/1460257944 shutdown_connections 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1460257944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 0x7f0d5410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1460257944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d54104370 0x7f0d54104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1460257944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d541054a0 0x7f0d541058b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:36.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 -- 172.21.15.5:0/1460257944 >> 172.21.15.5:0/1460257944 conn(0x7f0d540ff9d0 msgr2=0x7f0d54101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:18:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 -- 172.21.15.5:0/1460257944 shutdown_connections 2024-05-14T15:18:36.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 -- 172.21.15.5:0/1460257944 wait complete. 2024-05-14T15:18:36.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.923+0000 7f0d59d93700 1 Processor -- start 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 -- start start 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 0x7f0d54078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d541054a0 0x7f0d54076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 0x7f0d54077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5410fdd0 con 0x7f0d54104370 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5410fc50 con 0x7f0d54105e40 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d59d93700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5410ff50 con 0x7f0d541054a0 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 0x7f0d54078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 0x7f0d54078570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39180/0 (socket says 172.21.15.5:39180) 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 -- 172.21.15.5:0/1568403645 learned_addr learned my addr 172.21.15.5:0/1568403645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d52ffd700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d541054a0 0x7f0d54076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d53fff700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 0x7f0d54077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 -- 172.21.15.5:0/1568403645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d541054a0 msgr2=0x7f0d54076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d541054a0 0x7f0d54076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 -- 172.21.15.5:0/1568403645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 msgr2=0x7f0d54077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 0x7f0d54077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 -- 172.21.15.5:0/1568403645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540778c0 con 0x7f0d54104370 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d52ffd700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d541054a0 0x7f0d54076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d53fff700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 0x7f0d54077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.924+0000 7f0d537fe700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 0x7f0d54078570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400d950 tx=0x7f0d4400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.925+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44017070 con 0x7f0d54104370 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.925+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d541a49e0 con 0x7f0d54104370 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.925+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4400de30 con 0x7f0d54104370 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.925+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44003b10 con 0x7f0d54104370 2024-05-14T15:18:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.925+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d541a4ec0 con 0x7f0d54104370 2024-05-14T15:18:36.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.926+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d540512c0 con 0x7f0d54104370 2024-05-14T15:18:36.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.927+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0d44003cb0 con 0x7f0d54104370 2024-05-14T15:18:36.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.928+0000 7f0d50ff9700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:18:36.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.928+0000 7f0d52ffd700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:18:36.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.928+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f0d440e1cf0 con 0x7f0d54104370 2024-05-14T15:18:36.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.929+0000 7f0d52ffd700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0d480038a0 tx=0x7f0d4801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:18:36.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:36.932+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d44020030 con 0x7f0d54104370 2024-05-14T15:18:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:37.155+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0d5402ea50 con 0x7f0d3c0b2880 2024-05-14T15:18:37.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:37 smithi144 ceph-mon[35650]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T15:18:37.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:37 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:18:37.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:37 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:18:37.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:37 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:37.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:37 smithi193 ceph-mon[36001]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T15:18:37.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:37 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:18:37.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:37 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:18:37.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:37 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:37.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:37 smithi005 ceph-mon[31559]: osdmap e56: 9 total, 9 up, 9 in 2024-05-14T15:18:37.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:37 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:18:37.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:37 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:18:37.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:37 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:38.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:38 smithi144 ceph-mon[35650]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:18:38.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:38 smithi144 ceph-mon[35650]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:38.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:38 smithi193 ceph-mon[36001]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:18:38.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:38 smithi193 ceph-mon[36001]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:38.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:38 smithi005 ceph-mon[31559]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:18:38.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:38 smithi005 ceph-mon[31559]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:40.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:40 smithi005 ceph-mon[31559]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:40.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:40 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/28270737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038f4941-b7d8-4fb3-8e34-dda9b11b012a"}]: dispatch 2024-05-14T15:18:40.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:40 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/28270737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038f4941-b7d8-4fb3-8e34-dda9b11b012a"}]': finished 2024-05-14T15:18:40.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:40 smithi005 ceph-mon[31559]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T15:18:40.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:40.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:40 smithi144 ceph-mon[35650]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:40.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:40 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/28270737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038f4941-b7d8-4fb3-8e34-dda9b11b012a"}]: dispatch 2024-05-14T15:18:40.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:40 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/28270737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038f4941-b7d8-4fb3-8e34-dda9b11b012a"}]': finished 2024-05-14T15:18:40.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:40 smithi144 ceph-mon[35650]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T15:18:40.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:40 smithi193 ceph-mon[36001]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:40 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/28270737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038f4941-b7d8-4fb3-8e34-dda9b11b012a"}]: dispatch 2024-05-14T15:18:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:40 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/28270737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038f4941-b7d8-4fb3-8e34-dda9b11b012a"}]': finished 2024-05-14T15:18:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:40 smithi193 ceph-mon[36001]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T15:18:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:41.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:41 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/718519055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:18:41.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:41 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/718519055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:18:41.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:41 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/718519055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:18:42.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:42 smithi144 ceph-mon[35650]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T15:18:42.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:42 smithi193 ceph-mon[36001]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T15:18:42.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:42 smithi005 ceph-mon[31559]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T15:18:44.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:44 smithi144 ceph-mon[35650]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:18:44.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:44 smithi193 ceph-mon[36001]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:18:44.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:44 smithi005 ceph-mon[31559]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:18:46.494 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:46 smithi005 ceph-mon[31559]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T15:18:46.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:46 smithi144 ceph-mon[35650]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T15:18:46.801 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:46 smithi193 ceph-mon[36001]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T15:18:47.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T15:18:47.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:47 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:47.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T15:18:47.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:47 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:47.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T15:18:47.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:47 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:48.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:48 smithi005 ceph-mon[31559]: Deploying daemon osd.9 on smithi005 2024-05-14T15:18:48.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:48 smithi005 ceph-mon[31559]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:48 smithi144 ceph-mon[35650]: Deploying daemon osd.9 on smithi005 2024-05-14T15:18:48.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:48 smithi144 ceph-mon[35650]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:48.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:48 smithi193 ceph-mon[36001]: Deploying daemon osd.9 on smithi005 2024-05-14T15:18:48.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:48 smithi193 ceph-mon[36001]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:50.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:50 smithi005 ceph-mon[31559]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:50 smithi144 ceph-mon[35650]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:50 smithi193 ceph-mon[36001]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:18:52.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:52 smithi005 ceph-mon[31559]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-14T15:18:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:52 smithi144 ceph-mon[35650]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-14T15:18:52.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:52 smithi193 ceph-mon[36001]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-14T15:18:54.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:54.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:54.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:54.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:54 smithi144 ceph-mon[35650]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:54.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:54.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:54.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:54.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:54 smithi193 ceph-mon[36001]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:54.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:54.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:54.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:54.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:54 smithi005 ceph-mon[31559]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:56.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:56 smithi005 ceph-mon[31559]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:56.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:56.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:56.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:56 smithi144 ceph-mon[35650]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:56.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:56 smithi193 ceph-mon[36001]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:56.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:56.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:18:56.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:18:56.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.367+0000 7f0d50ff9700 1 -- 172.21.15.5:0/1568403645 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d48003940 con 0x7f0d3c0b2880 2024-05-14T15:18:57.367 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d3c0b2880 msgr2=0x7f0d3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0d480038a0 tx=0x7f0d4801a040 comp rx=0 tx=0).stop 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 msgr2=0x7f0d54078570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 0x7f0d54078570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400d950 tx=0x7f0d4400dd10 comp rx=0 tx=0).stop 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 shutdown_connections 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d3c0b2880 0x7f0d3c0b4d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d54105e40 0x7f0d54077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d54104370 0x7f0d54078570 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 --2- 172.21.15.5:0/1568403645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d541054a0 0x7f0d54076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.369+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 >> 172.21.15.5:0/1568403645 conn(0x7f0d540ff9d0 msgr2=0x7f0d541086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.370+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 shutdown_connections 2024-05-14T15:18:57.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:18:57.370+0000 7f0d59d93700 1 -- 172.21.15.5:0/1568403645 wait complete. 2024-05-14T15:18:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:57 smithi005 ceph-mon[31559]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T15:18:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:57 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:57 smithi144 ceph-mon[35650]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T15:18:57.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:57 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:57 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:57.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:57 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:57 smithi193 ceph-mon[36001]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T15:18:57.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:57 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:57 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:18:57.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:57 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:18:57.974 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.9.service 2024-05-14T15:18:57.976 INFO:tasks.cephadm:Deploying osd.10 on smithi144 with /dev/vg_nvme/lv_1... 2024-05-14T15:18:57.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T15:18:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:18:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:58 smithi144 ceph-mon[35650]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:58 smithi144 ceph-mon[35650]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T15:18:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:58 smithi144 ceph-mon[35650]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T15:18:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:58 smithi144 ceph-mon[35650]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:18:58.598 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:58 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:58 smithi193 ceph-mon[36001]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:58 smithi193 ceph-mon[36001]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T15:18:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:58 smithi193 ceph-mon[36001]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T15:18:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:58 smithi193 ceph-mon[36001]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:18:58.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:58 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:58.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:58 smithi005 ceph-mon[31559]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:18:58.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:58 smithi005 ceph-mon[31559]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T15:18:58.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:58 smithi005 ceph-mon[31559]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T15:18:58.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:58 smithi005 ceph-mon[31559]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-05-14T15:18:58.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:58 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.246 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:18:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:18:58.962+0000 7fb9ec8d5700 -1 osd.9 0 waiting for initial osdmap 2024-05-14T15:18:59.246 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:18:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:18:58.976+0000 7fb9e4682700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:18:59.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:59 smithi144 ceph-mon[35650]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:18:59.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:59 smithi144 ceph-mon[35650]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T15:18:59.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:59 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:59 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:59 smithi144 ceph-mon[35650]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' 2024-05-14T15:18:59.637 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:18:59 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:59 smithi193 ceph-mon[36001]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:18:59.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:59 smithi193 ceph-mon[36001]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T15:18:59.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:59 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:59 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:59 smithi193 ceph-mon[36001]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' 2024-05-14T15:18:59.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:18:59 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:59 smithi005 ceph-mon[31559]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-05-14T15:18:59.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:59 smithi005 ceph-mon[31559]: osdmap e59: 10 total, 9 up, 10 in 2024-05-14T15:18:59.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:59 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:59 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:18:59.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:59 smithi005 ceph-mon[31559]: from='osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668]' entity='osd.9' 2024-05-14T15:18:59.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:18:59 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:19:00.357 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T15:19:00.358 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T15:19:00.358 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-05-14T15:19:00.358 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-05-14T15:19:00.358 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113283 s, 926 MB/s 2024-05-14T15:19:00.358 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-05-14T15:19:00.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:00 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:19:00.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:00 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:19:00.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:00 smithi144 ceph-mon[35650]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:19:00.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:00 smithi144 ceph-mon[35650]: osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] boot 2024-05-14T15:19:00.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:00 smithi144 ceph-mon[35650]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T15:19:00.661 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:00 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:19:00.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:00 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:19:00.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:00 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:19:00.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:00 smithi005 ceph-mon[31559]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:19:00.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:00 smithi005 ceph-mon[31559]: osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] boot 2024-05-14T15:19:00.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:00 smithi005 ceph-mon[31559]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T15:19:00.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:00 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:19:00.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:00 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:19:00.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:00 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:19:00.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:00 smithi193 ceph-mon[36001]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-14T15:19:00.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:00 smithi193 ceph-mon[36001]: osd.9 [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] boot 2024-05-14T15:19:00.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:00 smithi193 ceph-mon[36001]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T15:19:00.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:00 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:19:01.020 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-05-14T15:19:01.265 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi005 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi005 to 4273M 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:01 smithi144 ceph-mon[35650]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi005 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi005 to 4273M 2024-05-14T15:19:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:01 smithi193 ceph-mon[36001]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: osdmap e61: 10 total, 10 up, 10 in 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi005 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi005 to 4273M 2024-05-14T15:19:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:01 smithi005 ceph-mon[31559]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:02.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 -- 172.21.15.144:0/1718985696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450103250 msgr2=0x7fb450103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:02.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 --2- 172.21.15.144:0/1718985696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450103250 0x7fb450103660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb440009990 tx=0x7fb440009ca0 comp rx=0 tx=0).stop 2024-05-14T15:19:02.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 -- 172.21.15.144:0/1718985696 shutdown_connections 2024-05-14T15:19:02.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 --2- 172.21.15.144:0/1718985696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb450104d20 0x7fb45010b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:02.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 --2- 172.21.15.144:0/1718985696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450103250 0x7fb450103660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:02.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 --2- 172.21.15.144:0/1718985696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450104380 0x7fb450104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:02.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 -- 172.21.15.144:0/1718985696 >> 172.21.15.144:0/1718985696 conn(0x7fb4500fe890 msgr2=0x7fb450100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:02.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 -- 172.21.15.144:0/1718985696 shutdown_connections 2024-05-14T15:19:02.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.891+0000 7fb454df0700 1 -- 172.21.15.144:0/1718985696 wait complete. 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 Processor -- start 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 -- start start 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450103250 0x7fb450199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 0x7fb450199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb450104d20 0x7fb4501a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb45010ecf0 con 0x7fb450103250 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb45010eb70 con 0x7fb450104380 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb454df0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb45010ee70 con 0x7fb450104d20 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb44dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 0x7fb450199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb44dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 0x7fb450199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42730/0 (socket says 172.21.15.144:42730) 2024-05-14T15:19:02.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb44dd9b700 1 -- 172.21.15.144:0/2450496831 learned_addr learned my addr 172.21.15.144:0/2450496831 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.892+0000 7fb44e59c700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450103250 0x7fb450199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44ed9d700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb450104d20 0x7fb4501a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44dd9b700 1 -- 172.21.15.144:0/2450496831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb450104d20 msgr2=0x7fb4501a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44dd9b700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb450104d20 0x7fb4501a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44dd9b700 1 -- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450103250 msgr2=0x7fb450199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44dd9b700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450103250 0x7fb450199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44dd9b700 1 -- 172.21.15.144:0/2450496831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4501a13c0 con 0x7fb450104380 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44e59c700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450103250 0x7fb450199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb44dd9b700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 0x7fb450199da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e9a0 tx=0x7fb44400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44400ca10 con 0x7fb450104380 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4501a16b0 con 0x7fb450104380 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44400cbb0 con 0x7fb450104380 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444018680 con 0x7fb450104380 2024-05-14T15:19:02.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.893+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4501a1c40 con 0x7fb450104380 2024-05-14T15:19:02.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.894+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4500512c0 con 0x7fb450104380 2024-05-14T15:19:02.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.896+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb444010430 con 0x7fb450104380 2024-05-14T15:19:02.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.897+0000 7fb43f7fe700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4380b2900 0x7fb4380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:02.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.897+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7fb4440e2530 con 0x7fb450104380 2024-05-14T15:19:02.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.897+0000 7fb44e59c700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4380b2900 0x7fb4380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:02.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.898+0000 7fb44e59c700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4380b2900 0x7fb4380b4e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb44000b6c0 tx=0x7fb44000b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:02.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:02.900+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb444026070 con 0x7fb450104380 2024-05-14T15:19:03.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:03.109+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb450063c00 con 0x7fb4380b2900 2024-05-14T15:19:03.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:19:03.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:19:03.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:03 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:03.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:19:03.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:19:03.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:03 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:03.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:19:03.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:19:03.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:03 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:04.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:04 smithi144 ceph-mon[35650]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:19:04.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:04 smithi144 ceph-mon[35650]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:04.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:04 smithi193 ceph-mon[36001]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:19:04.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:04 smithi193 ceph-mon[36001]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:04.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:04 smithi005 ceph-mon[31559]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:19:04.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:04 smithi005 ceph-mon[31559]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:06.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:06 smithi144 ceph-mon[35650]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:06.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:06 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/337416844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]: dispatch 2024-05-14T15:19:06.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:06 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]: dispatch 2024-05-14T15:19:06.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:06 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]': finished 2024-05-14T15:19:06.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:06 smithi144 ceph-mon[35650]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T15:19:06.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:06 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:06 smithi193 ceph-mon[36001]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:06 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/337416844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]: dispatch 2024-05-14T15:19:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:06 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]: dispatch 2024-05-14T15:19:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:06 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]': finished 2024-05-14T15:19:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:06 smithi193 ceph-mon[36001]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T15:19:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:06 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:06 smithi005 ceph-mon[31559]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:06 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/337416844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]: dispatch 2024-05-14T15:19:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:06 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]: dispatch 2024-05-14T15:19:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:06 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a017373b-14d7-440d-a525-b6bb24b2690d"}]': finished 2024-05-14T15:19:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:06 smithi005 ceph-mon[31559]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T15:19:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:06 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:07.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:07 smithi144 ceph-mon[35650]: from='client.? 172.21.15.144:0/152500650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:19:07.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:07 smithi193 ceph-mon[36001]: from='client.? 172.21.15.144:0/152500650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:19:07.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:07 smithi005 ceph-mon[31559]: from='client.? 172.21.15.144:0/152500650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:19:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:08 smithi144 ceph-mon[35650]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:08 smithi193 ceph-mon[36001]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:08.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:08 smithi005 ceph-mon[31559]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:10.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:10 smithi144 ceph-mon[35650]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:10.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:10 smithi193 ceph-mon[36001]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:10.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:10 smithi005 ceph-mon[31559]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:12 smithi144 ceph-mon[35650]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:12.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:12 smithi193 ceph-mon[36001]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:12 smithi005 ceph-mon[31559]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:13.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T15:19:13.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:13 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:13.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T15:19:13.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:13 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:13.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T15:19:13.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:13 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:14.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:14 smithi144 ceph-mon[35650]: Deploying daemon osd.10 on smithi144 2024-05-14T15:19:14.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:14 smithi144 ceph-mon[35650]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:14.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:14 smithi193 ceph-mon[36001]: Deploying daemon osd.10 on smithi144 2024-05-14T15:19:14.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:14 smithi193 ceph-mon[36001]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:14.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:14 smithi005 ceph-mon[31559]: Deploying daemon osd.10 on smithi144 2024-05-14T15:19:14.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:14 smithi005 ceph-mon[31559]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:16.711 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:16 smithi144 ceph-mon[35650]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:16.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:16 smithi193 ceph-mon[36001]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:16.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:16 smithi005 ceph-mon[31559]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:18.722 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:18 smithi144 ceph-mon[35650]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:18.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:18 smithi193 ceph-mon[36001]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:18.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:18 smithi005 ceph-mon[31559]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:20.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:20.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:20.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:20.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:19 smithi144 ceph-mon[35650]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:20.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:20.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:20.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:20.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:19 smithi193 ceph-mon[36001]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:20.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:20.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:20.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:20.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:19 smithi005 ceph-mon[31559]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:22 smithi144 ceph-mon[35650]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:22.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:22 smithi193 ceph-mon[36001]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:22.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:22.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:22.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:22 smithi005 ceph-mon[31559]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:22.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:22.881 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 10 on host 'smithi144' 2024-05-14T15:19:22.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.882+0000 7fb43f7fe700 1 -- 172.21.15.144:0/2450496831 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb450063c00 con 0x7fb4380b2900 2024-05-14T15:19:22.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4380b2900 msgr2=0x7fb4380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:22.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4380b2900 0x7fb4380b4e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb44000b6c0 tx=0x7fb44000b720 comp rx=0 tx=0).stop 2024-05-14T15:19:22.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 msgr2=0x7fb450199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:22.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 0x7fb450199da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e9a0 tx=0x7fb44400ecb0 comp rx=0 tx=0).stop 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 shutdown_connections 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4380b2900 0x7fb4380b4e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb450104d20 0x7fb4501a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb450103250 0x7fb450199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 --2- 172.21.15.144:0/2450496831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb450104380 0x7fb450199da0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 >> 172.21.15.144:0/2450496831 conn(0x7fb4500fe890 msgr2=0x7fb4501075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 shutdown_connections 2024-05-14T15:19:22.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-05-14T15:19:22.885+0000 7fb454df0700 1 -- 172.21.15.144:0/2450496831 wait complete. 2024-05-14T15:19:23.557 DEBUG:teuthology.orchestra.run.smithi144:osd.10> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.10.service 2024-05-14T15:19:23.559 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-05-14T15:19:23.559 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T15:19:23.805 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:19:24.177 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:23 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:24.177 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:23 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:24.178 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:23 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:24.178 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:23 smithi193 ceph-mon[36001]: from='osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T15:19:24.178 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:23 smithi193 ceph-mon[36001]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T15:19:24.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:23 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:24.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:23 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:24.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:23 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:24.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:23 smithi144 ceph-mon[35650]: from='osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T15:19:24.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:23 smithi144 ceph-mon[35650]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T15:19:24.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:23 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:24.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:23 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:24.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:23 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:24.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:23 smithi005 ceph-mon[31559]: from='osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T15:19:24.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:23 smithi005 ceph-mon[31559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T15:19:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:24 smithi144 ceph-mon[35650]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:24 smithi144 ceph-mon[35650]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T15:19:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:24 smithi144 ceph-mon[35650]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T15:19:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:24 smithi144 ceph-mon[35650]: from='osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:19:24.864 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:24 smithi144 ceph-mon[35650]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:19:25.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:24 smithi193 ceph-mon[36001]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:25.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:24 smithi193 ceph-mon[36001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T15:19:25.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:24 smithi193 ceph-mon[36001]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T15:19:25.211 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:25.212 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:24 smithi193 ceph-mon[36001]: from='osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:19:25.212 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:24 smithi193 ceph-mon[36001]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:19:25.296 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:19:24 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:19:24.864+0000 7f8d6f115700 -1 osd.10 0 waiting for initial osdmap 2024-05-14T15:19:25.296 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:19:24 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:19:24.875+0000 7f8d6aeca700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:19:25.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:24 smithi005 ceph-mon[31559]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:25.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:24 smithi005 ceph-mon[31559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T15:19:25.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:24 smithi005 ceph-mon[31559]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T15:19:25.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:25.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:24 smithi005 ceph-mon[31559]: from='osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:19:25.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:24 smithi005 ceph-mon[31559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-05-14T15:19:25.956 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T15:19:25.957 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T15:19:25.957 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-05-14T15:19:25.957 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-05-14T15:19:25.957 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112887 s, 929 MB/s 2024-05-14T15:19:25.957 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-05-14T15:19:26.031 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:25 smithi144 ceph-mon[35650]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:19:26.031 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:25 smithi144 ceph-mon[35650]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T15:19:26.031 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:25 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:26.031 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:25 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:26.280 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:25 smithi193 ceph-mon[36001]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:19:26.280 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:25 smithi193 ceph-mon[36001]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T15:19:26.280 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:26.280 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:26.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:25 smithi005 ceph-mon[31559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-05-14T15:19:26.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:25 smithi005 ceph-mon[31559]: osdmap e64: 11 total, 10 up, 11 in 2024-05-14T15:19:26.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:25 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:26.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:25 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:26.692 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-05-14T15:19:26.929 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] boot 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.191 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:26 smithi193 ceph-mon[36001]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T15:19:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:19:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:19:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] boot 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:27.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:26 smithi144 ceph-mon[35650]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T15:19:27.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:19:27.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:19:27.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-14T15:19:27.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: osd.10 [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] boot 2024-05-14T15:19:27.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T15:19:27.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:19:27.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:27.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:27.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:27.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:27.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:26 smithi005 ceph-mon[31559]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T15:19:28.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:27 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi144 2024-05-14T15:19:28.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:27 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi144 to 4272M 2024-05-14T15:19:28.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:27 smithi193 ceph-mon[36001]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T15:19:28.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:27 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi144 2024-05-14T15:19:28.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:27 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi144 to 4272M 2024-05-14T15:19:28.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:27 smithi144 ceph-mon[35650]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T15:19:28.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:27 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi144 2024-05-14T15:19:28.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:27 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi144 to 4272M 2024-05-14T15:19:28.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:27 smithi005 ceph-mon[31559]: osdmap e67: 11 total, 11 up, 11 in 2024-05-14T15:19:28.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3731764833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44104340 msgr2=0x7fbc44104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:28.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3731764833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44104340 0x7fbc44104750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000b0a0 tx=0x7fbc4000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:19:28.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3731764833 shutdown_connections 2024-05-14T15:19:28.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3731764833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 0x7fbc4410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:28.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3731764833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44104340 0x7fbc44104750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3731764833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44105470 0x7fbc44105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3731764833 >> 172.21.15.193:0/3731764833 conn(0x7fbc440ff9c0 msgr2=0x7fbc44101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.563+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3731764833 shutdown_connections 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3731764833 wait complete. 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 Processor -- start 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 -- start start 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44104340 0x7fbc4419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 0x7fbc4419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 0x7fbc441a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4410fe00 con 0x7fbc44104340 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4410fc80 con 0x7fbc44105e10 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc4c65f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4410ff80 con 0x7fbc44105470 2024-05-14T15:19:28.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc49bfa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 0x7fbc4419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.564+0000 7fbc49bfa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 0x7fbc4419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53370/0 (socket says 172.21.15.193:53370) 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 -- 172.21.15.193:0/3615075740 learned_addr learned my addr 172.21.15.193:0/3615075740 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc4a3fb700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44104340 0x7fbc4419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc4abfc700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 0x7fbc441a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 -- 172.21.15.193:0/3615075740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 msgr2=0x7fbc441a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 0x7fbc441a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 -- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44104340 msgr2=0x7fbc4419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44104340 0x7fbc4419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 -- 172.21.15.193:0/3615075740 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc441a24e0 con 0x7fbc44105470 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc4abfc700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 0x7fbc441a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc49bfa700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 0x7fbc4419aec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b530 tx=0x7fbc3400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34015020 con 0x7fbc44105470 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc34011ce0 con 0x7fbc44105470 2024-05-14T15:19:28.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34010e40 con 0x7fbc44105470 2024-05-14T15:19:28.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc441a27d0 con 0x7fbc44105470 2024-05-14T15:19:28.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.565+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc441a2d40 con 0x7fbc44105470 2024-05-14T15:19:28.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.566+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc440512c0 con 0x7fbc44105470 2024-05-14T15:19:28.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.568+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbc34018020 con 0x7fbc44105470 2024-05-14T15:19:28.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.568+0000 7fbc3b7fe700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc300b2900 0x7fbc300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:28.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.569+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7fbc340e1930 con 0x7fbc44105470 2024-05-14T15:19:28.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.569+0000 7fbc4a3fb700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc300b2900 0x7fbc300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:28.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.569+0000 7fbc4a3fb700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc300b2900 0x7fbc300b4e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009fd0 tx=0x7fbc4000b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:28.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.572+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc3401f020 con 0x7fbc44105470 2024-05-14T15:19:28.784 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:28.784+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbc44063c00 con 0x7fbc300b2900 2024-05-14T15:19:29.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:28 smithi144 ceph-mon[35650]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:29.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:19:29.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:19:29.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:29.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:28 smithi193 ceph-mon[36001]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:29.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:19:29.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:19:29.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:29.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:28 smithi005 ceph-mon[31559]: pgmap v229: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:29.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T15:19:29.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T15:19:29.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:30.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:29 smithi193 ceph-mon[36001]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:19:30.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:29 smithi144 ceph-mon[35650]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:19:30.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:29 smithi005 ceph-mon[31559]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:19:31.004 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:30 smithi193 ceph-mon[36001]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:31.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:30 smithi144 ceph-mon[35650]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:31.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:30 smithi005 ceph-mon[31559]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:31 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/3092340869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]: dispatch 2024-05-14T15:19:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:31 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]: dispatch 2024-05-14T15:19:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:31 smithi005 ceph-mon[31559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]': finished 2024-05-14T15:19:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:31 smithi005 ceph-mon[31559]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T15:19:32.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:32.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:31 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/3092340869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]: dispatch 2024-05-14T15:19:32.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:31 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]: dispatch 2024-05-14T15:19:32.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:31 smithi144 ceph-mon[35650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]': finished 2024-05-14T15:19:32.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:31 smithi144 ceph-mon[35650]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T15:19:32.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:31 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/3092340869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]: dispatch 2024-05-14T15:19:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:31 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]: dispatch 2024-05-14T15:19:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:31 smithi193 ceph-mon[36001]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0"}]': finished 2024-05-14T15:19:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:31 smithi193 ceph-mon[36001]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T15:19:32.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:33.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:32 smithi144 ceph-mon[35650]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T15:19:33.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:32 smithi144 ceph-mon[35650]: from='client.? 172.21.15.193:0/1861689281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:19:33.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:32 smithi193 ceph-mon[36001]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T15:19:33.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:32 smithi193 ceph-mon[36001]: from='client.? 172.21.15.193:0/1861689281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:19:33.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:32 smithi005 ceph-mon[31559]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T15:19:33.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:32 smithi005 ceph-mon[31559]: from='client.? 172.21.15.193:0/1861689281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T15:19:35.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:34 smithi144 ceph-mon[35650]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-14T15:19:35.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:34 smithi193 ceph-mon[36001]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-14T15:19:35.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:34 smithi005 ceph-mon[31559]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-14T15:19:36.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:36 smithi193 ceph-mon[36001]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:19:37.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:36 smithi144 ceph-mon[35650]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:19:37.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:36 smithi005 ceph-mon[31559]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T15:19:39.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:38 smithi193 ceph-mon[36001]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-14T15:19:39.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:38 smithi144 ceph-mon[35650]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-14T15:19:39.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:38 smithi005 ceph-mon[31559]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-14T15:19:40.030 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:39 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T15:19:40.030 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:39 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:40.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:39 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T15:19:40.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:39 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:39 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T15:19:40.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:39 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:41.070 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:40 smithi193 ceph-mon[36001]: Deploying daemon osd.11 on smithi193 2024-05-14T15:19:41.071 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:40 smithi193 ceph-mon[36001]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:19:41.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:40 smithi144 ceph-mon[35650]: Deploying daemon osd.11 on smithi193 2024-05-14T15:19:41.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:40 smithi144 ceph-mon[35650]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:19:41.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:40 smithi005 ceph-mon[31559]: Deploying daemon osd.11 on smithi193 2024-05-14T15:19:41.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:40 smithi005 ceph-mon[31559]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T15:19:42.259 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:41 smithi193 ceph-mon[36001]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:42.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:41 smithi144 ceph-mon[35650]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:42.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:41 smithi005 ceph-mon[31559]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:44.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:44 smithi144 ceph-mon[35650]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:44.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:44 smithi193 ceph-mon[36001]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:44.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:44 smithi005 ceph-mon[31559]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:46.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:46.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:46.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:46 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:46.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:46 smithi144 ceph-mon[35650]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:46.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:46.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:46.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:46 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:46.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:46 smithi193 ceph-mon[36001]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:46.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:46.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:46.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:46 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:46.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:46 smithi005 ceph-mon[31559]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:48.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:48 smithi193 ceph-mon[36001]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:48.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:48.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:48.728 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:48 smithi144 ceph-mon[35650]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:48.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:48 smithi005 ceph-mon[31559]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:48.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:48.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:48.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:48.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.205 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.205+0000 7fbc3b7fe700 1 -- 172.21.15.193:0/3615075740 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbc44063c00 con 0x7fbc300b2900 2024-05-14T15:19:49.206 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc300b2900 msgr2=0x7fbc300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc300b2900 0x7fbc300b4e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009fd0 tx=0x7fbc4000b7b0 comp rx=0 tx=0).stop 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 msgr2=0x7fbc4419aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 0x7fbc4419aec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b530 tx=0x7fbc3400b840 comp rx=0 tx=0).stop 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 shutdown_connections 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc300b2900 0x7fbc300b4e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:49.208 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44105e10 0x7fbc441a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:49.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44104340 0x7fbc4419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:49.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 --2- 172.21.15.193:0/3615075740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44105470 0x7fbc4419aec0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:49.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 >> 172.21.15.193:0/3615075740 conn(0x7fbc440ff9c0 msgr2=0x7fbc441086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:49.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 shutdown_connections 2024-05-14T15:19:49.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:19:49.207+0000 7fbc4c65f700 1 -- 172.21.15.193:0/3615075740 wait complete. 2024-05-14T15:19:49.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:49 smithi144 ceph-mon[35650]: from='osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T15:19:49.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:49 smithi144 ceph-mon[35650]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T15:19:49.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:49.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:49 smithi193 ceph-mon[36001]: from='osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T15:19:49.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:49 smithi193 ceph-mon[36001]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T15:19:49.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:49.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:49 smithi005 ceph-mon[31559]: from='osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T15:19:49.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:49 smithi005 ceph-mon[31559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T15:19:49.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:19:49.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:49.911 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.11.service 2024-05-14T15:19:49.914 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-14T15:19:49.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-14T15:19:50.176 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:19:50.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:50 smithi005 ceph-mon[31559]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:50.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:50 smithi005 ceph-mon[31559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T15:19:50.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:50 smithi005 ceph-mon[31559]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T15:19:50.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:50 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:50.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:50 smithi005 ceph-mon[31559]: from='osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:19:50.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:50 smithi005 ceph-mon[31559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:19:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:50 smithi144 ceph-mon[35650]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:50 smithi144 ceph-mon[35650]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T15:19:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:50 smithi144 ceph-mon[35650]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T15:19:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:50 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:50 smithi144 ceph-mon[35650]: from='osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:19:50.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:50 smithi144 ceph-mon[35650]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:19:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:50 smithi193 ceph-mon[36001]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:50 smithi193 ceph-mon[36001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T15:19:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:50 smithi193 ceph-mon[36001]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T15:19:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:50 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:50 smithi193 ceph-mon[36001]: from='osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:19:50.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:50 smithi193 ceph-mon[36001]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-05-14T15:19:51.252 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:19:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:19:50.922+0000 7f5b8ab45700 -1 osd.11 0 waiting for initial osdmap 2024-05-14T15:19:51.252 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:19:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:19:50.936+0000 7f5b828f2700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T15:19:51.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:51 smithi005 ceph-mon[31559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:19:51.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:51 smithi005 ceph-mon[31559]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T15:19:51.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:51 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:51.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:51 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:51.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:51 smithi005 ceph-mon[31559]: from='osd.11 ' entity='osd.11' 2024-05-14T15:19:51.781 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:51 smithi193 ceph-mon[36001]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:19:51.781 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:51 smithi193 ceph-mon[36001]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T15:19:51.781 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:51 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:51.781 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:51 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:51.781 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:51 smithi193 ceph-mon[36001]: from='osd.11 ' entity='osd.11' 2024-05-14T15:19:51.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:51 smithi144 ceph-mon[35650]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-05-14T15:19:51.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:51 smithi144 ceph-mon[35650]: osdmap e70: 12 total, 11 up, 12 in 2024-05-14T15:19:51.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:51 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:51.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:51 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:51.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:51 smithi144 ceph-mon[35650]: from='osd.11 ' entity='osd.11' 2024-05-14T15:19:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 -- 172.21.15.5:0/2066414929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c104d00 msgr2=0x7f7e5c10b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2066414929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c104d00 0x7f7e5c10b5b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400b0a0 tx=0x7f7e5400b3b0 comp rx=0 tx=0).stop 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 -- 172.21.15.5:0/2066414929 shutdown_connections 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2066414929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c104d00 0x7f7e5c10b5b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2066414929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e5c103230 0x7f7e5c103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2066414929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5c104360 0x7f7e5c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 -- 172.21.15.5:0/2066414929 >> 172.21.15.5:0/2066414929 conn(0x7f7e5c0fe870 msgr2=0x7f7e5c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 -- 172.21.15.5:0/2066414929 shutdown_connections 2024-05-14T15:19:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.894+0000 7f7e60afe700 1 -- 172.21.15.5:0/2066414929 wait complete. 2024-05-14T15:19:51.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 Processor -- start 2024-05-14T15:19:51.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 -- start start 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 0x7f7e5c106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e5c104360 0x7f7e5c1056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5c104d00 0x7f7e5c105be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c10ec90 con 0x7f7e5c103230 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c10eb10 con 0x7f7e5c104360 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e60afe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c10ee10 con 0x7f7e5c104d00 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 0x7f7e5c106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 0x7f7e5c106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56998/0 (socket says 172.21.15.5:56998) 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 -- 172.21.15.5:0/2880254778 learned_addr learned my addr 172.21.15.5:0/2880254778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5ad9d700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5c104d00 0x7f7e5c105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e59d9b700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e5c104360 0x7f7e5c1056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 -- 172.21.15.5:0/2880254778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5c104d00 msgr2=0x7f7e5c105be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5c104d00 0x7f7e5c105be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 -- 172.21.15.5:0/2880254778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e5c104360 msgr2=0x7f7e5c1056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e5c104360 0x7f7e5c1056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.895+0000 7f7e5a59c700 1 -- 172.21.15.5:0/2880254778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c106340 con 0x7f7e5c103230 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.896+0000 7f7e5a59c700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 0x7f7e5c106ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00e940 tx=0x7f7e4c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.896+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c004220 con 0x7f7e5c103230 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.896+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c1a4980 con 0x7f7e5c103230 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.896+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c0043c0 con 0x7f7e5c103230 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.896+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c003c60 con 0x7f7e5c103230 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.896+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c1a4eb0 con 0x7f7e5c103230 2024-05-14T15:19:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.897+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c0512c0 con 0x7f7e5c103230 2024-05-14T15:19:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.899+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7e4c00c7e0 con 0x7f7e5c103230 2024-05-14T15:19:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.899+0000 7f7e4b7fe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7e440b2880 0x7f7e440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.900+0000 7f7e59d9b700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7e440b2880 0x7f7e440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.900+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6847+0+0 (secure 0 0 0) 0x7f7e4c0e1ec0 con 0x7f7e5c103230 2024-05-14T15:19:51.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.900+0000 7f7e59d9b700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7e440b2880 0x7f7e440b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c1069a0 tx=0x7f7e50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:51.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:51.903+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e4c0aa230 con 0x7f7e5c103230 2024-05-14T15:19:52.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.120+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7e5c1a52a0 con 0x7f7e5c103230 2024-05-14T15:19:52.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.121+0000 7f7e4b7fe700 1 -- 172.21.15.5:0/2880254778 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f7e4c0a9b30 con 0x7f7e5c103230 2024-05-14T15:19:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:19:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7e440b2880 msgr2=0x7f7e440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7e440b2880 0x7f7e440b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c1069a0 tx=0x7f7e50009260 comp rx=0 tx=0).stop 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 msgr2=0x7f7e5c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 0x7f7e5c106ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00e940 tx=0x7f7e4c00ec50 comp rx=0 tx=0).stop 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 shutdown_connections 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7e440b2880 0x7f7e440b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5c104d00 0x7f7e5c105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e5c103230 0x7f7e5c106ff0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 --2- 172.21.15.5:0/2880254778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e5c104360 0x7f7e5c1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 >> 172.21.15.5:0/2880254778 conn(0x7f7e5c0fe870 msgr2=0x7f7e5c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 shutdown_connections 2024-05-14T15:19:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:52.123+0000 7f7e60afe700 1 -- 172.21.15.5:0/2880254778 wait complete. 2024-05-14T15:19:52.725 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: purged_snaps scrub starts 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: purged_snaps scrub ok 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] boot 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:52.726 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:52 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2880254778' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: purged_snaps scrub starts 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: purged_snaps scrub ok 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] boot 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:52.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:52 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2880254778' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T15:19:52.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: purged_snaps scrub starts 2024-05-14T15:19:52.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: purged_snaps scrub ok 2024-05-14T15:19:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-05-14T15:19:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: osd.11 [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] boot 2024-05-14T15:19:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T15:19:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:19:52.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:52 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2880254778' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T15:19:52.836 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1715699991,"num_in_osds":12,"osd_in_since":1715699971,"num_remapped_pgs":0} 2024-05-14T15:19:52.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:19:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: Detected new or changed devices on smithi193 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi193 to 4272M 2024-05-14T15:19:54.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:54.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:54.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:53 smithi193 ceph-mon[36001]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:54.208 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T15:19:54.208 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: Detected new or changed devices on smithi193 2024-05-14T15:19:54.208 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi193 to 4272M 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:53 smithi005 ceph-mon[31559]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:54.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T15:19:54.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: Detected new or changed devices on smithi193 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi193 to 4272M 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:19:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:53 smithi144 ceph-mon[35650]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 -- 172.21.15.5:0/2336638955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c108f10 msgr2=0x7f3c2c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 --2- 172.21.15.5:0/2336638955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c108f10 0x7f3c2c109320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c180099c0 tx=0x7f3c18009cd0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 -- 172.21.15.5:0/2336638955 shutdown_connections 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 --2- 172.21.15.5:0/2336638955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c2c109860 0x7f3c2c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 --2- 172.21.15.5:0/2336638955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 --2- 172.21.15.5:0/2336638955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c108f10 0x7f3c2c109320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 -- 172.21.15.5:0/2336638955 >> 172.21.15.5:0/2336638955 conn(0x7f3c2c0ffef0 msgr2=0x7f3c2c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 -- 172.21.15.5:0/2336638955 shutdown_connections 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 -- 172.21.15.5:0/2336638955 wait complete. 2024-05-14T15:19:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.765+0000 7f3c331f7700 1 Processor -- start 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 -- start start 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c2c108f10 0x7f3c2c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c109860 0x7f3c2c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c114470 con 0x7f3c2c1045e0 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1142f0 con 0x7f3c2c109860 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c331f7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c1145f0 con 0x7f3c2c108f10 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57016/0 (socket says 172.21.15.5:57016) 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 -- 172.21.15.5:0/1911663154 learned_addr learned my addr 172.21.15.5:0/1911663154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c2bfff700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c2c108f10 0x7f3c2c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c31794700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c109860 0x7f3c2c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 -- 172.21.15.5:0/1911663154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c2c108f10 msgr2=0x7f3c2c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c2c108f10 0x7f3c2c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 -- 172.21.15.5:0/1911663154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c109860 msgr2=0x7f3c2c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c109860 0x7f3c2c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.766+0000 7f3c30f93700 1 -- 172.21.15.5:0/1911663154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c19fcd0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.767+0000 7f3c30f93700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c19ae00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e9a0 tx=0x7f3c2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.767+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000ca10 con 0x7f3c2c1045e0 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.767+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c19ffc0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.767+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2000cbb0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.767+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20018680 con 0x7f3c2c1045e0 2024-05-14T15:19:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.767+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c1a78e0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.768+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c0512c0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.771+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3c20010430 con 0x7f3c2c1045e0 2024-05-14T15:19:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.772+0000 7f3c29ffb700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3c1c0b2880 0x7f3c1c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.772+0000 7f3c2bfff700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3c1c0b2880 0x7f3c1c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.772+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f3c200e22a0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.772+0000 7f3c2bfff700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3c1c0b2880 0x7f3c1c0b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c1801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:54.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.775+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c200aa3c0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.996+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c2c02e5f0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.997+0000 7f3c29ffb700 1 -- 172.21.15.5:0/1911663154 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18412 (secure 0 0 0) 0x7f3c200a9cc0 con 0x7f3c2c1045e0 2024-05-14T15:19:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:19:54.997 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:19:52.932880+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3c1c0b2880 msgr2=0x7f3c1c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3c1c0b2880 0x7f3c1c0b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3c18000c00 tx=0x7f3c1801a040 comp rx=0 tx=0).stop 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 msgr2=0x7f3c2c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c19ae00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e9a0 tx=0x7f3c2000ecb0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 shutdown_connections 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3c1c0b2880 0x7f3c1c0b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c2c109860 0x7f3c2c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c2c1045e0 0x7f3c2c19ae00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 --2- 172.21.15.5:0/1911663154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c2c108f10 0x7f3c2c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 >> 172.21.15.5:0/1911663154 conn(0x7f3c2c0ffef0 msgr2=0x7f3c2c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 shutdown_connections 2024-05-14T15:19:55.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:54.999+0000 7f3c331f7700 1 -- 172.21.15.5:0/1911663154 wait complete. 2024-05-14T15:19:55.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:55 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1911663154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:19:55.480 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:55 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1911663154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:19:55.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:55 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1911663154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:19:55.908 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T15:16:05.463462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T15:19:55.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-14T15:19:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:19:56.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:56 smithi005 ceph-mon[31559]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:56.407 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:56 smithi193 ceph-mon[36001]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:56.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:56 smithi144 ceph-mon[35650]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 -- 172.21.15.5:0/699994289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36841045d0 msgr2=0x7f36841049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 --2- 172.21.15.5:0/699994289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36841045d0 0x7f36841049e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36800099c0 tx=0x7f3680009cd0 comp rx=0 tx=0).stop 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 -- 172.21.15.5:0/699994289 shutdown_connections 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 --2- 172.21.15.5:0/699994289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3684109850 0x7f36841111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 --2- 172.21.15.5:0/699994289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36841045d0 0x7f36841049e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 --2- 172.21.15.5:0/699994289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3684108f00 0x7f3684109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 -- 172.21.15.5:0/699994289 >> 172.21.15.5:0/699994289 conn(0x7f36840ffea0 msgr2=0x7f36841022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:57.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 -- 172.21.15.5:0/699994289 shutdown_connections 2024-05-14T15:19:57.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.871+0000 7f368b76c700 1 -- 172.21.15.5:0/699994289 wait complete. 2024-05-14T15:19:57.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 Processor -- start 2024-05-14T15:19:57.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 -- start start 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36841045d0 0x7f368419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 0x7f368419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3684109850 0x7f368419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36841144a0 con 0x7f3684108f00 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3684114320 con 0x7f3684109850 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f368b76c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3684114620 con 0x7f36841045d0 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f3688d07700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 0x7f368419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f3688d07700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 0x7f368419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55924/0 (socket says 172.21.15.5:55924) 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.872+0000 7f3688d07700 1 -- 172.21.15.5:0/1639211985 learned_addr learned my addr 172.21.15.5:0/1639211985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3689508700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36841045d0 0x7f368419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3689d09700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3684109850 0x7f368419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3688d07700 1 -- 172.21.15.5:0/1639211985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36841045d0 msgr2=0x7f368419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3688d07700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36841045d0 0x7f368419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3688d07700 1 -- 172.21.15.5:0/1639211985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3684109850 msgr2=0x7f368419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3688d07700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3684109850 0x7f368419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3688d07700 1 -- 172.21.15.5:0/1639211985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368419fd60 con 0x7f3684108f00 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f3688d07700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 0x7f368419b430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9f0 tx=0x7f367800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678004a30 con 0x7f3684108f00 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367800cc70 con 0x7f3684108f00 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36841a0050 con 0x7f3684108f00 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678005530 con 0x7f3684108f00 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.873+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36841a7970 con 0x7f3684108f00 2024-05-14T15:19:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.874+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36840512c0 con 0x7f3684108f00 2024-05-14T15:19:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.876+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3678004bd0 con 0x7f3684108f00 2024-05-14T15:19:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.877+0000 7f36767fc700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36700b2880 0x7f36700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:19:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.877+0000 7f3689508700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36700b2880 0x7f36700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:19:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.877+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f36780e21b0 con 0x7f3684108f00 2024-05-14T15:19:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.877+0000 7f3689508700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36700b2880 0x7f36700b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3680000c00 tx=0x7f368001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:19:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:57.880+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3678054df0 con 0x7f3684108f00 2024-05-14T15:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.093+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f368419c1f0 con 0x7f3684108f00 2024-05-14T15:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.093+0000 7f36767fc700 1 -- 172.21.15.5:0/1639211985 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f367800ce20 con 0x7f3684108f00 2024-05-14T15:19:58.094 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-14T15:19:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36700b2880 msgr2=0x7f36700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36700b2880 0x7f36700b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3680000c00 tx=0x7f368001a040 comp rx=0 tx=0).stop 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 msgr2=0x7f368419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 0x7f368419b430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9f0 tx=0x7f367800ed00 comp rx=0 tx=0).stop 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 shutdown_connections 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36700b2880 0x7f36700b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3684109850 0x7f368419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36841045d0 0x7f368419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 --2- 172.21.15.5:0/1639211985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3684108f00 0x7f368419b430 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 >> 172.21.15.5:0/1639211985 conn(0x7f36840ffea0 msgr2=0x7f3684102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 shutdown_connections 2024-05-14T15:19:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:19:58.096+0000 7f368b76c700 1 -- 172.21.15.5:0/1639211985 wait complete. 2024-05-14T15:19:58.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:58 smithi005 ceph-mon[31559]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:58.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:19:58 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1639211985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:19:58.784 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.b on smithi144 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.e on smithi144 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.h on smithi144 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.k on smithi144 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-05-14T15:19:58.785 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-05-14T15:19:58.785 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-05-14T15:19:58.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:58 smithi144 ceph-mon[35650]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:58.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:19:58 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1639211985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:19:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:58 smithi193 ceph-mon[36001]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:19:58.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:19:58 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1639211985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:19:59.034 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:20:00.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 -- 172.21.15.193:0/1559090372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104d20 msgr2=0x7fd2d010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:00.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1559090372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104d20 0x7fd2d010b5d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800b0a0 tx=0x7fd2c800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 -- 172.21.15.193:0/1559090372 shutdown_connections 2024-05-14T15:20:00.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1559090372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104d20 0x7fd2d010b5d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1559090372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d0103250 0x7fd2d0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1559090372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0104380 0x7fd2d0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 -- 172.21.15.193:0/1559090372 >> 172.21.15.193:0/1559090372 conn(0x7fd2d00fe890 msgr2=0x7fd2d0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 -- 172.21.15.193:0/1559090372 shutdown_connections 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.691+0000 7fd2d8707700 1 -- 172.21.15.193:0/1559090372 wait complete. 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 Processor -- start 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 -- start start 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0103250 0x7fd2d0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 0x7fd2d0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d0104d20 0x7fd2d01a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010ecd0 con 0x7fd2d0103250 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010eb50 con 0x7fd2d0104d20 2024-05-14T15:20:00.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d8707700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010ee50 con 0x7fd2d0104380 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 0x7fd2d0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 0x7fd2d0199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:44756/0 (socket says 172.21.15.193:44756) 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 -- 172.21.15.193:0/1287240880 learned_addr learned my addr 172.21.15.193:0/1287240880 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d64a3700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0103250 0x7fd2d0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d6ca4700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d0104d20 0x7fd2d01a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 -- 172.21.15.193:0/1287240880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d0104d20 msgr2=0x7fd2d01a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d0104d20 0x7fd2d01a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 -- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0103250 msgr2=0x7fd2d0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0103250 0x7fd2d0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.692+0000 7fd2d5ca2700 1 -- 172.21.15.193:0/1287240880 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d01a12e0 con 0x7fd2d0104380 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.693+0000 7fd2d5ca2700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 0x7fd2d0199cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0007910 tx=0x7fd2c0007c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:00.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.693+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0007e60 con 0x7fd2d0104380 2024-05-14T15:20:00.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.693+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c00044d0 con 0x7fd2d0104380 2024-05-14T15:20:00.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.693+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d0075a50 con 0x7fd2d0104380 2024-05-14T15:20:00.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.693+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0005350 con 0x7fd2d0104380 2024-05-14T15:20:00.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.693+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d0075f90 con 0x7fd2d0104380 2024-05-14T15:20:00.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.694+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d00512c0 con 0x7fd2d0104380 2024-05-14T15:20:00.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.695+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd2c001f020 con 0x7fd2d0104380 2024-05-14T15:20:00.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.696+0000 7fd2c77fe700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd2bc0b2880 0x7fd2bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:00.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.696+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd2c00e3800 con 0x7fd2d0104380 2024-05-14T15:20:00.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.696+0000 7fd2d64a3700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd2bc0b2880 0x7fd2bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:00.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.697+0000 7fd2d64a3700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd2bc0b2880 0x7fd2bc0b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0104280 tx=0x7fd2cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:00.697 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:00 smithi193 ceph-mon[36001]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:00.698 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:00 smithi193 ceph-mon[36001]: overall HEALTH_OK 2024-05-14T15:20:00.700 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.700+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2c0026030 con 0x7fd2d0104380 2024-05-14T15:20:00.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:00 smithi144 ceph-mon[35650]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:00.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:00 smithi144 ceph-mon[35650]: overall HEALTH_OK 2024-05-14T15:20:00.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:00 smithi005 ceph-mon[31559]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:00.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:00 smithi005 ceph-mon[31559]: overall HEALTH_OK 2024-05-14T15:20:00.912 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.912+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7fd2d00761f0 con 0x7fd2bc0b2880 2024-05-14T15:20:00.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.922+0000 7fd2c77fe700 1 -- 172.21.15.193:0/1287240880 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd2d00761f0 con 0x7fd2bc0b2880 2024-05-14T15:20:00.921 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd2bc0b2880 msgr2=0x7fd2bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd2bc0b2880 0x7fd2bc0b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0104280 tx=0x7fd2cc009260 comp rx=0 tx=0).stop 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 msgr2=0x7fd2d0199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 0x7fd2d0199cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0007910 tx=0x7fd2c0007c20 comp rx=0 tx=0).stop 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 shutdown_connections 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd2bc0b2880 0x7fd2bc0b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d0104d20 0x7fd2d01a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0103250 0x7fd2d0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 --2- 172.21.15.193:0/1287240880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2d0104380 0x7fd2d0199cc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:00.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 >> 172.21.15.193:0/1287240880 conn(0x7fd2d00fe890 msgr2=0x7fd2d01075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:00.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 shutdown_connections 2024-05-14T15:20:00.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-05-14T15:20:00.924+0000 7fd2d8707700 1 -- 172.21.15.193:0/1287240880 wait complete. 2024-05-14T15:20:01.624 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.a.service 2024-05-14T15:20:01.628 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.d.service 2024-05-14T15:20:01.672 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.g.service 2024-05-14T15:20:01.715 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.j.service 2024-05-14T15:20:01.759 DEBUG:teuthology.orchestra.run.smithi144:mds.b> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.b.service 2024-05-14T15:20:01.762 DEBUG:teuthology.orchestra.run.smithi144:mds.e> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.e.service 2024-05-14T15:20:01.804 DEBUG:teuthology.orchestra.run.smithi144:mds.h> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.h.service 2024-05-14T15:20:01.847 DEBUG:teuthology.orchestra.run.smithi144:mds.k> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.k.service 2024-05-14T15:20:01.890 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.c.service 2024-05-14T15:20:01.893 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.f.service 2024-05-14T15:20:01.935 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.i.service 2024-05-14T15:20:01.979 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.l.service 2024-05-14T15:20:02.021 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-14T15:20:02.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:20:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T15:20:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:02.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:01 smithi144 ceph-mon[35650]: Deploying daemon mds.l on smithi193 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:02.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:02.172 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:01 smithi193 ceph-mon[36001]: Deploying daemon mds.l on smithi193 2024-05-14T15:20:02.269 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:20:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T15:20:02.314 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:01 smithi005 ceph-mon[31559]: Deploying daemon mds.l on smithi193 2024-05-14T15:20:03.802 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:03 smithi193 systemd[1]: Starting Ceph mds.l for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 -- 172.21.15.5:0/2422847100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 msgr2=0x7fbedc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 --2- 172.21.15.5:0/2422847100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc1038d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0099c0 tx=0x7fbecc009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 -- 172.21.15.5:0/2422847100 shutdown_connections 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 --2- 172.21.15.5:0/2422847100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 0x7fbedc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 --2- 172.21.15.5:0/2422847100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc1038d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 --2- 172.21.15.5:0/2422847100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 0x7fbedc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 -- 172.21.15.5:0/2422847100 >> 172.21.15.5:0/2422847100 conn(0x7fbedc0fedf0 msgr2=0x7fbedc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 -- 172.21.15.5:0/2422847100 shutdown_connections 2024-05-14T15:20:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.998+0000 7fbee1085700 1 -- 172.21.15.5:0/2422847100 wait complete. 2024-05-14T15:20:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:03.999+0000 7fbee1085700 1 Processor -- start 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 -- start start 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 0x7fbedc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 0x7fbedc19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1144c0 con 0x7fbedc1034c0 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc114340 con 0x7fbedc107df0 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbee1085700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc114640 con 0x7fbedc108740 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbedad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbedad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55938/0 (socket says 172.21.15.5:55938) 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbedad9d700 1 -- 172.21.15.5:0/3344787440 learned_addr learned my addr 172.21.15.5:0/3344787440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:04.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbeda59c700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 0x7fbedc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.000+0000 7fbedb59e700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 0x7fbedc19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedad9d700 1 -- 172.21.15.5:0/3344787440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 msgr2=0x7fbedc19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedad9d700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 0x7fbedc19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedad9d700 1 -- 172.21.15.5:0/3344787440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 msgr2=0x7fbedc19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedad9d700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 0x7fbedc19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedad9d700 1 -- 172.21.15.5:0/3344787440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbedc19fbf0 con 0x7fbedc1034c0 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedb59e700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 0x7fbedc19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbeda59c700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 0x7fbedc19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbedad9d700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc19add0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0098e0 tx=0x7fbecc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc027070 con 0x7fbedc1034c0 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbedc19fe80 con 0x7fbedc1034c0 2024-05-14T15:20:04.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbecc003f20 con 0x7fbedc1034c0 2024-05-14T15:20:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc022ca0 con 0x7fbedc1034c0 2024-05-14T15:20:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.001+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbedc1a03b0 con 0x7fbedc1034c0 2024-05-14T15:20:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.002+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbedc0512c0 con 0x7fbedc1034c0 2024-05-14T15:20:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.004+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbecc02a070 con 0x7fbedc1034c0 2024-05-14T15:20:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.004+0000 7fbec3fff700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbec40b2880 0x7fbec40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.005+0000 7fbeda59c700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbec40b2880 0x7fbec40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.005+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbecc0e2b90 con 0x7fbedc1034c0 2024-05-14T15:20:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.005+0000 7fbeda59c700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbec40b2880 0x7fbec40b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbedc102400 tx=0x7fbed000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.010+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbecc013070 con 0x7fbedc1034c0 2024-05-14T15:20:04.150 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:03 smithi193 podman[65585]: 2024-05-14 15:20:03.795550579 +0000 UTC m=+0.017187650 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:20:04.150 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:03 smithi193 podman[65585]: 2024-05-14 15:20:03.922995731 +0000 UTC m=+0.144632821 container create c93296aea431feea17dd75d6ed279514687dc5f18baa45bf418cae022b1f3b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T15:20:04.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.278+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbedc19c130 con 0x7fbedc1034c0 2024-05-14T15:20:04.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.279+0000 7fbec3fff700 1 -- 172.21.15.5:0/3344787440 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fbecc0aade0 con 0x7fbedc1034c0 2024-05-14T15:20:04.280 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:04.280 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbec40b2880 msgr2=0x7fbec40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbec40b2880 0x7fbec40b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbedc102400 tx=0x7fbed000a380 comp rx=0 tx=0).stop 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 msgr2=0x7fbedc19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc19add0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0098e0 tx=0x7fbecc0057c0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 shutdown_connections 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbec40b2880 0x7fbec40b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbedc108740 0x7fbedc19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbedc1034c0 0x7fbedc19add0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 --2- 172.21.15.5:0/3344787440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbedc107df0 0x7fbedc19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:04.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 >> 172.21.15.5:0/3344787440 conn(0x7fbedc0fedf0 msgr2=0x7fbedc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:04.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 shutdown_connections 2024-05-14T15:20:04.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:04.281+0000 7fbee1085700 1 -- 172.21.15.5:0/3344787440 wait complete. 2024-05-14T15:20:04.283 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-05-14T15:20:04.502 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:04 smithi193 podman[65585]: 2024-05-14 15:20:04.151030891 +0000 UTC m=+0.372667947 container init c93296aea431feea17dd75d6ed279514687dc5f18baa45bf418cae022b1f3b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-14T15:20:04.502 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:04 smithi193 podman[65585]: 2024-05-14 15:20:04.153983777 +0000 UTC m=+0.375620842 container start c93296aea431feea17dd75d6ed279514687dc5f18baa45bf418cae022b1f3b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:20:04.502 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mds[65604]: -- 172.21.15.193:0/3655586786 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fc2b95a40 con 0x563fc2bdda80 2024-05-14T15:20:04.502 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:04 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l[65600]: starting mds.l at 2024-05-14T15:20:04.502 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:04 smithi193 bash[65585]: c93296aea431feea17dd75d6ed279514687dc5f18baa45bf418cae022b1f3b81 2024-05-14T15:20:04.502 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:20:04 smithi193 systemd[1]: Started Ceph mds.l for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:04.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6832/2582549673,v1:172.21.15.193:6833/2582549673] up:boot 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: fsmap 1 up:standby 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3344787440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:04.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:04 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:04.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:04.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6832/2582549673,v1:172.21.15.193:6833/2582549673] up:boot 2024-05-14T15:20:04.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: fsmap 1 up:standby 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3344787440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:04.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:04 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:04.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:04.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6832/2582549673,v1:172.21.15.193:6833/2582549673] up:boot 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: fsmap 1 up:standby 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3344787440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:04.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:04 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:05 smithi144 ceph-mon[35650]: Deploying daemon mds.k on smithi144 2024-05-14T15:20:05.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:05 smithi193 ceph-mon[36001]: Deploying daemon mds.k on smithi144 2024-05-14T15:20:05.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:05 smithi005 ceph-mon[31559]: Deploying daemon mds.k on smithi144 2024-05-14T15:20:06.709 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:06 smithi144 ceph-mon[35650]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:06 smithi193 ceph-mon[36001]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:06 smithi005 ceph-mon[31559]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:07.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:07.275 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:07.296 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:20:07 smithi144 systemd[1]: Started Ceph mds.k for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: Deploying daemon mds.j on smithi005 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6832/3788502407,v1:172.21.15.144:6833/3788502407] up:boot 2024-05-14T15:20:08.423 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: fsmap 2 up:standby 2024-05-14T15:20:08.424 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:08 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T15:20:08.437 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: Deploying daemon mds.j on smithi005 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6832/3788502407,v1:172.21.15.144:6833/3788502407] up:boot 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: fsmap 2 up:standby 2024-05-14T15:20:08.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:08 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T15:20:08.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: Deploying daemon mds.j on smithi005 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6832/3788502407,v1:172.21.15.144:6833/3788502407] up:boot 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: fsmap 2 up:standby 2024-05-14T15:20:08.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:08 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T15:20:10.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 -- 172.21.15.5:0/3225767912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 msgr2=0x7fbb941038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:10.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 --2- 172.21.15.5:0/3225767912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb941038d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbb840099c0 tx=0x7fbb84009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 -- 172.21.15.5:0/3225767912 shutdown_connections 2024-05-14T15:20:10.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 --2- 172.21.15.5:0/3225767912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb94108740 0x7fbb941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 --2- 172.21.15.5:0/3225767912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb941038d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 --2- 172.21.15.5:0/3225767912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb94107df0 0x7fbb94108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 -- 172.21.15.5:0/3225767912 >> 172.21.15.5:0/3225767912 conn(0x7fbb940fedc0 msgr2=0x7fbb941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 -- 172.21.15.5:0/3225767912 shutdown_connections 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.168+0000 7fbb993d5700 1 -- 172.21.15.5:0/3225767912 wait complete. 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 Processor -- start 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 -- start start 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb9419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb94107df0 0x7fbb9419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb94108740 0x7fbb9419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941144c0 con 0x7fbb941034c0 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.169+0000 7fbb993d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114340 con 0x7fbb94107df0 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb993d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114640 con 0x7fbb94108740 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb9419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:10.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb9419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37530/0 (socket says 172.21.15.5:37530) 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 -- 172.21.15.5:0/967953144 learned_addr learned my addr 172.21.15.5:0/967953144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb8a7fc700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb94107df0 0x7fbb9419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb937fe700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb94108740 0x7fbb9419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 -- 172.21.15.5:0/967953144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb94108740 msgr2=0x7fbb9419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb94108740 0x7fbb9419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 -- 172.21.15.5:0/967953144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb94107df0 msgr2=0x7fbb9419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb94107df0 0x7fbb9419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 -- 172.21.15.5:0/967953144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9419fca0 con 0x7fbb941034c0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb92ffd700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb9419add0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400b580 tx=0x7fbb840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.170+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84027070 con 0x7fbb941034c0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.171+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb84003f20 con 0x7fbb941034c0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.171+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9419ff30 con 0x7fbb941034c0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.171+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84022d10 con 0x7fbb941034c0 2024-05-14T15:20:10.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.171+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a0410 con 0x7fbb941034c0 2024-05-14T15:20:10.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.172+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb940512c0 con 0x7fbb941034c0 2024-05-14T15:20:10.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.173+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbb84022470 con 0x7fbb941034c0 2024-05-14T15:20:10.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.174+0000 7fbb90ff9700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbb800b2880 0x7fbb800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:10.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.174+0000 7fbb8a7fc700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbb800b2880 0x7fbb800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:10.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.174+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbb840e2d90 con 0x7fbb941034c0 2024-05-14T15:20:10.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.175+0000 7fbb8a7fc700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbb800b2880 0x7fbb800b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb941023d0 tx=0x7fbb7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:10.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.178+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb84013070 con 0x7fbb941034c0 2024-05-14T15:20:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.468+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb9402e5f0 con 0x7fbb941034c0 2024-05-14T15:20:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.469+0000 7fbb90ff9700 1 -- 172.21.15.5:0/967953144 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fbb840afe20 con 0x7fbb941034c0 2024-05-14T15:20:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:10.469 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbb800b2880 msgr2=0x7fbb800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbb800b2880 0x7fbb800b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb941023d0 tx=0x7fbb7c00b410 comp rx=0 tx=0).stop 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 msgr2=0x7fbb9419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb9419add0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400b580 tx=0x7fbb840047b0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 shutdown_connections 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbb800b2880 0x7fbb800b4d90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb94108740 0x7fbb9419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb941034c0 0x7fbb9419add0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 --2- 172.21.15.5:0/967953144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb94107df0 0x7fbb9419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 >> 172.21.15.5:0/967953144 conn(0x7fbb940fedc0 msgr2=0x7fbb941096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 shutdown_connections 2024-05-14T15:20:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:10.471+0000 7fbb993d5700 1 -- 172.21.15.5:0/967953144 wait complete. 2024-05-14T15:20:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-05-14T15:20:10.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:10 smithi144 ceph-mon[35650]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:10.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:10 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/967953144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:10.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:10 smithi193 ceph-mon[36001]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:10.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:10 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/967953144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:10.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:10 smithi005 ceph-mon[31559]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:10.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:10 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/967953144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:11.378 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:20:11 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-j[72254]: starting mds.j at 2024-05-14T15:20:11.378 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:20:11 smithi005 bash[72224]: e865c0ba747275dbe5bfd2c65bd16da49c22d418cf5e29c2c36b2c6252b5a2c6 2024-05-14T15:20:11.814 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:20:11 smithi005 systemd[1]: Started Ceph mds.j for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: Deploying daemon mds.d on smithi005 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.5:6834/1133415705,v1:172.21.15.5:6835/1133415705] up:boot 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: fsmap 3 up:standby 2024-05-14T15:20:12.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:12 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T15:20:12.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: Deploying daemon mds.d on smithi005 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.5:6834/1133415705,v1:172.21.15.5:6835/1133415705] up:boot 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: fsmap 3 up:standby 2024-05-14T15:20:12.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:12 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: Deploying daemon mds.d on smithi005 2024-05-14T15:20:12.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.5:6834/1133415705,v1:172.21.15.5:6835/1133415705] up:boot 2024-05-14T15:20:12.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: fsmap 3 up:standby 2024-05-14T15:20:12.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:12 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T15:20:13.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:14.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:14 smithi005 ceph-mon[31559]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:14 smithi144 ceph-mon[35650]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:14.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:14 smithi193 ceph-mon[36001]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 -- 172.21.15.5:0/1828529731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 msgr2=0x7fd2101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 --2- 172.21.15.5:0/1828529731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd2101100a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd20800b0d0 tx=0x7fd20800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 -- 172.21.15.5:0/1828529731 shutdown_connections 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 --2- 172.21.15.5:0/1828529731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd2101100a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 --2- 172.21.15.5:0/1828529731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2101034c0 0x7fd2101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 --2- 172.21.15.5:0/1828529731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd210107df0 0x7fd210108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 -- 172.21.15.5:0/1828529731 >> 172.21.15.5:0/1828529731 conn(0x7fd2100fedc0 msgr2=0x7fd2101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 -- 172.21.15.5:0/1828529731 shutdown_connections 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.293+0000 7fd214834700 1 -- 172.21.15.5:0/1828529731 wait complete. 2024-05-14T15:20:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.294+0000 7fd214834700 1 Processor -- start 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 -- start start 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2101034c0 0x7fd21019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd210107df0 0x7fd21019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd21019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd210114510 con 0x7fd210108740 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd210114390 con 0x7fd210107df0 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd214834700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd210114690 con 0x7fd2101034c0 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd20ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd21019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd210107df0 0x7fd21019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd20ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd21019f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37550/0 (socket says 172.21.15.5:37550) 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd20ed9d700 1 -- 172.21.15.5:0/3774043237 learned_addr learned my addr 172.21.15.5:0/3774043237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.295+0000 7fd20e59c700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2101034c0 0x7fd21019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd20ed9d700 1 -- 172.21.15.5:0/3774043237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2101034c0 msgr2=0x7fd21019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd20ed9d700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2101034c0 0x7fd21019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd20ed9d700 1 -- 172.21.15.5:0/3774043237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd210107df0 msgr2=0x7fd21019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd20ed9d700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd210107df0 0x7fd21019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd20ed9d700 1 -- 172.21.15.5:0/3774043237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21019fb40 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd20ed9d700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd21019f550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2080079a0 tx=0x7fd2080079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20801f070 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21019fdd0 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd208005070 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20801b680 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.296+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2101a02b0 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.297+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2100512c0 con 0x7fd210108740 2024-05-14T15:20:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.299+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd208022070 con 0x7fd210108740 2024-05-14T15:20:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.299+0000 7fd207fff700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1fc0b2880 0x7fd1fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.300+0000 7fd20e59c700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1fc0b2880 0x7fd1fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.300+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd20800f020 con 0x7fd210108740 2024-05-14T15:20:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.300+0000 7fd20e59c700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1fc0b2880 0x7fd1fc0b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2000072d0 tx=0x7fd200010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:15.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.303+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd208029020 con 0x7fd210108740 2024-05-14T15:20:15.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.571+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd21002e5f0 con 0x7fd210108740 2024-05-14T15:20:15.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.571+0000 7fd207fff700 1 -- 172.21.15.5:0/3774043237 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fd2080ab5f0 con 0x7fd210108740 2024-05-14T15:20:15.572 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:15.572 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1fc0b2880 msgr2=0x7fd1fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1fc0b2880 0x7fd1fc0b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2000072d0 tx=0x7fd200010040 comp rx=0 tx=0).stop 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 msgr2=0x7fd21019f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd21019f550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2080079a0 tx=0x7fd2080079d0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 shutdown_connections 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1fc0b2880 0x7fd1fc0b4d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd210108740 0x7fd21019f550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2101034c0 0x7fd21019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 --2- 172.21.15.5:0/3774043237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd210107df0 0x7fd21019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 >> 172.21.15.5:0/3774043237 conn(0x7fd2100fedc0 msgr2=0x7fd210107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 shutdown_connections 2024-05-14T15:20:15.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:15.574+0000 7fd214834700 1 -- 172.21.15.5:0/3774043237 wait complete. 2024-05-14T15:20:15.575 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-05-14T15:20:16.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:16.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3774043237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:16.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:16 smithi005 ceph-mon[31559]: Deploying daemon mds.e on smithi144 2024-05-14T15:20:16.713 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:16.713 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3774043237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:16.713 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.713 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.713 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.714 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:16.714 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:16.714 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:16.714 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:16 smithi144 ceph-mon[35650]: Deploying daemon mds.e on smithi144 2024-05-14T15:20:16.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:16.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3774043237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:16.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:16 smithi193 ceph-mon[36001]: Deploying daemon mds.e on smithi144 2024-05-14T15:20:17.787 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:17 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.5:6836/3415932983,v1:172.21.15.5:6837/3415932983] up:boot 2024-05-14T15:20:17.788 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:17 smithi144 ceph-mon[35650]: fsmap 4 up:standby 2024-05-14T15:20:17.788 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:17 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T15:20:17.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:17 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.5:6836/3415932983,v1:172.21.15.5:6837/3415932983] up:boot 2024-05-14T15:20:17.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:17 smithi193 ceph-mon[36001]: fsmap 4 up:standby 2024-05-14T15:20:17.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:17 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T15:20:17.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:17 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.5:6836/3415932983,v1:172.21.15.5:6837/3415932983] up:boot 2024-05-14T15:20:17.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:17 smithi005 ceph-mon[31559]: fsmap 4 up:standby 2024-05-14T15:20:17.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:17 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T15:20:18.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:18.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:18 smithi005 ceph-mon[31559]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:18.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:18 smithi144 ceph-mon[35650]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:18.796 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:20:18 smithi144 systemd[1]: Started Ceph mds.e for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:18.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:18 smithi193 ceph-mon[36001]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:19.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: Deploying daemon mds.f on smithi193 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6834/3352720235,v1:172.21.15.144:6835/3352720235] up:boot 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: fsmap 5 up:standby 2024-05-14T15:20:19.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:19 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:20:20.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:20.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: Deploying daemon mds.f on smithi193 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6834/3352720235,v1:172.21.15.144:6835/3352720235] up:boot 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: fsmap 5 up:standby 2024-05-14T15:20:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:19 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:20:20.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:20.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:20.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:20.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: Deploying daemon mds.f on smithi193 2024-05-14T15:20:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6834/3352720235,v1:172.21.15.144:6835/3352720235] up:boot 2024-05-14T15:20:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: fsmap 5 up:standby 2024-05-14T15:20:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:19 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 -- 172.21.15.5:0/922527309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc108f30 msgr2=0x7feabc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 --2- 172.21.15.5:0/922527309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc108f30 0x7feabc109340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feaa40099c0 tx=0x7feaa4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 -- 172.21.15.5:0/922527309 shutdown_connections 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 --2- 172.21.15.5:0/922527309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc109880 0x7feabc1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 --2- 172.21.15.5:0/922527309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc104650 0x7feabc104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 --2- 172.21.15.5:0/922527309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc108f30 0x7feabc109340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 -- 172.21.15.5:0/922527309 >> 172.21.15.5:0/922527309 conn(0x7feabc079f00 msgr2=0x7feabc07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:20.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 -- 172.21.15.5:0/922527309 shutdown_connections 2024-05-14T15:20:20.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.330+0000 7feac1aae700 1 -- 172.21.15.5:0/922527309 wait complete. 2024-05-14T15:20:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 Processor -- start 2024-05-14T15:20:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 -- start start 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc104650 0x7feabc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc108f30 0x7feabc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 0x7feabc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1144e0 con 0x7feabc109880 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feabc114360 con 0x7feabc104650 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.331+0000 7feac1aae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feabc114660 con 0x7feabc108f30 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 0x7feabc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 0x7feabc19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58380/0 (socket says 172.21.15.5:58380) 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 -- 172.21.15.5:0/2087756695 learned_addr learned my addr 172.21.15.5:0/2087756695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabb7fe700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc104650 0x7feabc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feab3fff700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc108f30 0x7feabc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 -- 172.21.15.5:0/2087756695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc108f30 msgr2=0x7feabc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc108f30 0x7feabc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 -- 172.21.15.5:0/2087756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc104650 msgr2=0x7feabc19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc104650 0x7feabc19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 -- 172.21.15.5:0/2087756695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc19fd60 con 0x7feabc109880 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feab3fff700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc108f30 0x7feabc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabb7fe700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc104650 0x7feabc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feabbfff700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 0x7feabc19f6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feab4007ee0 tx=0x7feab400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400ddd0 con 0x7feabc109880 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1a0050 con 0x7feabc109880 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab4004020 con 0x7feabc109880 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4015660 con 0x7feabc109880 2024-05-14T15:20:20.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.332+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a7970 con 0x7feabc109880 2024-05-14T15:20:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.333+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc0512c0 con 0x7feabc109880 2024-05-14T15:20:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.335+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7feab40044f0 con 0x7feabc109880 2024-05-14T15:20:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.336+0000 7feab97fa700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feaa80b2880 0x7feaa80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.336+0000 7feabb7fe700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feaa80b2880 0x7feaa80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.336+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7feab40e2120 con 0x7feabc109880 2024-05-14T15:20:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.336+0000 7feabb7fe700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feaa80b2880 0x7feaa80b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaac005950 tx=0x7feaac005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.339+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feab4010070 con 0x7feabc109880 2024-05-14T15:20:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.604+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feabc19c1f0 con 0x7feabc109880 2024-05-14T15:20:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.604+0000 7feab97fa700 1 -- 172.21.15.5:0/2087756695 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7feab40aa240 con 0x7feabc109880 2024-05-14T15:20:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:20.604 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T15:20:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.606+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feaa80b2880 msgr2=0x7feaa80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.606+0000 7feac1aae700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feaa80b2880 0x7feaa80b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaac005950 tx=0x7feaac005f50 comp rx=0 tx=0).stop 2024-05-14T15:20:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.606+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 msgr2=0x7feabc19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 0x7feabc19f6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feab4007ee0 tx=0x7feab400d3b0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 shutdown_connections 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7feaa80b2880 0x7feaa80b4d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feabc109880 0x7feabc19f6c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feabc104650 0x7feabc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 --2- 172.21.15.5:0/2087756695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feabc108f30 0x7feabc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 >> 172.21.15.5:0/2087756695 conn(0x7feabc079f00 msgr2=0x7feabc102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 shutdown_connections 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:20.607+0000 7feac1aae700 1 -- 172.21.15.5:0/2087756695 wait complete. 2024-05-14T15:20:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-05-14T15:20:20.874 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:20 smithi005 ceph-mon[31559]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:20.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:20 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2087756695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:20 smithi144 ceph-mon[35650]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:20 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2087756695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:20 smithi193 ceph-mon[36001]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:20 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2087756695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:21.053 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:20 smithi193 systemd[1]: Starting Ceph mds.f for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:20:21.460 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 podman[65890]: 2024-05-14 15:20:21.08885623 +0000 UTC m=+0.017016598 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:20:21.460 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 podman[65890]: 2024-05-14 15:20:21.210917016 +0000 UTC m=+0.139077420 container create 2edd9b4dfe167bf08a7fbc6b45722f256f74a8c061a0da408c0a1eb083ad03f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-14T15:20:21.461 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 podman[65890]: 2024-05-14 15:20:21.461375188 +0000 UTC m=+0.389535543 container init 2edd9b4dfe167bf08a7fbc6b45722f256f74a8c061a0da408c0a1eb083ad03f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-14T15:20:21.803 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 podman[65890]: 2024-05-14 15:20:21.463970943 +0000 UTC m=+0.392131298 container start 2edd9b4dfe167bf08a7fbc6b45722f256f74a8c061a0da408c0a1eb083ad03f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-14T15:20:21.803 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 ceph-mds[65909]: -- 172.21.15.193:0/4015554707 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634bc901a40 con 0x5634bc949a80 2024-05-14T15:20:21.803 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f[65905]: starting mds.f at 2024-05-14T15:20:21.803 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 bash[65890]: 2edd9b4dfe167bf08a7fbc6b45722f256f74a8c061a0da408c0a1eb083ad03f4 2024-05-14T15:20:21.803 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:20:21 smithi193 systemd[1]: Started Ceph mds.f for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: Deploying daemon mds.c on smithi193 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6834/4153907810,v1:172.21.15.193:6835/4153907810] up:boot 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: fsmap 6 up:standby 2024-05-14T15:20:22.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:22 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:20:23.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:23.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:23.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:23.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: Deploying daemon mds.c on smithi193 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6834/4153907810,v1:172.21.15.193:6835/4153907810] up:boot 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: fsmap 6 up:standby 2024-05-14T15:20:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:22 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:20:23.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:23.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:23.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:23.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:23.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:23.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:23.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:23.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: Deploying daemon mds.c on smithi193 2024-05-14T15:20:23.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6834/4153907810,v1:172.21.15.193:6835/4153907810] up:boot 2024-05-14T15:20:23.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: fsmap 6 up:standby 2024-05-14T15:20:23.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:22 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:20:23.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:23.991 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:23 smithi193 systemd[1]: Starting Ceph mds.c for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:20:23.991 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:23 smithi193 podman[66191]: 2024-05-14 15:20:23.894229795 +0000 UTC m=+0.018093417 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:20:24.269 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 podman[66191]: 2024-05-14 15:20:24.026102707 +0000 UTC m=+0.149966315 container create 9e98e3d9fb013e6aa0e42f2a6bbbd5243e8b66051e3ffca559bc914e3ae06e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-14T15:20:24.552 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 podman[66191]: 2024-05-14 15:20:24.270428515 +0000 UTC m=+0.394292124 container init 9e98e3d9fb013e6aa0e42f2a6bbbd5243e8b66051e3ffca559bc914e3ae06e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:20:24.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 podman[66191]: 2024-05-14 15:20:24.273073877 +0000 UTC m=+0.396937495 container start 9e98e3d9fb013e6aa0e42f2a6bbbd5243e8b66051e3ffca559bc914e3ae06e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:20:24.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mds[66210]: -- 172.21.15.193:0/1904191281 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e7ba0fa40 con 0x555e7ba57180 2024-05-14T15:20:24.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c[66206]: starting mds.c at 2024-05-14T15:20:24.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 bash[66191]: 9e98e3d9fb013e6aa0e42f2a6bbbd5243e8b66051e3ffca559bc914e3ae06e99 2024-05-14T15:20:24.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:20:24 smithi193 systemd[1]: Started Ceph mds.c for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:24.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:24 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:24.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:24.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:24.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:24.859 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:24.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:24.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:24.860 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:24 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:25.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.014+0000 7f79aae5a700 1 -- 172.21.15.5:0/2590049448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 msgr2=0x7f79a41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.014+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2590049448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a41100a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b0d0 tx=0x7f799c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.014+0000 7f79aae5a700 1 -- 172.21.15.5:0/2590049448 shutdown_connections 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2590049448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a41100a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2590049448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a41034c0 0x7f79a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2590049448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a4107df0 0x7f79a4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 -- 172.21.15.5:0/2590049448 >> 172.21.15.5:0/2590049448 conn(0x7f79a40fedc0 msgr2=0x7f79a41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 -- 172.21.15.5:0/2590049448 shutdown_connections 2024-05-14T15:20:25.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 -- 172.21.15.5:0/2590049448 wait complete. 2024-05-14T15:20:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.015+0000 7f79aae5a700 1 Processor -- start 2024-05-14T15:20:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 -- start start 2024-05-14T15:20:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a41034c0 0x7f79a419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:25.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a4107df0 0x7f79a419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41144c0 con 0x7f79a4108740 2024-05-14T15:20:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4114340 con 0x7f79a41034c0 2024-05-14T15:20:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79aae5a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4114640 con 0x7f79a4107df0 2024-05-14T15:20:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:25.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58398/0 (socket says 172.21.15.5:58398) 2024-05-14T15:20:25.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 -- 172.21.15.5:0/2460795753 learned_addr learned my addr 172.21.15.5:0/2460795753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:25.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a8bf6700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a41034c0 0x7f79a419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:25.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a3fff700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a4107df0 0x7f79a419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:25.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 -- 172.21.15.5:0/2460795753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a4107df0 msgr2=0x7f79a419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:25.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a4107df0 0x7f79a419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 -- 172.21.15.5:0/2460795753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a41034c0 msgr2=0x7f79a419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:25.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a41034c0 0x7f79a419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.016+0000 7f79a93f7700 1 -- 172.21.15.5:0/2460795753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a419fc60 con 0x7f79a4108740 2024-05-14T15:20:25.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.017+0000 7f79a93f7700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a419f5c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f799c0095c0 tx=0x7f799c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:25.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.017+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c01f070 con 0x7f79a4108740 2024-05-14T15:20:25.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.017+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c007e20 con 0x7f79a4108740 2024-05-14T15:20:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.017+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a419fef0 con 0x7f79a4108740 2024-05-14T15:20:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.017+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c01b4a0 con 0x7f79a4108740 2024-05-14T15:20:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.017+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a03d0 con 0x7f79a4108740 2024-05-14T15:20:25.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.018+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a40512c0 con 0x7f79a4108740 2024-05-14T15:20:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.020+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f799c022070 con 0x7f79a4108740 2024-05-14T15:20:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.020+0000 7f79a1ffb700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f79940b2880 0x7f79940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.020+0000 7f79a8bf6700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f79940b2880 0x7f79940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.020+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f799c00f020 con 0x7f79a4108740 2024-05-14T15:20:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.021+0000 7f79a8bf6700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f79940b2880 0x7f79940b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7998005950 tx=0x7f799800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.024+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f799c029020 con 0x7f79a4108740 2024-05-14T15:20:25.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:25.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:25.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:24 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.288+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79a402e5f0 con 0x7f79a4108740 2024-05-14T15:20:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.289+0000 7f79a1ffb700 1 -- 172.21.15.5:0/2460795753 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f799c0ab590 con 0x7f79a4108740 2024-05-14T15:20:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:25.289 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T15:20:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f79940b2880 msgr2=0x7f79940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f79940b2880 0x7f79940b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7998005950 tx=0x7f799800b410 comp rx=0 tx=0).stop 2024-05-14T15:20:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 msgr2=0x7f79a419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a419f5c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f799c0095c0 tx=0x7f799c007a20 comp rx=0 tx=0).stop 2024-05-14T15:20:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 shutdown_connections 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f79940b2880 0x7f79940b4d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a4108740 0x7f79a419f5c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79a41034c0 0x7f79a419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 --2- 172.21.15.5:0/2460795753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a4107df0 0x7f79a419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 >> 172.21.15.5:0/2460795753 conn(0x7f79a40fedc0 msgr2=0x7f79a4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 shutdown_connections 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:25.291+0000 7f79aae5a700 1 -- 172.21.15.5:0/2460795753 wait complete. 2024-05-14T15:20:25.292 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-05-14T15:20:25.708 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:25 smithi193 ceph-mon[36001]: Deploying daemon mds.h on smithi144 2024-05-14T15:20:25.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:25 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6836/1910760961,v1:172.21.15.193:6837/1910760961] up:boot 2024-05-14T15:20:25.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:25 smithi193 ceph-mon[36001]: fsmap 7 up:standby 2024-05-14T15:20:25.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:25 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:20:25.709 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2460795753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:25.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:25 smithi144 ceph-mon[35650]: Deploying daemon mds.h on smithi144 2024-05-14T15:20:25.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:25 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6836/1910760961,v1:172.21.15.193:6837/1910760961] up:boot 2024-05-14T15:20:25.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:25 smithi144 ceph-mon[35650]: fsmap 7 up:standby 2024-05-14T15:20:25.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:25 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:20:25.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2460795753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:25.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:25 smithi005 ceph-mon[31559]: Deploying daemon mds.h on smithi144 2024-05-14T15:20:25.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:25 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6836/1910760961,v1:172.21.15.193:6837/1910760961] up:boot 2024-05-14T15:20:25.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:25 smithi005 ceph-mon[31559]: fsmap 7 up:standby 2024-05-14T15:20:25.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:25 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:20:25.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2460795753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:26.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:26 smithi144 ceph-mon[35650]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:27.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:26 smithi193 ceph-mon[36001]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:27.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:26 smithi005 ceph-mon[31559]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:27.099 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:20:27 smithi144 podman[66069]: 2024-05-14 15:20:27.067248277 +0000 UTC m=+0.353887515 container start 4b34cf157ad684386cab930f423e685275034d1de3b650f4ee9aa83e21aad11f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5) 2024-05-14T15:20:27.546 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:20:27 smithi144 ceph-mds[66088]: -- 172.21.15.144:0/3585166738 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636e44fba40 con 0x5636e4543600 2024-05-14T15:20:27.547 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:20:27 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h[66084]: starting mds.h at 2024-05-14T15:20:27.547 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:20:27 smithi144 bash[66069]: 4b34cf157ad684386cab930f423e685275034d1de3b650f4ee9aa83e21aad11f 2024-05-14T15:20:27.547 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:20:27 smithi144 systemd[1]: Started Ceph mds.h for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:27.914 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: Deploying daemon mds.i on smithi193 2024-05-14T15:20:28.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:28.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] up:boot 2024-05-14T15:20:28.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: fsmap 8 up:standby 2024-05-14T15:20:28.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:28 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:20:28.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:28.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:28.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:28.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: Deploying daemon mds.i on smithi193 2024-05-14T15:20:28.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:28.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] up:boot 2024-05-14T15:20:28.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: fsmap 8 up:standby 2024-05-14T15:20:28.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:28 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:20:28.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:28.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:28.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:28.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:28.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: Deploying daemon mds.i on smithi193 2024-05-14T15:20:28.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:28.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] up:boot 2024-05-14T15:20:28.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: fsmap 8 up:standby 2024-05-14T15:20:28.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:28 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:20:29.553 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 systemd[1]: Starting Ceph mds.i for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 -- 172.21.15.5:0/2425891716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 msgr2=0x7f2da01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 --2- 172.21.15.5:0/2425891716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01100c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800b0d0 tx=0x7f2d9800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 -- 172.21.15.5:0/2425891716 shutdown_connections 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 --2- 172.21.15.5:0/2425891716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01100c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 --2- 172.21.15.5:0/2425891716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 0x7f2da01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 --2- 172.21.15.5:0/2425891716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2da0107e10 0x7f2da0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 -- 172.21.15.5:0/2425891716 >> 172.21.15.5:0/2425891716 conn(0x7f2da00fef30 msgr2=0x7f2da0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 -- 172.21.15.5:0/2425891716 shutdown_connections 2024-05-14T15:20:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.833+0000 7f2da75e1700 1 -- 172.21.15.5:0/2425891716 wait complete. 2024-05-14T15:20:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 Processor -- start 2024-05-14T15:20:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 -- start start 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 0x7f2da01a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2da0107e10 0x7f2da01a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0113390 con 0x7f2da0108760 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0113210 con 0x7f2da01034e0 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da75e1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0113510 con 0x7f2da0107e10 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da5b7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da5b7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48220/0 (socket says 172.21.15.5:48220) 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da4b7c700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2da0107e10 0x7f2da01a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.834+0000 7f2da5b7e700 1 -- 172.21.15.5:0/835810920 learned_addr learned my addr 172.21.15.5:0/835810920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da537d700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 0x7f2da01a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da5b7e700 1 -- 172.21.15.5:0/835810920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2da0107e10 msgr2=0x7f2da01a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da5b7e700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2da0107e10 0x7f2da01a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da5b7e700 1 -- 172.21.15.5:0/835810920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 msgr2=0x7f2da01a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da5b7e700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 0x7f2da01a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da5b7e700 1 -- 172.21.15.5:0/835810920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da01a1920 con 0x7f2da0108760 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da537d700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 0x7f2da01a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da5b7e700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01a1240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009850 tx=0x7f2d98007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9801f070 con 0x7f2da0108760 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da01a1bb0 con 0x7f2da0108760 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d98004c50 con 0x7f2da0108760 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9801b7b0 con 0x7f2da0108760 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.835+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01a2090 con 0x7f2da0108760 2024-05-14T15:20:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.836+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da0104990 con 0x7f2da0108760 2024-05-14T15:20:29.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.839+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2d98022070 con 0x7f2da0108760 2024-05-14T15:20:29.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.840+0000 7f2d967fc700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2d8c0b2880 0x7f2d8c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:29.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.840+0000 7f2da537d700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2d8c0b2880 0x7f2d8c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:29.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.840+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2d9800f020 con 0x7f2da0108760 2024-05-14T15:20:29.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.840+0000 7f2da537d700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2d8c0b2880 0x7f2d8c0b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2da01a2540 tx=0x7f2d9c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:29.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:29.843+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d98029020 con 0x7f2da0108760 2024-05-14T15:20:29.854 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 podman[66507]: 2024-05-14 15:20:29.506700738 +0000 UTC m=+0.017166497 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:20:29.854 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 podman[66507]: 2024-05-14 15:20:29.607953462 +0000 UTC m=+0.118419223 container create 037ab062289e9af0206bd1bebf7162f21de743761035f79f29c2ac9f6755b28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-14T15:20:29.854 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 podman[66507]: 2024-05-14 15:20:29.852584777 +0000 UTC m=+0.363050545 container init 037ab062289e9af0206bd1bebf7162f21de743761035f79f29c2ac9f6755b28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-14T15:20:29.854 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 podman[66507]: 2024-05-14 15:20:29.855052389 +0000 UTC m=+0.365518149 container start 037ab062289e9af0206bd1bebf7162f21de743761035f79f29c2ac9f6755b28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-05-14T15:20:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.119+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2da00512c0 con 0x7f2da0108760 2024-05-14T15:20:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.120+0000 7f2d967fc700 1 -- 172.21.15.5:0/835810920 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f2da00512c0 con 0x7f2da0108760 2024-05-14T15:20:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:30.120 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T15:20:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2d8c0b2880 msgr2=0x7f2d8c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2d8c0b2880 0x7f2d8c0b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2da01a2540 tx=0x7f2d9c00a680 comp rx=0 tx=0).stop 2024-05-14T15:20:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 msgr2=0x7f2da01a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01a1240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009850 tx=0x7f2d98007a20 comp rx=0 tx=0).stop 2024-05-14T15:20:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 shutdown_connections 2024-05-14T15:20:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2d8c0b2880 0x7f2d8c0b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2da0108760 0x7f2da01a1240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2da01034e0 0x7f2da01a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 --2- 172.21.15.5:0/835810920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2da0107e10 0x7f2da01a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.122+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 >> 172.21.15.5:0/835810920 conn(0x7f2da00fef30 msgr2=0x7f2da0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.123+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 shutdown_connections 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:30.123+0000 7f2da75e1700 1 -- 172.21.15.5:0/835810920 wait complete. 2024-05-14T15:20:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-14T15:20:30.302 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 ceph-mds[66525]: -- 172.21.15.193:0/172139396 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d711d45a40 con 0x55d711d8d600 2024-05-14T15:20:30.302 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i[66521]: starting mds.i at 2024-05-14T15:20:30.302 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 bash[66507]: 037ab062289e9af0206bd1bebf7162f21de743761035f79f29c2ac9f6755b28d 2024-05-14T15:20:30.303 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:20:29 smithi193 systemd[1]: Started Ceph mds.i for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:30.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:30.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:30.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:30.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:30.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:30 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/835810920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:30.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:30.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:30.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:30.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:30.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:30 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/835810920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:30.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:30.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:30.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:30.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:30.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:30.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:30 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/835810920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:31.636 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:31 smithi005 ceph-mon[31559]: Deploying daemon mds.g on smithi005 2024-05-14T15:20:31.636 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] up:boot 2024-05-14T15:20:31.637 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:31 smithi005 ceph-mon[31559]: fsmap 9 up:standby 2024-05-14T15:20:31.637 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:31 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:20:31.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:31 smithi144 ceph-mon[35650]: Deploying daemon mds.g on smithi005 2024-05-14T15:20:31.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] up:boot 2024-05-14T15:20:31.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:31 smithi144 ceph-mon[35650]: fsmap 9 up:standby 2024-05-14T15:20:31.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:31 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:20:31.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:31 smithi193 ceph-mon[36001]: Deploying daemon mds.g on smithi005 2024-05-14T15:20:31.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] up:boot 2024-05-14T15:20:31.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:31 smithi193 ceph-mon[36001]: fsmap 9 up:standby 2024-05-14T15:20:31.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:31 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:20:32.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:32 smithi005 ceph-mon[31559]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:32.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:32.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:32 smithi144 ceph-mon[35650]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:32.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:32 smithi193 ceph-mon[36001]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:33.176 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:20:32 smithi005 systemd[1]: Started Ceph mds.g for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:34.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: Deploying daemon mds.a on smithi005 2024-05-14T15:20:34.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:34.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.5:6838/45851617,v1:172.21.15.5:6839/45851617] up:boot 2024-05-14T15:20:34.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: fsmap 10 up:standby 2024-05-14T15:20:34.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T15:20:34.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: Deploying daemon mds.a on smithi005 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.5:6838/45851617,v1:172.21.15.5:6839/45851617] up:boot 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: fsmap 10 up:standby 2024-05-14T15:20:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T15:20:34.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:34.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: Deploying daemon mds.a on smithi005 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.5:6838/45851617,v1:172.21.15.5:6839/45851617] up:boot 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: fsmap 10 up:standby 2024-05-14T15:20:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 -- 172.21.15.5:0/2629958148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 msgr2=0x7f597c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2629958148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c1100e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f597400b0d0 tx=0x7f597400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 -- 172.21.15.5:0/2629958148 shutdown_connections 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2629958148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c1100e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2629958148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c103550 0x7f597c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2629958148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c107e30 0x7f597c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 -- 172.21.15.5:0/2629958148 >> 172.21.15.5:0/2629958148 conn(0x7f597c078e00 msgr2=0x7f597c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 -- 172.21.15.5:0/2629958148 shutdown_connections 2024-05-14T15:20:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.920+0000 7f5983e3e700 1 -- 172.21.15.5:0/2629958148 wait complete. 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 Processor -- start 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 -- start start 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c103550 0x7f597c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c107e30 0x7f597c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f597c113400 con 0x7f597c108780 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f597c113280 con 0x7f597c103550 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f5983e3e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f597c113580 con 0x7f597c107e30 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f59823db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f59823db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48250/0 (socket says 172.21.15.5:48250) 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.921+0000 7f59823db700 1 -- 172.21.15.5:0/2674392284 learned_addr learned my addr 172.21.15.5:0/2674392284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f5981bda700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c103550 0x7f597c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:35.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59823db700 1 -- 172.21.15.5:0/2674392284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c107e30 msgr2=0x7f597c19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59813d9700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c107e30 0x7f597c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59823db700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c107e30 0x7f597c19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59823db700 1 -- 172.21.15.5:0/2674392284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c103550 msgr2=0x7f597c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59823db700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c103550 0x7f597c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59823db700 1 -- 172.21.15.5:0/2674392284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c1a6f40 con 0x7f597c108780 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f5981bda700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c103550 0x7f597c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59813d9700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c107e30 0x7f597c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f59823db700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c19c720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f59740079a0 tx=0x7f5974007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597401f070 con 0x7f597c108780 2024-05-14T15:20:35.349 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c1a71d0 con 0x7f597c108780 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5974003c00 con 0x7f597c108780 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597401b6c0 con 0x7f597c108780 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.922+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a7680 con 0x7f597c108780 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.923+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c0512c0 con 0x7f597c108780 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.925+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5974022070 con 0x7f597c108780 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.926+0000 7f596effd700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f59680b2900 0x7f59680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:35.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.926+0000 7f5981bda700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f59680b2900 0x7f59680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.926+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f597400f020 con 0x7f597c108780 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.927+0000 7f5981bda700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f59680b2900 0x7f59680b4e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5970005fd0 tx=0x7f5970005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:34.930+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5974029020 con 0x7f597c108780 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.200+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597c1a79d0 con 0x7f597c108780 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.200+0000 7f596effd700 1 -- 172.21.15.5:0/2674392284 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f59740ab660 con 0x7f597c108780 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f59680b2900 msgr2=0x7f59680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f59680b2900 0x7f59680b4e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5970005fd0 tx=0x7f5970005e30 comp rx=0 tx=0).stop 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 msgr2=0x7f597c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c19c720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f59740079a0 tx=0x7f5974007a80 comp rx=0 tx=0).stop 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 shutdown_connections 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f59680b2900 0x7f59680b4e10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f597c108780 0x7f597c19c720 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f597c103550 0x7f597c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 --2- 172.21.15.5:0/2674392284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f597c107e30 0x7f597c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:35.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 >> 172.21.15.5:0/2674392284 conn(0x7f597c078e00 msgr2=0x7f597c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 shutdown_connections 2024-05-14T15:20:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:35.203+0000 7f5983e3e700 1 -- 172.21.15.5:0/2674392284 wait complete. 2024-05-14T15:20:35.352 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-05-14T15:20:35.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:35 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2674392284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:35.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:35 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2674392284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:35.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2674392284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:36.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:36 smithi005 ceph-mon[31559]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:36.546 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:36 smithi144 ceph-mon[35650]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:36.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:36 smithi193 ceph-mon[36001]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:37.564 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:20:37 smithi005 systemd[1]: Started Ceph mds.a for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:37.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.5:6840/1161784438,v1:172.21.15.5:6841/1161784438] up:boot 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: fsmap 11 up:standby 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:38.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:38.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:38.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:38 smithi005 ceph-mon[31559]: Deploying daemon mds.b on smithi144 2024-05-14T15:20:38.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:38.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.5:6840/1161784438,v1:172.21.15.5:6841/1161784438] up:boot 2024-05-14T15:20:38.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: fsmap 11 up:standby 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:38.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:38 smithi144 ceph-mon[35650]: Deploying daemon mds.b on smithi144 2024-05-14T15:20:38.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:38.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.5:6840/1161784438,v1:172.21.15.5:6841/1161784438] up:boot 2024-05-14T15:20:38.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: fsmap 11 up:standby 2024-05-14T15:20:38.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:38.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:38 smithi193 ceph-mon[36001]: Deploying daemon mds.b on smithi144 2024-05-14T15:20:39.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 -- 172.21.15.5:0/3346915017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 msgr2=0x7f2f901111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 --2- 172.21.15.5:0/3346915017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f901111c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800b0d0 tx=0x7f2f8800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 -- 172.21.15.5:0/3346915017 shutdown_connections 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 --2- 172.21.15.5:0/3346915017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f901111c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 --2- 172.21.15.5:0/3346915017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 0x7f2f901049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 --2- 172.21.15.5:0/3346915017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90108f10 0x7f2f90109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 -- 172.21.15.5:0/3346915017 >> 172.21.15.5:0/3346915017 conn(0x7f2f900ffe70 msgr2=0x7f2f901022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 -- 172.21.15.5:0/3346915017 shutdown_connections 2024-05-14T15:20:39.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.940+0000 7f2f948df700 1 -- 172.21.15.5:0/3346915017 wait complete. 2024-05-14T15:20:39.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.941+0000 7f2f948df700 1 Processor -- start 2024-05-14T15:20:39.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.941+0000 7f2f948df700 1 -- start start 2024-05-14T15:20:39.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f948df700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 0x7f2f9019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:39.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f948df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90108f10 0x7f2f9019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f948df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f9019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f948df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90114490 con 0x7f2f90109860 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f948df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90114310 con 0x7f2f90108f10 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f948df700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90114610 con 0x7f2f901045e0 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f9019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f9019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53938/0 (socket says 172.21.15.5:53938) 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 -- 172.21.15.5:0/2291765644 learned_addr learned my addr 172.21.15.5:0/2291765644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8dd9b700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90108f10 0x7f2f9019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8e59c700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 0x7f2f9019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 -- 172.21.15.5:0/2291765644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 msgr2=0x7f2f9019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 0x7f2f9019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 -- 172.21.15.5:0/2291765644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90108f10 msgr2=0x7f2f9019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90108f10 0x7f2f9019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:39.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 -- 172.21.15.5:0/2291765644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9019fcf0 con 0x7f2f90109860 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8e59c700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 0x7f2f9019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f8ed9d700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f9019f650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009850 tx=0x7f2f88007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.942+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8801f070 con 0x7f2f90109860 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.943+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9019ff80 con 0x7f2f90109860 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.943+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f88004c50 con 0x7f2f90109860 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.943+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8801b7b0 con 0x7f2f90109860 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.943+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a0460 con 0x7f2f90109860 2024-05-14T15:20:39.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.944+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f900512c0 con 0x7f2f90109860 2024-05-14T15:20:39.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.946+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2f88022070 con 0x7f2f90109860 2024-05-14T15:20:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.947+0000 7f2f7f7fe700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2f780b2880 0x7f2f780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.947+0000 7f2f8e59c700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2f780b2880 0x7f2f780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.947+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2f8800f020 con 0x7f2f90109860 2024-05-14T15:20:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.948+0000 7f2f8e59c700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2f780b2880 0x7f2f780b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f80005950 tx=0x7f2f80005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:39.950+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f88029020 con 0x7f2f90109860 2024-05-14T15:20:40.046 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:39 smithi144 systemd[1]: Starting Ceph mds.b for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:20:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.221+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f9002e5f0 con 0x7f2f90109860 2024-05-14T15:20:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.222+0000 7f2f7f7fe700 1 -- 172.21.15.5:0/2291765644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f2f880ab540 con 0x7f2f90109860 2024-05-14T15:20:40.222 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:40.222 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T15:20:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2f780b2880 msgr2=0x7f2f780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2f780b2880 0x7f2f780b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f80005950 tx=0x7f2f80005f50 comp rx=0 tx=0).stop 2024-05-14T15:20:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 msgr2=0x7f2f9019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f9019f650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009850 tx=0x7f2f88007a20 comp rx=0 tx=0).stop 2024-05-14T15:20:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 shutdown_connections 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2f780b2880 0x7f2f780b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f90109860 0x7f2f9019f650 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f901045e0 0x7f2f9019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 --2- 172.21.15.5:0/2291765644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f90108f10 0x7f2f9019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.224+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 >> 172.21.15.5:0/2291765644 conn(0x7f2f900ffe70 msgr2=0x7f2f90106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.225+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 shutdown_connections 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:40.225+0000 7f2f948df700 1 -- 172.21.15.5:0/2291765644 wait complete. 2024-05-14T15:20:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-05-14T15:20:40.328 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 podman[66373]: 2024-05-14 15:20:39.953394012 +0000 UTC m=+0.017627328 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:20:40.328 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 podman[66373]: 2024-05-14 15:20:40.066963519 +0000 UTC m=+0.131196864 container create 099fd1712df50f41e3dd0695f2ae0b3ea8ce6fa4c60c57e03893a866d5eed1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=) 2024-05-14T15:20:40.328 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 podman[66373]: 2024-05-14 15:20:40.328147361 +0000 UTC m=+0.392380674 container init 099fd1712df50f41e3dd0695f2ae0b3ea8ce6fa4c60c57e03893a866d5eed1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-14T15:20:40.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:40 smithi144 ceph-mon[35650]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:40.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:40 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2291765644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:40 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 podman[66373]: 2024-05-14 15:20:40.331442775 +0000 UTC m=+0.395676086 container start 099fd1712df50f41e3dd0695f2ae0b3ea8ce6fa4c60c57e03893a866d5eed1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 ceph-mds[66392]: -- 172.21.15.144:0/2329155850 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4b78cda40 con 0x55f4b7915a80 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: starting mds.b at 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 bash[66373]: 099fd1712df50f41e3dd0695f2ae0b3ea8ce6fa4c60c57e03893a866d5eed1a7 2024-05-14T15:20:40.797 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:20:40 smithi144 systemd[1]: Started Ceph mds.b for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:20:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:40 smithi193 ceph-mon[36001]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:40 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2291765644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:40.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:40 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:40.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:40 smithi005 ceph-mon[31559]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:40.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:40 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2291765644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:40.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:40.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:40 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] up:boot 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: fsmap 12 up:standby 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:41.773 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] up:boot 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: fsmap 12 up:standby 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:41.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] up:boot 2024-05-14T15:20:41.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: fsmap 12 up:standby 2024-05-14T15:20:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:20:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:42.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:42.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:42.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:20:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:43.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:43.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:43.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:43.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:43 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:43.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:43.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:43.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:43.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:43 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:43.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:43.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:43.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:43.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:43 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:20:44.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:44 smithi144 ceph-mon[35650]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:44.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:44 smithi193 ceph-mon[36001]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:44.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:44 smithi005 ceph-mon[31559]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 -- 172.21.15.5:0/452824525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61041034c0 msgr2=0x7f61041038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 --2- 172.21.15.5:0/452824525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61041034c0 0x7f61041038d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f60f40099c0 tx=0x7f60f4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 -- 172.21.15.5:0/452824525 shutdown_connections 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 --2- 172.21.15.5:0/452824525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104108740 0x7f61041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 --2- 172.21.15.5:0/452824525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61041034c0 0x7f61041038d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 --2- 172.21.15.5:0/452824525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6104107df0 0x7f6104108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 -- 172.21.15.5:0/452824525 >> 172.21.15.5:0/452824525 conn(0x7f61040fedc0 msgr2=0x7f61041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 -- 172.21.15.5:0/452824525 shutdown_connections 2024-05-14T15:20:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.977+0000 7f6108baf700 1 -- 172.21.15.5:0/452824525 wait complete. 2024-05-14T15:20:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 Processor -- start 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 -- start start 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61041034c0 0x7f6104110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 0x7f6104110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6104108740 0x7f6104111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61041144c0 con 0x7f6104108740 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6104114340 con 0x7f6104107df0 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6108baf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6104114640 con 0x7f61041034c0 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6102d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6104108740 0x7f6104111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6101d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 0x7f6104110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6101d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 0x7f6104110fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37178/0 (socket says 172.21.15.5:37178) 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f6101d9b700 1 -- 172.21.15.5:0/2813182067 learned_addr learned my addr 172.21.15.5:0/2813182067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.978+0000 7f610259c700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61041034c0 0x7f6104110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6101d9b700 1 -- 172.21.15.5:0/2813182067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61041034c0 msgr2=0x7f6104110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6101d9b700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61041034c0 0x7f6104110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6101d9b700 1 -- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6104108740 msgr2=0x7f6104111500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6101d9b700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6104108740 0x7f6104111500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6101d9b700 1 -- 172.21.15.5:0/2813182067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61041a9170 con 0x7f6104107df0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f610259c700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61041034c0 0x7f6104110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6101d9b700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 0x7f6104110fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00d980 tx=0x7f60ec00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec017070 con 0x7f6104107df0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec009de0 con 0x7f6104107df0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61041a9460 con 0x7f6104107df0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00f3c0 con 0x7f6104107df0 2024-05-14T15:20:44.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.979+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61041a99f0 con 0x7f6104107df0 2024-05-14T15:20:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.980+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61040512c0 con 0x7f6104107df0 2024-05-14T15:20:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.982+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f60ec009930 con 0x7f6104107df0 2024-05-14T15:20:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.982+0000 7f60fb7fe700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60f00b2900 0x7f60f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.983+0000 7f610259c700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60f00b2900 0x7f60f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.983+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f60ec0e1bb0 con 0x7f6104107df0 2024-05-14T15:20:44.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.983+0000 7f610259c700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60f00b2900 0x7f60f00b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006010 tx=0x7f60f401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:44.987+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60ec0a9d70 con 0x7f6104107df0 2024-05-14T15:20:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.254+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6104111fb0 con 0x7f6104107df0 2024-05-14T15:20:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.255+0000 7f60fb7fe700 1 -- 172.21.15.5:0/2813182067 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f60ec0a9670 con 0x7f6104107df0 2024-05-14T15:20:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:45.255 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-14T15:20:45.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.257+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60f00b2900 msgr2=0x7f60f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.257+0000 7f6108baf700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60f00b2900 0x7f60f00b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006010 tx=0x7f60f401a040 comp rx=0 tx=0).stop 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 msgr2=0x7f6104110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 0x7f6104110fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00d980 tx=0x7f60ec00dc90 comp rx=0 tx=0).stop 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 shutdown_connections 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60f00b2900 0x7f60f00b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6104108740 0x7f6104111500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61041034c0 0x7f6104110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 --2- 172.21.15.5:0/2813182067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6104107df0 0x7f6104110fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 >> 172.21.15.5:0/2813182067 conn(0x7f61040fedc0 msgr2=0x7f6104101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 shutdown_connections 2024-05-14T15:20:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:45.258+0000 7f6108baf700 1 -- 172.21.15.5:0/2813182067 wait complete. 2024-05-14T15:20:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-05-14T15:20:45.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:45 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:45.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:45 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:45.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:45 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:45.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:45 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:45.972 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:45 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2813182067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:46.000 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-14T15:20:46.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:45 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:45 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:45 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:45 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:45 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2813182067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:46.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:45 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:45 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:45 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:45 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:46.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:45 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2813182067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:20:46.435 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:47.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:46 smithi005 ceph-mon[31559]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:47.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:46 smithi144 ceph-mon[35650]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:47.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:46 smithi193 ceph-mon[36001]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 -- 172.21.15.5:0/2794277406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a601031d0 msgr2=0x7f0a60111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 --2- 172.21.15.5:0/2794277406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a601031d0 0x7f0a60111170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800b0a0 tx=0x7f0a5800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 -- 172.21.15.5:0/2794277406 shutdown_connections 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 --2- 172.21.15.5:0/2794277406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a601031d0 0x7f0a60111170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 --2- 172.21.15.5:0/2794277406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a60109300 0x7f0a60109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 --2- 172.21.15.5:0/2794277406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a60102880 0x7f0a60102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 -- 172.21.15.5:0/2794277406 >> 172.21.15.5:0/2794277406 conn(0x7f0a600fe6f0 msgr2=0x7f0a60100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 -- 172.21.15.5:0/2794277406 shutdown_connections 2024-05-14T15:20:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.064+0000 7f0a66f97700 1 -- 172.21.15.5:0/2794277406 wait complete. 2024-05-14T15:20:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 Processor -- start 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 -- start start 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 0x7f0a6019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a601031d0 0x7f0a6019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a60109300 0x7f0a601a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a601145f0 con 0x7f0a60102880 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a60114470 con 0x7f0a601031d0 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a66f97700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a60114770 con 0x7f0a60109300 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a64d33700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 0x7f0a6019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a5ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a601031d0 0x7f0a6019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a64d33700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 0x7f0a6019d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45496/0 (socket says 172.21.15.5:45496) 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a64d33700 1 -- 172.21.15.5:0/802502662 learned_addr learned my addr 172.21.15.5:0/802502662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.065+0000 7f0a5ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a601031d0 0x7f0a6019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:36256/0 (socket says 172.21.15.5:36256) 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a65534700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a60109300 0x7f0a601a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a64d33700 1 -- 172.21.15.5:0/802502662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a60109300 msgr2=0x7f0a601a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a64d33700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a60109300 0x7f0a601a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a64d33700 1 -- 172.21.15.5:0/802502662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a601031d0 msgr2=0x7f0a6019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a64d33700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a601031d0 0x7f0a6019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a64d33700 1 -- 172.21.15.5:0/802502662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a601a2030 con 0x7f0a60102880 2024-05-14T15:20:48.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a64d33700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 0x7f0a6019d160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400d950 tx=0x7f0a5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:48.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54017070 con 0x7f0a60102880 2024-05-14T15:20:48.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a601a2320 con 0x7f0a60102880 2024-05-14T15:20:48.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5400f690 con 0x7f0a60102880 2024-05-14T15:20:48.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.066+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54003aa0 con 0x7f0a60102880 2024-05-14T15:20:48.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.067+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a601a9c40 con 0x7f0a60102880 2024-05-14T15:20:48.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.067+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a600512c0 con 0x7f0a60102880 2024-05-14T15:20:48.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.069+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0a5400f840 con 0x7f0a60102880 2024-05-14T15:20:48.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.070+0000 7f0a5dffb700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a500b2880 0x7f0a500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:48.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.070+0000 7f0a5ffff700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a500b2880 0x7f0a500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:48.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.070+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0a540e1f00 con 0x7f0a60102880 2024-05-14T15:20:48.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.071+0000 7f0a5ffff700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a500b2880 0x7f0a500b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a601104a0 tx=0x7f0a4c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.075+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a54020030 con 0x7f0a60102880 2024-05-14T15:20:48.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.301+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0a601a9e10 con 0x7f0a60102880 2024-05-14T15:20:48.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.302+0000 7f0a5dffb700 1 -- 172.21.15.5:0/802502662 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f0a540aa0c0 con 0x7f0a60102880 2024-05-14T15:20:48.303 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:20:48.303 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-14T15:20:48.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:20:48.304 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-14T15:20:48.304 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-14T15:20:48.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:20:48.304 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-14T15:20:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a500b2880 msgr2=0x7f0a500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a500b2880 0x7f0a500b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a601104a0 tx=0x7f0a4c00b410 comp rx=0 tx=0).stop 2024-05-14T15:20:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 msgr2=0x7f0a6019d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 0x7f0a6019d160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400d950 tx=0x7f0a5400dc60 comp rx=0 tx=0).stop 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 shutdown_connections 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0a500b2880 0x7f0a500b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a60109300 0x7f0a601a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a60102880 0x7f0a6019d160 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 --2- 172.21.15.5:0/802502662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a601031d0 0x7f0a6019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.304+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 >> 172.21.15.5:0/802502662 conn(0x7f0a600fe6f0 msgr2=0x7f0a60110e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.305+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 shutdown_connections 2024-05-14T15:20:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:48.305+0000 7f0a66f97700 1 -- 172.21.15.5:0/802502662 wait complete. 2024-05-14T15:20:48.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T15:20:48.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T15:20:48.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:48 smithi005 ceph-mon[31559]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:48.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:48.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:48 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:48.966 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:48 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/802502662' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T15:20:49.218 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:49.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:48 smithi144 ceph-mon[35650]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:49.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:49.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:48 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:49.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:48 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/802502662' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T15:20:49.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:48 smithi193 ceph-mon[36001]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:49.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:49.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:48 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:49.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:48 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/802502662' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T15:20:50.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:50.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:50.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:50.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:20:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:20:50.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:20:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 -- 172.21.15.5:0/3909972518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 msgr2=0x7fe0b01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/3909972518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b01038f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe0980099c0 tx=0x7fe098009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:50.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:50 smithi005 ceph-mon[31559]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 -- 172.21.15.5:0/3909972518 shutdown_connections 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/3909972518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0b0108760 0x7fe0b01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/3909972518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b01038f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/3909972518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 0x7fe0b0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 -- 172.21.15.5:0/3909972518 >> 172.21.15.5:0/3909972518 conn(0x7fe0b00fef30 msgr2=0x7fe0b0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 -- 172.21.15.5:0/3909972518 shutdown_connections 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.968+0000 7fe0b5f70700 1 -- 172.21.15.5:0/3909972518 wait complete. 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.969+0000 7fe0b5f70700 1 Processor -- start 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.969+0000 7fe0b5f70700 1 -- start start 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.969+0000 7fe0b5f70700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.969+0000 7fe0b5f70700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 0x7fe0b019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.969+0000 7fe0b5f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0b0108760 0x7fe0b019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0b5f70700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0113400 con 0x7fe0b01034e0 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0b5f70700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0113280 con 0x7fe0b0108760 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0b5f70700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0113580 con 0x7fe0b0107e10 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b019aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45516/0 (socket says 172.21.15.5:45516) 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 -- 172.21.15.5:0/2083702078 learned_addr learned my addr 172.21.15.5:0/2083702078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0aeffd700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 0x7fe0b019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0affff700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0b0108760 0x7fe0b019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 -- 172.21.15.5:0/2083702078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 msgr2=0x7fe0b019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 0x7fe0b019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 -- 172.21.15.5:0/2083702078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0b0108760 msgr2=0x7fe0b019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0b0108760 0x7fe0b019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 -- 172.21.15.5:0/2083702078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b019fd90 con 0x7fe0b01034e0 2024-05-14T15:20:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0aeffd700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 0x7fe0b019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.970+0000 7fe0af7fe700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b019aec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b580 tx=0x7fe0980037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.971+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098027070 con 0x7fe0b01034e0 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.971+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b01a0020 con 0x7fe0b01034e0 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.971+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe098004110 con 0x7fe0b01034e0 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.971+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09800f460 con 0x7fe0b01034e0 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.971+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01a0500 con 0x7fe0b01034e0 2024-05-14T15:20:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.972+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b00512c0 con 0x7fe0b01034e0 2024-05-14T15:20:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.973+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe09802a070 con 0x7fe0b01034e0 2024-05-14T15:20:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.975+0000 7fe0acff9700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe09c0b2880 0x7fe09c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.975+0000 7fe0aeffd700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe09c0b2880 0x7fe09c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.975+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fe0980e3130 con 0x7fe0b01034e0 2024-05-14T15:20:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.975+0000 7fe0aeffd700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe09c0b2880 0x7fe09c0b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe0b01021e0 tx=0x7fe0a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:50.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:50.978+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe098031080 con 0x7fe0b01034e0 2024-05-14T15:20:51.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.193+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe0b002e5f0 con 0x7fe0b01034e0 2024-05-14T15:20:51.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:50 smithi144 ceph-mon[35650]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:51.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:50 smithi193 ceph-mon[36001]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.877+0000 7fe0acff9700 1 -- 172.21.15.5:0/2083702078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fe098013070 con 0x7fe0b01034e0 2024-05-14T15:20:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.880+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe09c0b2880 msgr2=0x7fe09c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.880+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe09c0b2880 0x7fe09c0b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe0b01021e0 tx=0x7fe0a0008040 comp rx=0 tx=0).stop 2024-05-14T15:20:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.880+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 msgr2=0x7fe0b019aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.880+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b019aec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b580 tx=0x7fe0980037c0 comp rx=0 tx=0).stop 2024-05-14T15:20:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 shutdown_connections 2024-05-14T15:20:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe09c0b2880 0x7fe09c0b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0b0108760 0x7fe0b019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b01034e0 0x7fe0b019aec0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 --2- 172.21.15.5:0/2083702078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0107e10 0x7fe0b019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 >> 172.21.15.5:0/2083702078 conn(0x7fe0b00fef30 msgr2=0x7fe0b01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 shutdown_connections 2024-05-14T15:20:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:51.881+0000 7fe0b5f70700 1 -- 172.21.15.5:0/2083702078 wait complete. 2024-05-14T15:20:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-14T15:20:52.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:51 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2083702078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T15:20:52.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:51 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2083702078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T15:20:52.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:51 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2083702078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T15:20:52.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-14T15:20:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:53.188 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:52 smithi005 ceph-mon[31559]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:53.188 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:52 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2083702078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T15:20:53.188 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:52 smithi005 ceph-mon[31559]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T15:20:53.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:52 smithi144 ceph-mon[35650]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:53.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:52 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2083702078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T15:20:53.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:52 smithi144 ceph-mon[35650]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T15:20:53.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:52 smithi193 ceph-mon[36001]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:53.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:52 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2083702078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T15:20:53.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:52 smithi193 ceph-mon[36001]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T15:20:54.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:53 smithi005 ceph-mon[31559]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T15:20:54.198 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:53 smithi144 ceph-mon[35650]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T15:20:54.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:53 smithi193 ceph-mon[36001]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T15:20:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 -- 172.21.15.5:0/1559392688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090108f00 msgr2=0x7f7090109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 --2- 172.21.15.5:0/1559392688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090108f00 0x7f7090109310 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f70780099c0 tx=0x7f7078009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 -- 172.21.15.5:0/1559392688 shutdown_connections 2024-05-14T15:20:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 --2- 172.21.15.5:0/1559392688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7090109850 0x7f70901111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 --2- 172.21.15.5:0/1559392688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70901045d0 0x7f70901049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 --2- 172.21.15.5:0/1559392688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090108f00 0x7f7090109310 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 -- 172.21.15.5:0/1559392688 >> 172.21.15.5:0/1559392688 conn(0x7f70900ffea0 msgr2=0x7f70901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 -- 172.21.15.5:0/1559392688 shutdown_connections 2024-05-14T15:20:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.430+0000 7f70958b3700 1 -- 172.21.15.5:0/1559392688 wait complete. 2024-05-14T15:20:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 Processor -- start 2024-05-14T15:20:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 -- start start 2024-05-14T15:20:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70901045d0 0x7f709019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7090108f00 0x7f709019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 0x7f709019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7090114460 con 0x7f7090109850 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.431+0000 7f70958b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70901142e0 con 0x7f7090108f00 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f70958b3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70901145e0 con 0x7f70901045d0 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 0x7f709019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70901045d0 0x7f709019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 0x7f709019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45526/0 (socket says 172.21.15.5:45526) 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 -- 172.21.15.5:0/285959741 learned_addr learned my addr 172.21.15.5:0/285959741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708e7fc700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7090108f00 0x7f709019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 -- 172.21.15.5:0/285959741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70901045d0 msgr2=0x7f709019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70901045d0 0x7f709019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 -- 172.21.15.5:0/285959741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7090108f00 msgr2=0x7f709019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7090108f00 0x7f709019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 -- 172.21.15.5:0/285959741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709019fce0 con 0x7f7090109850 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708e7fc700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7090108f00 0x7f709019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:20:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708effd700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70901045d0 0x7f709019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f708f7fe700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 0x7f709019f640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f708800d980 tx=0x7f708800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088017070 con 0x7f7090109850 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7088005020 con 0x7f7090109850 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.432+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709019ffd0 con 0x7f7090109850 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.433+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70880099b0 con 0x7f7090109850 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.433+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901a78f0 con 0x7f7090109850 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.434+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7090105a80 con 0x7f7090109850 2024-05-14T15:20:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.435+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7088009b50 con 0x7f7090109850 2024-05-14T15:20:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.436+0000 7f70948b1700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f707c0b2880 0x7f707c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.437+0000 7f708effd700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f707c0b2880 0x7f707c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.437+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f70880e2180 con 0x7f7090109850 2024-05-14T15:20:54.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.437+0000 7f708effd700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f707c0b2880 0x7f707c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7080009fd0 tx=0x7f70800091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.441+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7088020020 con 0x7f7090109850 2024-05-14T15:20:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.659+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f709019c170 con 0x7f7090109850 2024-05-14T15:20:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.897+0000 7f70948b1700 1 -- 172.21.15.5:0/285959741 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f70880aa1d0 con 0x7f7090109850 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f707c0b2880 msgr2=0x7f707c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f707c0b2880 0x7f707c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7080009fd0 tx=0x7f70800091b0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 msgr2=0x7f709019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 0x7f709019f640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f708800d980 tx=0x7f708800dc90 comp rx=0 tx=0).stop 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 shutdown_connections 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f707c0b2880 0x7f707c0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7090109850 0x7f709019f640 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70901045d0 0x7f709019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 --2- 172.21.15.5:0/285959741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7090108f00 0x7f709019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 >> 172.21.15.5:0/285959741 conn(0x7f70900ffea0 msgr2=0x7f70901086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 shutdown_connections 2024-05-14T15:20:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:54.901+0000 7f70958b3700 1 -- 172.21.15.5:0/285959741 wait complete. 2024-05-14T15:20:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-14T15:20:55.216 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:54 smithi144 ceph-mon[35650]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:55.216 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:54 smithi144 ceph-mon[35650]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:20:55.216 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:54 smithi144 ceph-mon[35650]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T15:20:55.216 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:54 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/285959741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-14T15:20:55.262 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:54 smithi005 ceph-mon[31559]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:55.263 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:54 smithi005 ceph-mon[31559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:20:55.263 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:54 smithi005 ceph-mon[31559]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T15:20:55.263 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:54 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/285959741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-14T15:20:55.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:54 smithi193 ceph-mon[36001]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:55.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:54 smithi193 ceph-mon[36001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:20:55.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:54 smithi193 ceph-mon[36001]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T15:20:55.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:54 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/285959741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-14T15:20:55.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T15:20:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:56.191 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:55 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/285959741' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-14T15:20:56.192 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:55 smithi005 ceph-mon[31559]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T15:20:56.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:55 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/285959741' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-14T15:20:56.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:55 smithi144 ceph-mon[35650]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T15:20:56.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:55 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/285959741' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-14T15:20:56.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:55 smithi193 ceph-mon[36001]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T15:20:57.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:56 smithi005 ceph-mon[31559]: pgmap v283: 97 pgs: 16 creating+peering, 16 active+clean, 65 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:57.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:56 smithi005 ceph-mon[31559]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T15:20:57.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:56 smithi144 ceph-mon[35650]: pgmap v283: 97 pgs: 16 creating+peering, 16 active+clean, 65 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:57.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:56 smithi144 ceph-mon[35650]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T15:20:57.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:56 smithi193 ceph-mon[36001]: pgmap v283: 97 pgs: 16 creating+peering, 16 active+clean, 65 unknown; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:57.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:56 smithi193 ceph-mon[36001]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.537+0000 7f8848e45700 1 -- 172.21.15.5:0/2419001249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844107df0 msgr2=0x7f8844108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.537+0000 7f8848e45700 1 --2- 172.21.15.5:0/2419001249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844107df0 0x7f8844108200 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f88380099c0 tx=0x7f8838009cd0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 -- 172.21.15.5:0/2419001249 shutdown_connections 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 --2- 172.21.15.5:0/2419001249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8844108740 0x7f88441100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 --2- 172.21.15.5:0/2419001249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88441034c0 0x7f88441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 --2- 172.21.15.5:0/2419001249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844107df0 0x7f8844108200 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 -- 172.21.15.5:0/2419001249 >> 172.21.15.5:0/2419001249 conn(0x7f88440fedc0 msgr2=0x7f88441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 -- 172.21.15.5:0/2419001249 shutdown_connections 2024-05-14T15:20:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 -- 172.21.15.5:0/2419001249 wait complete. 2024-05-14T15:20:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.538+0000 7f8848e45700 1 Processor -- start 2024-05-14T15:20:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 -- start start 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88441034c0 0x7f884419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8844107df0 0x7f884419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 0x7f884419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88441133c0 con 0x7f8844108740 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8844113240 con 0x7f8844107df0 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8848e45700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8844113540 con 0x7f88441034c0 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 0x7f884419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 0x7f884419c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55154/0 (socket says 172.21.15.5:55154) 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 -- 172.21.15.5:0/1828787516 learned_addr learned my addr 172.21.15.5:0/1828787516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f884259c700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88441034c0 0x7f884419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8841d9b700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8844107df0 0x7f884419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 -- 172.21.15.5:0/1828787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88441034c0 msgr2=0x7f884419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88441034c0 0x7f884419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 -- 172.21.15.5:0/1828787516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8844107df0 msgr2=0x7f884419c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8844107df0 0x7f884419c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.539+0000 7f8842d9d700 1 -- 172.21.15.5:0/1828787516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88441a7030 con 0x7f8844108740 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.540+0000 7f8842d9d700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 0x7f884419c780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f883c007ee0 tx=0x7f883c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.540+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c004bb0 con 0x7f8844108740 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.540+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883c004d50 con 0x7f8844108740 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.540+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88441a7320 con 0x7f8844108740 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.540+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c005920 con 0x7f8844108740 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.540+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88441a77e0 con 0x7f8844108740 2024-05-14T15:20:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.541+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88440512c0 con 0x7f8844108740 2024-05-14T15:20:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.542+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f883c01a020 con 0x7f8844108740 2024-05-14T15:20:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.543+0000 7f88337fe700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f882c0b2880 0x7f882c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:20:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.543+0000 7f884259c700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f882c0b2880 0x7f882c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:20:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.543+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f883c0e1d50 con 0x7f8844108740 2024-05-14T15:20:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.544+0000 7f884259c700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f882c0b2880 0x7f882c0b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8834009a10 tx=0x7f8834009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:20:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.547+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f883c010070 con 0x7f8844108740 2024-05-14T15:20:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.808+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f884402e5f0 con 0x7f8844108740 2024-05-14T15:20:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.916+0000 7f88337fe700 1 -- 172.21.15.5:0/1828787516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f883c0a9c20 con 0x7f8844108740 2024-05-14T15:20:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.918+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f882c0b2880 msgr2=0x7f882c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.918+0000 7f8848e45700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f882c0b2880 0x7f882c0b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8834009a10 tx=0x7f8834009190 comp rx=0 tx=0).stop 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.918+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 msgr2=0x7f884419c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.918+0000 7f8848e45700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 0x7f884419c780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f883c007ee0 tx=0x7f883c00d3b0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 shutdown_connections 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f882c0b2880 0x7f882c0b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8844108740 0x7f884419c780 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88441034c0 0x7f884419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 --2- 172.21.15.5:0/1828787516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8844107df0 0x7f884419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 >> 172.21.15.5:0/1828787516 conn(0x7f88440fedc0 msgr2=0x7f8844107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 shutdown_connections 2024-05-14T15:20:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:20:57.919+0000 7f8848e45700 1 -- 172.21.15.5:0/1828787516 wait complete. 2024-05-14T15:20:57.921 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-05-14T15:20:58.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:57 smithi005 ceph-mon[31559]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T15:20:58.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:57 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1828787516' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T15:20:58.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:57 smithi005 ceph-mon[31559]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T15:20:58.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:57 smithi144 ceph-mon[35650]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T15:20:58.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:57 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1828787516' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T15:20:58.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:57 smithi144 ceph-mon[35650]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T15:20:58.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:57 smithi193 ceph-mon[36001]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T15:20:58.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:57 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1828787516' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T15:20:58.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:57 smithi193 ceph-mon[36001]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T15:20:58.603 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-14T15:20:58.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-14T15:20:58.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:20:59.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: pgmap v286: 97 pgs: 35 creating+peering, 41 active+clean, 21 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:59.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T15:20:59.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1828787516' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T15:20:59.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T15:20:59.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T15:20:59.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] up:active 2024-05-14T15:20:59.253 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:58 smithi005 ceph-mon[31559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:20:59.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: pgmap v286: 97 pgs: 35 creating+peering, 41 active+clean, 21 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:59.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T15:20:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1828787516' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T15:20:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T15:20:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T15:20:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] up:active 2024-05-14T15:20:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:58 smithi144 ceph-mon[35650]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:20:59.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: pgmap v286: 97 pgs: 35 creating+peering, 41 active+clean, 21 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:20:59.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T15:20:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1828787516' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T15:20:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T15:20:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T15:20:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] up:active 2024-05-14T15:20:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:58 smithi193 ceph-mon[36001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:20:59.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:20:59 smithi005 ceph-mon[31559]: pgmap v288: 97 pgs: 35 creating+peering, 56 active+clean, 6 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:00.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:20:59 smithi144 ceph-mon[35650]: pgmap v288: 97 pgs: 35 creating+peering, 56 active+clean, 6 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:00.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:20:59 smithi193 ceph-mon[36001]: pgmap v288: 97 pgs: 35 creating+peering, 56 active+clean, 6 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 -- 172.21.15.5:0/2343851353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618108740 msgr2=0x7f06181100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 --2- 172.21.15.5:0/2343851353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618108740 0x7f06181100a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f061000b0d0 tx=0x7f061000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 -- 172.21.15.5:0/2343851353 shutdown_connections 2024-05-14T15:21:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 --2- 172.21.15.5:0/2343851353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618108740 0x7f06181100a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 --2- 172.21.15.5:0/2343851353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06181034c0 0x7f06181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 --2- 172.21.15.5:0/2343851353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0618107df0 0x7f0618108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 -- 172.21.15.5:0/2343851353 >> 172.21.15.5:0/2343851353 conn(0x7f06180fedf0 msgr2=0x7f0618101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:00.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 -- 172.21.15.5:0/2343851353 shutdown_connections 2024-05-14T15:21:00.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 -- 172.21.15.5:0/2343851353 wait complete. 2024-05-14T15:21:00.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.551+0000 7f061f77c700 1 Processor -- start 2024-05-14T15:21:00.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 -- start start 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06181034c0 0x7f06180ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 0x7f06180ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0618108740 0x7f06180ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113390 con 0x7f0618107df0 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113210 con 0x7f06181034c0 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061f77c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113510 con 0x7f0618108740 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 0x7f06180ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 0x7f06180ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55164/0 (socket says 172.21.15.5:55164) 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 -- 172.21.15.5:0/179145989 learned_addr learned my addr 172.21.15.5:0/179145989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061dd19700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0618108740 0x7f06180ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061d518700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06181034c0 0x7f06180ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 -- 172.21.15.5:0/179145989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0618108740 msgr2=0x7f06180ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0618108740 0x7f06180ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 -- 172.21.15.5:0/179145989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06181034c0 msgr2=0x7f06180ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06181034c0 0x7f06180ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 -- 172.21.15.5:0/179145989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06181a9150 con 0x7f0618107df0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.552+0000 7f061cd17700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 0x7f06180ff810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f060800d980 tx=0x7f060800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.553+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608017070 con 0x7f0618107df0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.553+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06181a9440 con 0x7f0618107df0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.553+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0608009de0 con 0x7f0618107df0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.553+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060800f3c0 con 0x7f0618107df0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.553+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06181a99d0 con 0x7f0618107df0 2024-05-14T15:21:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.554+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06180512c0 con 0x7f0618107df0 2024-05-14T15:21:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.556+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0608009930 con 0x7f0618107df0 2024-05-14T15:21:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.556+0000 7f060e7fc700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f06040b2880 0x7f06040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.556+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f06080e1b70 con 0x7f0618107df0 2024-05-14T15:21:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.556+0000 7f061d518700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f06040b2880 0x7f06040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.557+0000 7f061d518700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f06040b2880 0x7f06040b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0614005950 tx=0x7f0614005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.560+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06080a99e0 con 0x7f0618107df0 2024-05-14T15:21:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.781+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0618100800 con 0x7f0618107df0 2024-05-14T15:21:00.796 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:21:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: 2024-05-14T15:21:00.508+0000 7f04bf855700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T15:21:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.930+0000 7f060e7fc700 1 -- 172.21.15.5:0/179145989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f06080a92e0 con 0x7f0618107df0 2024-05-14T15:21:00.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.933+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f06040b2880 msgr2=0x7f06040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.933+0000 7f061f77c700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f06040b2880 0x7f06040b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0614005950 tx=0x7f0614005eb0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.933+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 msgr2=0x7f06180ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.933+0000 7f061f77c700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 0x7f06180ff810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f060800d980 tx=0x7f060800dc90 comp rx=0 tx=0).stop 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 shutdown_connections 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f06040b2880 0x7f06040b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0618108740 0x7f06180ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06181034c0 0x7f06180ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 --2- 172.21.15.5:0/179145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618107df0 0x7f06180ff810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 >> 172.21.15.5:0/179145989 conn(0x7f06180fedf0 msgr2=0x7f0618075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 shutdown_connections 2024-05-14T15:21:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:00.934+0000 7f061f77c700 1 -- 172.21.15.5:0/179145989 wait complete. 2024-05-14T15:21:01.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:00 smithi005 ceph-mon[31559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T15:21:01.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:00 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:21:01.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:00 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/179145989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-14T15:21:01.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:00 smithi144 ceph-mon[35650]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T15:21:01.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:00 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:21:01.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:00 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/179145989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-14T15:21:01.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:00 smithi193 ceph-mon[36001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T15:21:01.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:00 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:21:01.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:00 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/179145989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-14T15:21:01.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T15:21:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:02.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:01 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/179145989' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-14T15:21:02.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:01 smithi005 ceph-mon[31559]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T15:21:02.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:01 smithi005 ceph-mon[31559]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-05-14T15:21:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:01 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/179145989' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-14T15:21:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:01 smithi144 ceph-mon[35650]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T15:21:02.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:01 smithi144 ceph-mon[35650]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-05-14T15:21:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:01 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/179145989' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-14T15:21:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:01 smithi193 ceph-mon[36001]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T15:21:02.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:01 smithi193 ceph-mon[36001]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-05-14T15:21:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 -- 172.21.15.5:0/1028328291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 msgr2=0x7f3664108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 --2- 172.21.15.5:0/1028328291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f3664108200 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f36540099c0 tx=0x7f3654009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 -- 172.21.15.5:0/1028328291 shutdown_connections 2024-05-14T15:21:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 --2- 172.21.15.5:0/1028328291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3664108740 0x7f36641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 --2- 172.21.15.5:0/1028328291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36641034c0 0x7f36641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 --2- 172.21.15.5:0/1028328291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f3664108200 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 -- 172.21.15.5:0/1028328291 >> 172.21.15.5:0/1028328291 conn(0x7f36640fedf0 msgr2=0x7f3664101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 -- 172.21.15.5:0/1028328291 shutdown_connections 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.599+0000 7f366c2b1700 1 -- 172.21.15.5:0/1028328291 wait complete. 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 Processor -- start 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 -- start start 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36641034c0 0x7f3664199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f366419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3664108740 0x7f366419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3664113350 con 0x7f3664107df0 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36641131d0 con 0x7f3664108740 2024-05-14T15:21:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366c2b1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36641134d0 con 0x7f36641034c0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366984c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f366419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366984c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f366419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55182/0 (socket says 172.21.15.5:55182) 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366984c700 1 -- 172.21.15.5:0/664094407 learned_addr learned my addr 172.21.15.5:0/664094407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366a04d700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36641034c0 0x7f3664199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.600+0000 7f366a84e700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3664108740 0x7f366419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366984c700 1 -- 172.21.15.5:0/664094407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36641034c0 msgr2=0x7f3664199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366984c700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36641034c0 0x7f3664199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366984c700 1 -- 172.21.15.5:0/664094407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3664108740 msgr2=0x7f366419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366984c700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3664108740 0x7f366419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366984c700 1 -- 172.21.15.5:0/664094407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366419ec10 con 0x7f3664107df0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366984c700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f366419a2e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36540098e0 tx=0x7f36540057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654027070 con 0x7f3664107df0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3654003f20 con 0x7f3664107df0 2024-05-14T15:21:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366419eea0 con 0x7f3664107df0 2024-05-14T15:21:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654022c70 con 0x7f3664107df0 2024-05-14T15:21:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.601+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36641a6820 con 0x7f3664107df0 2024-05-14T15:21:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.602+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36640512c0 con 0x7f3664107df0 2024-05-14T15:21:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.604+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f365402a070 con 0x7f3664107df0 2024-05-14T15:21:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.604+0000 7f365b7fe700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36500b2900 0x7f36500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.605+0000 7f366a04d700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36500b2900 0x7f36500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.605+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f36540e37d0 con 0x7f3664107df0 2024-05-14T15:21:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.605+0000 7f366a04d700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36500b2900 0x7f36500b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3660005950 tx=0x7f366000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.608+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3654031080 con 0x7f3664107df0 2024-05-14T15:21:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:03.826+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f366402e5f0 con 0x7f3664107df0 2024-05-14T15:21:04.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:04 smithi144 ceph-mon[35650]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-14T15:21:04.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:04 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/664094407' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T15:21:04.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:04 smithi193 ceph-mon[36001]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-14T15:21:04.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:04 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/664094407' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T15:21:04.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:04 smithi005 ceph-mon[31559]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-14T15:21:04.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:04 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/664094407' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T15:21:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.516+0000 7f365b7fe700 1 -- 172.21.15.5:0/664094407 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f3654013070 con 0x7f3664107df0 2024-05-14T15:21:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.519+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36500b2900 msgr2=0x7f36500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.519+0000 7f366c2b1700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36500b2900 0x7f36500b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3660005950 tx=0x7f366000b410 comp rx=0 tx=0).stop 2024-05-14T15:21:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 msgr2=0x7f366419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f366419a2e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f36540098e0 tx=0x7f36540057c0 comp rx=0 tx=0).stop 2024-05-14T15:21:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 shutdown_connections 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f36500b2900 0x7f36500b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3664108740 0x7f366419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36641034c0 0x7f3664199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 --2- 172.21.15.5:0/664094407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3664107df0 0x7f366419a2e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 >> 172.21.15.5:0/664094407 conn(0x7f36640fedf0 msgr2=0x7f3664101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 shutdown_connections 2024-05-14T15:21:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:05.520+0000 7f366c2b1700 1 -- 172.21.15.5:0/664094407 wait complete. 2024-05-14T15:21:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' created 2024-05-14T15:21:05.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:05 smithi144 ceph-mon[35650]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T15:21:05.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:05 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/664094407' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T15:21:05.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:05 smithi144 ceph-mon[35650]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T15:21:05.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:05 smithi193 ceph-mon[36001]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T15:21:05.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:05 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/664094407' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T15:21:05.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:05 smithi193 ceph-mon[36001]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T15:21:05.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:05 smithi005 ceph-mon[31559]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T15:21:05.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:05 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/664094407' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T15:21:05.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:05 smithi005 ceph-mon[31559]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T15:21:06.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-14T15:21:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:06.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:06 smithi144 ceph-mon[35650]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-14T15:21:06.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:06 smithi144 ceph-mon[35650]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T15:21:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:06 smithi193 ceph-mon[36001]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-14T15:21:06.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:06 smithi193 ceph-mon[36001]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T15:21:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:06 smithi005 ceph-mon[31559]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-14T15:21:06.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:06 smithi005 ceph-mon[31559]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T15:21:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 -- 172.21.15.5:0/2582851015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 msgr2=0x7f0dec110780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/2582851015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec110780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009990 tx=0x7f0de0009ca0 comp rx=0 tx=0).stop 2024-05-14T15:21:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 -- 172.21.15.5:0/2582851015 shutdown_connections 2024-05-14T15:21:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/2582851015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec110cc0 0x7f0dec1130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/2582851015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec103110 0x7f0dec103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/2582851015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec110780 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 -- 172.21.15.5:0/2582851015 >> 172.21.15.5:0/2582851015 conn(0x7f0dec078e00 msgr2=0x7f0dec079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 -- 172.21.15.5:0/2582851015 shutdown_connections 2024-05-14T15:21:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.293+0000 7f0df3f0e700 1 -- 172.21.15.5:0/2582851015 wait complete. 2024-05-14T15:21:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.294+0000 7f0df3f0e700 1 Processor -- start 2024-05-14T15:21:08.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.294+0000 7f0df3f0e700 1 -- start start 2024-05-14T15:21:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df3f0e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec103110 0x7f0dec19d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df3f0e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df3f0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec110cc0 0x7f0dec1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df3f0e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec116690 con 0x7f0dec103a60 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df3f0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec116510 con 0x7f0dec110cc0 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df3f0e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec116810 con 0x7f0dec103110 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44054/0 (socket says 172.21.15.5:44054) 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 -- 172.21.15.5:0/477557784 learned_addr learned my addr 172.21.15.5:0/477557784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df1caa700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec103110 0x7f0dec19d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df24ab700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec110cc0 0x7f0dec1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 -- 172.21.15.5:0/477557784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec103110 msgr2=0x7f0dec19d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec103110 0x7f0dec19d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 -- 172.21.15.5:0/477557784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec110cc0 msgr2=0x7f0dec1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec110cc0 0x7f0dec1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 -- 172.21.15.5:0/477557784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec1a1fb0 con 0x7f0dec103a60 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df24ab700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec110cc0 0x7f0dec1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:21:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0df14a9700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec19d680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.295+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0027070 con 0x7f0dec103a60 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.296+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec1a2240 con 0x7f0dec103a60 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.296+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de0003fa0 con 0x7f0dec103a60 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.296+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec1a9bc0 con 0x7f0dec103a60 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.296+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0022dd0 con 0x7f0dec103a60 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.297+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd8005330 con 0x7f0dec103a60 2024-05-14T15:21:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.299+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0de002a070 con 0x7f0dec103a60 2024-05-14T15:21:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.299+0000 7f0deaffd700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0dd40b2900 0x7f0dd40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.300+0000 7f0df1caa700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0dd40b2900 0x7f0dd40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.300+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f0de00e3390 con 0x7f0dec103a60 2024-05-14T15:21:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.300+0000 7f0df1caa700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0dd40b2900 0x7f0dd40b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009d10 tx=0x7f0ddc009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:08.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.303+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0de0013070 con 0x7f0dec103a60 2024-05-14T15:21:08.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:08.522+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f0dd8005620 con 0x7f0dec103a60 2024-05-14T15:21:08.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:08 smithi005 ceph-mon[31559]: pgmap v296: 129 pgs: 13 creating+peering, 18 unknown, 98 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:08.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:08 smithi005 ceph-mon[31559]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T15:21:08.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:08 smithi005 ceph-mon[31559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:21:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:08 smithi144 ceph-mon[35650]: pgmap v296: 129 pgs: 13 creating+peering, 18 unknown, 98 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:08.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:08 smithi144 ceph-mon[35650]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T15:21:08.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:08 smithi144 ceph-mon[35650]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:21:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:08 smithi193 ceph-mon[36001]: pgmap v296: 129 pgs: 13 creating+peering, 18 unknown, 98 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:08 smithi193 ceph-mon[36001]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T15:21:08.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:08 smithi193 ceph-mon[36001]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T15:21:09.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.526+0000 7f0deaffd700 1 -- 172.21.15.5:0/477557784 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f0de00aaf20 con 0x7f0dec103a60 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0dd40b2900 msgr2=0x7f0dd40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0dd40b2900 0x7f0dd40b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009d10 tx=0x7f0ddc009040 comp rx=0 tx=0).stop 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 msgr2=0x7f0dec19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec19d680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de00047b0 comp rx=0 tx=0).stop 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 shutdown_connections 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0dd40b2900 0x7f0dd40b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dec110cc0 0x7f0dec1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec103110 0x7f0dec19d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 --2- 172.21.15.5:0/477557784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dec103a60 0x7f0dec19d680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 >> 172.21.15.5:0/477557784 conn(0x7f0dec078e00 msgr2=0x7f0dec1121e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 shutdown_connections 2024-05-14T15:21:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:09.531+0000 7f0df3f0e700 1 -- 172.21.15.5:0/477557784 wait complete. 2024-05-14T15:21:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:set pool 4 allow_ec_overwrites to true 2024-05-14T15:21:09.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:09 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/477557784' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T15:21:09.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:09 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/477557784' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T15:21:09.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:09 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/477557784' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-14T15:21:10.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-14T15:21:10.496 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:10.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:10 smithi144 ceph-mon[35650]: pgmap v298: 129 pgs: 13 creating+peering, 13 unknown, 103 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:10.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:10 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/477557784' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-14T15:21:10.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:10 smithi144 ceph-mon[35650]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T15:21:10.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:10 smithi193 ceph-mon[36001]: pgmap v298: 129 pgs: 13 creating+peering, 13 unknown, 103 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:10.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:10 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/477557784' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-14T15:21:10.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:10 smithi193 ceph-mon[36001]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T15:21:10.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:10 smithi005 ceph-mon[31559]: pgmap v298: 129 pgs: 13 creating+peering, 13 unknown, 103 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:10.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:10 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/477557784' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-14T15:21:10.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:10 smithi005 ceph-mon[31559]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T15:21:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 -- 172.21.15.5:0/1025614526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 msgr2=0x7f16a4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1025614526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a4102bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1698009990 tx=0x7f1698009ca0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 -- 172.21.15.5:0/1025614526 shutdown_connections 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1025614526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 0x7f16a4111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1025614526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a410a3e0 0x7f16a410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1025614526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a4102bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 -- 172.21.15.5:0/1025614526 >> 172.21.15.5:0/1025614526 conn(0x7f16a40fe6f0 msgr2=0x7f16a4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 -- 172.21.15.5:0/1025614526 shutdown_connections 2024-05-14T15:21:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.213+0000 7f16a93a4700 1 -- 172.21.15.5:0/1025614526 wait complete. 2024-05-14T15:21:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 Processor -- start 2024-05-14T15:21:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 -- start start 2024-05-14T15:21:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a41039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 0x7f16a4103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a410a3e0 0x7f16a4104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4114550 con 0x7f16a41027e0 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41143d0 con 0x7f16a4103130 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a93a4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41146d0 con 0x7f16a410a3e0 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a41039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a41039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44062/0 (socket says 172.21.15.5:44062) 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a2ffd700 1 -- 172.21.15.5:0/1835938834 learned_addr learned my addr 172.21.15.5:0/1835938834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.215+0000 7f16a37fe700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a410a3e0 0x7f16a4104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a27fc700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 0x7f16a4103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a2ffd700 1 -- 172.21.15.5:0/1835938834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a410a3e0 msgr2=0x7f16a4104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:12.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a2ffd700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a410a3e0 0x7f16a4104440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a2ffd700 1 -- 172.21.15.5:0/1835938834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 msgr2=0x7f16a4103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a2ffd700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 0x7f16a4103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a2ffd700 1 -- 172.21.15.5:0/1835938834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a41a9320 con 0x7f16a41027e0 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a27fc700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 0x7f16a4103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a2ffd700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a41039c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f169400b530 tx=0x7f169400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694015020 con 0x7f16a41027e0 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1694011ce0 con 0x7f16a41027e0 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a41a9610 con 0x7f16a41027e0 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694010e40 con 0x7f16a41027e0 2024-05-14T15:21:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.216+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41a9ba0 con 0x7f16a41027e0 2024-05-14T15:21:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.217+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a410c850 con 0x7f16a41027e0 2024-05-14T15:21:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.220+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1694018020 con 0x7f16a41027e0 2024-05-14T15:21:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.221+0000 7f168bfff700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f168c0b2900 0x7f168c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.221+0000 7f16a27fc700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f168c0b2900 0x7f168c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.221+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f16940e21c0 con 0x7f16a41027e0 2024-05-14T15:21:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.222+0000 7f16a27fc700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f168c0b2900 0x7f168c0b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1698006010 tx=0x7f169801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.225+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f169401f020 con 0x7f16a41027e0 2024-05-14T15:21:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.489+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f16a40512c0 con 0x7f16a41027e0 2024-05-14T15:21:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.530+0000 7f168bfff700 1 -- 172.21.15.5:0/1835938834 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f16940a9c60 con 0x7f16a41027e0 2024-05-14T15:21:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f168c0b2900 msgr2=0x7f168c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f168c0b2900 0x7f168c0b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1698006010 tx=0x7f169801a040 comp rx=0 tx=0).stop 2024-05-14T15:21:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 msgr2=0x7f16a41039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a41039c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f169400b530 tx=0x7f169400b840 comp rx=0 tx=0).stop 2024-05-14T15:21:12.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 shutdown_connections 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f168c0b2900 0x7f168c0b4e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a410a3e0 0x7f16a4104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a41027e0 0x7f16a41039c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 --2- 172.21.15.5:0/1835938834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a4103130 0x7f16a4103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.533+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 >> 172.21.15.5:0/1835938834 conn(0x7f16a40fe6f0 msgr2=0x7f16a4100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.534+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 shutdown_connections 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:12.534+0000 7f16a93a4700 1 -- 172.21.15.5:0/1835938834 wait complete. 2024-05-14T15:21:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:added data pool 4 to fsmap 2024-05-14T15:21:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:12 smithi144 ceph-mon[35650]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:12.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:12 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1835938834' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-14T15:21:12.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:12 smithi193 ceph-mon[36001]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:12.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:12 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1835938834' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-14T15:21:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:12 smithi005 ceph-mon[31559]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:12.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:12 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1835938834' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-14T15:21:13.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:21:13.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:13 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1835938834' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-14T15:21:13.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:13 smithi005 ceph-mon[31559]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T15:21:13.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:13 smithi005 ceph-mon[31559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:13.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:13 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1835938834' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-14T15:21:13.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:13 smithi144 ceph-mon[35650]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T15:21:13.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:13 smithi144 ceph-mon[35650]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:13.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:13 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1835938834' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-14T15:21:13.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:13 smithi193 ceph-mon[36001]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T15:21:13.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:13 smithi193 ceph-mon[36001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:14 smithi144 ceph-mon[35650]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:14 smithi144 ceph-mon[35650]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T15:21:14.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:14 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:21:14.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:14 smithi193 ceph-mon[36001]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:14.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:14 smithi193 ceph-mon[36001]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T15:21:14.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:14 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:21:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:14 smithi005 ceph-mon[31559]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:14 smithi005 ceph-mon[31559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-14T15:21:14.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:14 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:21:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 -- 172.21.15.5:0/1492351901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5370107df0 msgr2=0x7f5370108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 --2- 172.21.15.5:0/1492351901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5370107df0 0x7f5370108200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53600099c0 tx=0x7f5360009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 -- 172.21.15.5:0/1492351901 shutdown_connections 2024-05-14T15:21:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 --2- 172.21.15.5:0/1492351901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370108740 0x7f53701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 --2- 172.21.15.5:0/1492351901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53701034c0 0x7f53701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 --2- 172.21.15.5:0/1492351901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5370107df0 0x7f5370108200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 -- 172.21.15.5:0/1492351901 >> 172.21.15.5:0/1492351901 conn(0x7f53700fedf0 msgr2=0x7f5370101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 -- 172.21.15.5:0/1492351901 shutdown_connections 2024-05-14T15:21:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.343+0000 7f5377d64700 1 -- 172.21.15.5:0/1492351901 wait complete. 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 Processor -- start 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 -- start start 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 0x7f53700ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370107df0 0x7f53700ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5370108740 0x7f53700ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5370113350 con 0x7f53701034c0 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53701131d0 con 0x7f5370108740 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5377d64700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53701134d0 con 0x7f5370107df0 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 0x7f53700ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 0x7f53700ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44082/0 (socket says 172.21.15.5:44082) 2024-05-14T15:21:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 -- 172.21.15.5:0/1442715571 learned_addr learned my addr 172.21.15.5:0/1442715571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5376301700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5370108740 0x7f53700ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f53752ff700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370107df0 0x7f53700ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 -- 172.21.15.5:0/1442715571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370107df0 msgr2=0x7f53700ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370107df0 0x7f53700ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 -- 172.21.15.5:0/1442715571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5370108740 msgr2=0x7f53700ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5370108740 0x7f53700ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.344+0000 7f5375b00700 1 -- 172.21.15.5:0/1442715571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53701a9180 con 0x7f53701034c0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f53752ff700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370107df0 0x7f53700ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5376301700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5370108740 0x7f53700ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5375b00700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 0x7f53700ff2d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c920 tx=0x7f536c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c007b00 con 0x7f53701034c0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c004ce0 con 0x7f53701034c0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c005650 con 0x7f53701034c0 2024-05-14T15:21:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701a9470 con 0x7f53701034c0 2024-05-14T15:21:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.345+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701a9980 con 0x7f53701034c0 2024-05-14T15:21:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.346+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53700512c0 con 0x7f53701034c0 2024-05-14T15:21:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.349+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f536c004750 con 0x7f53701034c0 2024-05-14T15:21:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.349+0000 7f5366ffd700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f535c0b2900 0x7f535c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.349+0000 7f53752ff700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f535c0b2900 0x7f535c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.349+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f536c0e23b0 con 0x7f53701034c0 2024-05-14T15:21:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.350+0000 7f53752ff700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f535c0b2900 0x7f535c0b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5360000c00 tx=0x7f536001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.353+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f536c014030 con 0x7f53701034c0 2024-05-14T15:21:15.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.618+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f537002e5f0 con 0x7f53701034c0 2024-05-14T15:21:15.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.619+0000 7f5366ffd700 1 -- 172.21.15.5:0/1442715571 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11791 (secure 0 0 0) 0x7f536c0a9f10 con 0x7f53701034c0 2024-05-14T15:21:15.619 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:15.619 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:21:12.490627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24515,"qdb_cluster":[24515]},"id":1}]} 2024-05-14T15:21:15.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f535c0b2900 msgr2=0x7f535c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f535c0b2900 0x7f535c0b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5360000c00 tx=0x7f536001a040 comp rx=0 tx=0).stop 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 msgr2=0x7f53700ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 0x7f53700ff2d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c920 tx=0x7f536c00cce0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 shutdown_connections 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f535c0b2900 0x7f535c0b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5370108740 0x7f53700ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53701034c0 0x7f53700ff2d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 --2- 172.21.15.5:0/1442715571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5370107df0 0x7f53700ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.621+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 >> 172.21.15.5:0/1442715571 conn(0x7f53700fedf0 msgr2=0x7f5370075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.622+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 shutdown_connections 2024-05-14T15:21:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:15.622+0000 7f5377d64700 1 -- 172.21.15.5:0/1442715571 wait complete. 2024-05-14T15:21:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-05-14T15:21:16.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:21:16.649 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:16.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:16 smithi005 ceph-mon[31559]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:16.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:16 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1442715571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:16 smithi193 ceph-mon[36001]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:16 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1442715571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:21:17.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:16 smithi144 ceph-mon[35650]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:17.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:16 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1442715571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:21:18.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 -- 172.21.15.5:0/2992272986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c109850 msgr2=0x7f235c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 --2- 172.21.15.5:0/2992272986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c109850 0x7f235c1111b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23540099c0 tx=0x7f2354009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 -- 172.21.15.5:0/2992272986 shutdown_connections 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 --2- 172.21.15.5:0/2992272986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c109850 0x7f235c1111b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 --2- 172.21.15.5:0/2992272986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f235c1045d0 0x7f235c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 --2- 172.21.15.5:0/2992272986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f235c108f00 0x7f235c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 -- 172.21.15.5:0/2992272986 >> 172.21.15.5:0/2992272986 conn(0x7f235c0ffe80 msgr2=0x7f235c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 -- 172.21.15.5:0/2992272986 shutdown_connections 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.368+0000 7f2363158700 1 -- 172.21.15.5:0/2992272986 wait complete. 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 Processor -- start 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 -- start start 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f235c1045d0 0x7f235c076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 0x7f235c077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f235c109850 0x7f235c077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f235c1144d0 con 0x7f235c108f00 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f235c114350 con 0x7f235c1045d0 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.369+0000 7f2363158700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f235c114650 con 0x7f235c109850 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 0x7f235c077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 0x7f235c077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51976/0 (socket says 172.21.15.5:51976) 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 -- 172.21.15.5:0/795354093 learned_addr learned my addr 172.21.15.5:0/795354093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f23616f5700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f235c109850 0x7f235c077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f2360ef4700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f235c1045d0 0x7f235c076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 -- 172.21.15.5:0/795354093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f235c109850 msgr2=0x7f235c077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f235c109850 0x7f235c077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 -- 172.21.15.5:0/795354093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f235c1045d0 msgr2=0x7f235c076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f235c1045d0 0x7f235c076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 -- 172.21.15.5:0/795354093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c1a91e0 con 0x7f235c108f00 2024-05-14T15:21:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f235bfff700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 0x7f235c077110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f234400d950 tx=0x7f234400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344014030 con 0x7f235c108f00 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23440044d0 con 0x7f235c108f00 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440052c0 con 0x7f235c108f00 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.370+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c1a94d0 con 0x7f235c108f00 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.371+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c1a99c0 con 0x7f235c108f00 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.371+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c0512c0 con 0x7f235c108f00 2024-05-14T15:21:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.373+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2344005520 con 0x7f235c108f00 2024-05-14T15:21:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.374+0000 7f2359ffb700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f23480b2900 0x7f23480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.374+0000 7f2360ef4700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f23480b2900 0x7f23480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.374+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f23440e1f80 con 0x7f235c108f00 2024-05-14T15:21:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.375+0000 7f2360ef4700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f23480b2900 0x7f23480b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f234c009550 tx=0x7f234c0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.378+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23440a9a60 con 0x7f235c108f00 2024-05-14T15:21:18.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.594+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f235c02e7b0 con 0x7f235c108f00 2024-05-14T15:21:18.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.595+0000 7f2359ffb700 1 -- 172.21.15.5:0/795354093 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24084 (secure 0 0 0) 0x7f23440a9360 con 0x7f235c108f00 2024-05-14T15:21:18.596 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:18.596 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:12.490470+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:21:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f23480b2900 msgr2=0x7f23480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f23480b2900 0x7f23480b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f234c009550 tx=0x7f234c0091b0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 msgr2=0x7f235c077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 0x7f235c077110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f234400d950 tx=0x7f234400dc60 comp rx=0 tx=0).stop 2024-05-14T15:21:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 shutdown_connections 2024-05-14T15:21:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f23480b2900 0x7f23480b4e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f235c109850 0x7f235c077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f235c1045d0 0x7f235c076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 --2- 172.21.15.5:0/795354093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f235c108f00 0x7f235c077110 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:18.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 >> 172.21.15.5:0/795354093 conn(0x7f235c0ffe80 msgr2=0x7f235c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:18.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 shutdown_connections 2024-05-14T15:21:18.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:18.598+0000 7f2363158700 1 -- 172.21.15.5:0/795354093 wait complete. 2024-05-14T15:21:18.802 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:18 smithi193 ceph-mon[36001]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:18.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:18 smithi005 ceph-mon[31559]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:19.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:18 smithi144 ceph-mon[35650]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:19.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:21:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:19.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:19 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/795354093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:20.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:19 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/795354093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:20.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:19 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/795354093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:20.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:20 smithi005 ceph-mon[31559]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:21.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:20 smithi144 ceph-mon[35650]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:21.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:20 smithi193 ceph-mon[36001]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 -- 172.21.15.5:0/3131118556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 msgr2=0x7f5aa01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 --2- 172.21.15.5:0/3131118556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa01100a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800b0d0 tx=0x7f5a9800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 -- 172.21.15.5:0/3131118556 shutdown_connections 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 --2- 172.21.15.5:0/3131118556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa01100a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 --2- 172.21.15.5:0/3131118556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa01034c0 0x7f5aa01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 --2- 172.21.15.5:0/3131118556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5aa0107df0 0x7f5aa0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.406+0000 7f5aa878c700 1 -- 172.21.15.5:0/3131118556 >> 172.21.15.5:0/3131118556 conn(0x7f5aa00fedc0 msgr2=0x7f5aa01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.407+0000 7f5aa878c700 1 -- 172.21.15.5:0/3131118556 shutdown_connections 2024-05-14T15:21:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.407+0000 7f5aa878c700 1 -- 172.21.15.5:0/3131118556 wait complete. 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.407+0000 7f5aa878c700 1 Processor -- start 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.407+0000 7f5aa878c700 1 -- start start 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa878c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5aa01034c0 0x7f5aa019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa878c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa0107df0 0x7f5aa019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa878c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa878c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa01133e0 con 0x7f5aa0108740 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa878c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0113260 con 0x7f5aa0107df0 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa878c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0113560 con 0x7f5aa01034c0 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51996/0 (socket says 172.21.15.5:51996) 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 -- 172.21.15.5:0/1629408978 learned_addr learned my addr 172.21.15.5:0/1629408978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6528700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5aa01034c0 0x7f5aa019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa5d27700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa0107df0 0x7f5aa019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 -- 172.21.15.5:0/1629408978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5aa01034c0 msgr2=0x7f5aa019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5aa01034c0 0x7f5aa019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 -- 172.21.15.5:0/1629408978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa0107df0 msgr2=0x7f5aa019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa0107df0 0x7f5aa019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 -- 172.21.15.5:0/1629408978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa019fc90 con 0x7f5aa0108740 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5aa6d29700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa019f5f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a980095c0 tx=0x7f5a98007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.408+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9801f070 con 0x7f5aa0108740 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.409+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a98003c00 con 0x7f5aa0108740 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.409+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa019ff20 con 0x7f5aa0108740 2024-05-14T15:21:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.409+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9801b6a0 con 0x7f5aa0108740 2024-05-14T15:21:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.409+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a0400 con 0x7f5aa0108740 2024-05-14T15:21:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.410+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa00512c0 con 0x7f5aa0108740 2024-05-14T15:21:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.411+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5a98022070 con 0x7f5aa0108740 2024-05-14T15:21:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.412+0000 7f5a977fe700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5a8c0b2900 0x7f5a8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.412+0000 7f5aa6528700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5a8c0b2900 0x7f5a8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.412+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f5a980e3650 con 0x7f5aa0108740 2024-05-14T15:21:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.412+0000 7f5aa6528700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5a8c0b2900 0x7f5a8c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c005950 tx=0x7f5a9c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.416+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a980ab160 con 0x7f5aa0108740 2024-05-14T15:21:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.632+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5aa002e5f0 con 0x7f5aa0108740 2024-05-14T15:21:21.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.634+0000 7f5a977fe700 1 -- 172.21.15.5:0/1629408978 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24084 (secure 0 0 0) 0x7f5a980aaa60 con 0x7f5aa0108740 2024-05-14T15:21:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:21.634 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:12.490470+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:21:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5a8c0b2900 msgr2=0x7f5a8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5a8c0b2900 0x7f5a8c0b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c005950 tx=0x7f5a9c00b410 comp rx=0 tx=0).stop 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 msgr2=0x7f5aa019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa019f5f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a980095c0 tx=0x7f5a98007a20 comp rx=0 tx=0).stop 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 shutdown_connections 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f5a8c0b2900 0x7f5a8c0b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aa0108740 0x7f5aa019f5f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5aa01034c0 0x7f5aa019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 --2- 172.21.15.5:0/1629408978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa0107df0 0x7f5aa019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.636+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 >> 172.21.15.5:0/1629408978 conn(0x7f5aa00fedc0 msgr2=0x7f5aa006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.637+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 shutdown_connections 2024-05-14T15:21:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:21.637+0000 7f5aa878c700 1 -- 172.21.15.5:0/1629408978 wait complete. 2024-05-14T15:21:22.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:21:22.664 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:22.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:22 smithi005 ceph-mon[31559]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:22.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:22 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1629408978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:23.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:22 smithi144 ceph-mon[35650]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:23.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:22 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1629408978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:23.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:22 smithi193 ceph-mon[36001]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:23.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:22 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1629408978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 -- 172.21.15.5:0/1188236097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b81045f0 msgr2=0x7f12b8104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 --2- 172.21.15.5:0/1188236097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b81045f0 0x7f12b8104a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12b40099c0 tx=0x7f12b4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 -- 172.21.15.5:0/1188236097 shutdown_connections 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 --2- 172.21.15.5:0/1188236097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b8109870 0x7f12b81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 --2- 172.21.15.5:0/1188236097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b81045f0 0x7f12b8104a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 --2- 172.21.15.5:0/1188236097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b8108f20 0x7f12b8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.314+0000 7f12c078d700 1 -- 172.21.15.5:0/1188236097 >> 172.21.15.5:0/1188236097 conn(0x7f12b8100040 msgr2=0x7f12b8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.315+0000 7f12c078d700 1 -- 172.21.15.5:0/1188236097 shutdown_connections 2024-05-14T15:21:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.315+0000 7f12c078d700 1 -- 172.21.15.5:0/1188236097 wait complete. 2024-05-14T15:21:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.315+0000 7f12c078d700 1 Processor -- start 2024-05-14T15:21:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 -- start start 2024-05-14T15:21:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b81045f0 0x7f12b81a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 0x7f12b81a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b8109870 0x7f12b819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81144a0 con 0x7f12b8108f20 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8114320 con 0x7f12b8109870 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12c078d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8114620 con 0x7f12b81045f0 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 0x7f12b81a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 0x7f12b81a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52014/0 (socket says 172.21.15.5:52014) 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 -- 172.21.15.5:0/2875436037 learned_addr learned my addr 172.21.15.5:0/2875436037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12be529700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b81045f0 0x7f12b81a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bed2a700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b8109870 0x7f12b819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 -- 172.21.15.5:0/2875436037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b81045f0 msgr2=0x7f12b81a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b81045f0 0x7f12b81a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 -- 172.21.15.5:0/2875436037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b8109870 msgr2=0x7f12b819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b8109870 0x7f12b819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.316+0000 7f12bdd28700 1 -- 172.21.15.5:0/2875436037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b819b080 con 0x7f12b8108f20 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.317+0000 7f12bdd28700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 0x7f12b81a0d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c9e0 tx=0x7f12a800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.317+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8007aa0 con 0x7f12b8108f20 2024-05-14T15:21:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.317+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8007c40 con 0x7f12b8108f20 2024-05-14T15:21:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.317+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b819b370 con 0x7f12b8108f20 2024-05-14T15:21:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.317+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800fe60 con 0x7f12b8108f20 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.317+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a76b0 con 0x7f12b8108f20 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.318+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b80512c0 con 0x7f12b8108f20 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.320+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f12a800f450 con 0x7f12b8108f20 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.320+0000 7f12af7fe700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12a40b2880 0x7f12a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.321+0000 7f12be529700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12a40b2880 0x7f12a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.321+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f12a80e2680 con 0x7f12b8108f20 2024-05-14T15:21:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.321+0000 7f12be529700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12a40b2880 0x7f12a40b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12b4003800 tx=0x7f12b401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.324+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12a8014030 con 0x7f12b8108f20 2024-05-14T15:21:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.594+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12b81a7880 con 0x7f12b8108f20 2024-05-14T15:21:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.595+0000 7f12af7fe700 1 -- 172.21.15.5:0/2875436037 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11791 (secure 0 0 0) 0x7f12a80a5070 con 0x7f12b8108f20 2024-05-14T15:21:24.595 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:24.595 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:21:12.490627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24515,"qdb_cluster":[24515]},"id":1}]} 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12a40b2880 msgr2=0x7f12a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12a40b2880 0x7f12a40b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12b4003800 tx=0x7f12b401a040 comp rx=0 tx=0).stop 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 msgr2=0x7f12b81a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 0x7f12b81a0d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c9e0 tx=0x7f12a800ccf0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 shutdown_connections 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12a40b2880 0x7f12a40b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b8109870 0x7f12b819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b81045f0 0x7f12b81a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 --2- 172.21.15.5:0/2875436037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b8108f20 0x7f12b81a0d60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 >> 172.21.15.5:0/2875436037 conn(0x7f12b8100040 msgr2=0x7f12b8106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 shutdown_connections 2024-05-14T15:21:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:24.597+0000 7f12c078d700 1 -- 172.21.15.5:0/2875436037 wait complete. 2024-05-14T15:21:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-05-14T15:21:24.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:24 smithi144 ceph-mon[35650]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:24.855 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:24 smithi193 ceph-mon[36001]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:24.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:24 smithi005 ceph-mon[31559]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:25.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:21:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:25.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2875436037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:21:26.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2875436037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:21:26.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2875436037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:21:26.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:26 smithi005 ceph-mon[31559]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:27.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:26 smithi144 ceph-mon[35650]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:27.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:26 smithi193 ceph-mon[36001]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 -- 172.21.15.5:0/1000374864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 msgr2=0x7f8f38113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 --2- 172.21.15.5:0/1000374864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f38113050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b0a0 tx=0x7f8f3000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 -- 172.21.15.5:0/1000374864 shutdown_connections 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 --2- 172.21.15.5:0/1000374864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f38113050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 --2- 172.21.15.5:0/1000374864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f38103080 0x7f8f38103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 --2- 172.21.15.5:0/1000374864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 0x7f8f381106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 -- 172.21.15.5:0/1000374864 >> 172.21.15.5:0/1000374864 conn(0x7f8f380fedf0 msgr2=0x7f8f38101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 -- 172.21.15.5:0/1000374864 shutdown_connections 2024-05-14T15:21:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.407+0000 7f8f3c932700 1 -- 172.21.15.5:0/1000374864 wait complete. 2024-05-14T15:21:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 Processor -- start 2024-05-14T15:21:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 -- start start 2024-05-14T15:21:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f38103080 0x7f8f3819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 0x7f8f3819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f381a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f381166b0 con 0x7f8f38110c30 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f38116530 con 0x7f8f38103080 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.408+0000 7f8f3c932700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f38116830 con 0x7f8f381039d0 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f381a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f381a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57676/0 (socket says 172.21.15.5:57676) 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 -- 172.21.15.5:0/734830522 learned_addr learned my addr 172.21.15.5:0/734830522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f3659c700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f38103080 0x7f8f3819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f35d9b700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 0x7f8f3819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 -- 172.21.15.5:0/734830522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 msgr2=0x7f8f3819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 0x7f8f3819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 -- 172.21.15.5:0/734830522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f38103080 msgr2=0x7f8f3819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f38103080 0x7f8f3819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 -- 172.21.15.5:0/734830522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f381a1e20 con 0x7f8f38110c30 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f35d9b700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 0x7f8f3819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f36d9d700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f381a1830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b830 tx=0x7f8f30004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30021070 con 0x7f8f38110c30 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f381a20b0 con 0x7f8f38110c30 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f30007e30 con 0x7f8f38110c30 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f300047d0 con 0x7f8f38110c30 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.409+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f381a2640 con 0x7f8f38110c30 2024-05-14T15:21:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.410+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f380512c0 con 0x7f8f38110c30 2024-05-14T15:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.412+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8f30007980 con 0x7f8f38110c30 2024-05-14T15:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.413+0000 7f8f2f7fe700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8f240b2880 0x7f8f240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.413+0000 7f8f3659c700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8f240b2880 0x7f8f240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.413+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f8f3000f020 con 0x7f8f38110c30 2024-05-14T15:21:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.413+0000 7f8f3659c700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8f240b2880 0x7f8f240b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005950 tx=0x7f8f2800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.416+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f3002a020 con 0x7f8f38110c30 2024-05-14T15:21:27.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.634+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f381a9ae0 con 0x7f8f38110c30 2024-05-14T15:21:27.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.636+0000 7f8f2f7fe700 1 -- 172.21.15.5:0/734830522 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24084 (secure 0 0 0) 0x7f8f300ab960 con 0x7f8f38110c30 2024-05-14T15:21:27.636 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:27.636 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:12.490470+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:21:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8f240b2880 msgr2=0x7f8f240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8f240b2880 0x7f8f240b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005950 tx=0x7f8f2800b410 comp rx=0 tx=0).stop 2024-05-14T15:21:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 msgr2=0x7f8f381a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f381a1830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b830 tx=0x7f8f30004ce0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 shutdown_connections 2024-05-14T15:21:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8f240b2880 0x7f8f240b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f38110c30 0x7f8f381a1830 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f38103080 0x7f8f3819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 --2- 172.21.15.5:0/734830522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f381039d0 0x7f8f3819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:27.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 >> 172.21.15.5:0/734830522 conn(0x7f8f380fedf0 msgr2=0x7f8f381096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:27.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 shutdown_connections 2024-05-14T15:21:27.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:27.638+0000 7f8f3c932700 1 -- 172.21.15.5:0/734830522 wait complete. 2024-05-14T15:21:28.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-14T15:20:57.810761+0000', 'modified': '2024-05-14T15:21:12.490627+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24515}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6839/4183436774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 4183436774}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 4183436774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24515, 'qdb_cluster': [24515]} 2024-05-14T15:21:28.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-14T15:21:28.396 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T15:21:28.661 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:28.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:28 smithi005 ceph-mon[31559]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:28.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:28 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/734830522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:29.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:28 smithi144 ceph-mon[35650]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:29.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:28 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/734830522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:29.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:28 smithi193 ceph-mon[36001]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:29.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:28 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/734830522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 -- 172.21.15.5:0/2336693137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c107df0 msgr2=0x7fb46c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 --2- 172.21.15.5:0/2336693137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c107df0 0x7fb46c108200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0099c0 tx=0x7fb45c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 -- 172.21.15.5:0/2336693137 shutdown_connections 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 --2- 172.21.15.5:0/2336693137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c108740 0x7fb46c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 --2- 172.21.15.5:0/2336693137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c1034c0 0x7fb46c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 --2- 172.21.15.5:0/2336693137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c107df0 0x7fb46c108200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 -- 172.21.15.5:0/2336693137 >> 172.21.15.5:0/2336693137 conn(0x7fb46c0fedc0 msgr2=0x7fb46c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.426+0000 7fb4737cc700 1 -- 172.21.15.5:0/2336693137 shutdown_connections 2024-05-14T15:21:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.427+0000 7fb4737cc700 1 -- 172.21.15.5:0/2336693137 wait complete. 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.427+0000 7fb4737cc700 1 Processor -- start 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.427+0000 7fb4737cc700 1 -- start start 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb4737cc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 0x7fb46c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb4737cc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c107df0 0x7fb46c19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb4737cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c108740 0x7fb46c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb4737cc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c113350 con 0x7fb46c1034c0 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb4737cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1131d0 con 0x7fb46c108740 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb4737cc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1134d0 con 0x7fb46c107df0 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 0x7fb46c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 0x7fb46c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57706/0 (socket says 172.21.15.5:57706) 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 -- 172.21.15.5:0/899851158 learned_addr learned my addr 172.21.15.5:0/899851158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb470d67700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c107df0 0x7fb46c19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471d69700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c108740 0x7fb46c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 -- 172.21.15.5:0/899851158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c107df0 msgr2=0x7fb46c19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c107df0 0x7fb46c19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 -- 172.21.15.5:0/899851158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c108740 msgr2=0x7fb46c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c108740 0x7fb46c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 -- 172.21.15.5:0/899851158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1a6f10 con 0x7fb46c1034c0 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471d69700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c108740 0x7fb46c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.428+0000 7fb471568700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 0x7fb46c19bc30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c9b0 tx=0x7fb46800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.429+0000 7fb4627fc700 1 -- 172.21.15.5:0/899851158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468007b00 con 0x7fb46c1034c0 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.429+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c1a7200 con 0x7fb46c1034c0 2024-05-14T15:21:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.429+0000 7fb4627fc700 1 -- 172.21.15.5:0/899851158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb468007ca0 con 0x7fb46c1034c0 2024-05-14T15:21:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.429+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1a7710 con 0x7fb46c1034c0 2024-05-14T15:21:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.429+0000 7fb4627fc700 1 -- 172.21.15.5:0/899851158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468018660 con 0x7fb46c1034c0 2024-05-14T15:21:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.433+0000 7fb4627fc700 1 -- 172.21.15.5:0/899851158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb46800f450 con 0x7fb46c1034c0 2024-05-14T15:21:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.433+0000 7fb4821e0b80 1 Processor -- start 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.433+0000 7fb4821e0b80 1 -- start start 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4627fc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4580b6c90 0x7fb4580b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb471d69700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4580b6c90 0x7fb4580b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4627fc700 1 -- 172.21.15.5:0/899851158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fb4680e2430 con 0x7fb46c1034c0 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x56244187e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x562441878880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x5624418791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56244179a8f0 con 0x56244187ed30 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5624416ffd50 con 0x562441878dc0 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5624417968b0 con 0x5624416fadf0 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x562441878880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x562441878880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57712/0 (socket says 172.21.15.5:57712) 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 -- 172.21.15.5:0/503296002 learned_addr learned my addr 172.21.15.5:0/503296002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb471568700 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x56244187e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb471d69700 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x5624418791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 -- 172.21.15.5:0/503296002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 msgr2=0x56244187e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x56244187e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb471d69700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4580b6c90 0x7fb4580b91a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb46c06c6f0 tx=0x7fb4640077e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 -- 172.21.15.5:0/503296002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 msgr2=0x5624418791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x5624418791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb470d67700 1 -- 172.21.15.5:0/503296002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624416e7880 con 0x56244187ed30 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.434+0000 7fb471568700 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x56244187e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.435+0000 7fb470d67700 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x562441878880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0099c0 tx=0x7fb45c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f2f177b3cbf6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.435+0000 7fb4577fe700 1 -- 172.21.15.5:0/503296002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003ae0 con 0x56244187ed30 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.435+0000 7fb4577fe700 1 -- 172.21.15.5:0/503296002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c005580 con 0x56244187ed30 2024-05-14T15:21:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.435+0000 7fb4577fe700 1 -- 172.21.15.5:0/503296002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00bcd0 con 0x56244187ed30 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 -- 172.21.15.5:0/503296002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 msgr2=0x562441878880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x562441878880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0099c0 tx=0x7fb45c009710 comp rx=0 tx=0).stop 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 -- 172.21.15.5:0/503296002 shutdown_connections 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x5624418791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x56244187e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/503296002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x562441878880 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 -- 172.21.15.5:0/503296002 >> 172.21.15.5:0/503296002 conn(0x5624417425e0 msgr2=0x5624418739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 -- 172.21.15.5:0/503296002 shutdown_connections 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 -- 172.21.15.5:0/503296002 wait complete. 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 -- start start 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.436+0000 7fb4821e0b80 1 Processor -- start 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x5624418823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441882900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56244187ed30 0x562441882e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562441885710 con 0x5624416fadf0 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562441753e80 con 0x562441878dc0 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562441885890 con 0x56244187ed30 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb4821e0b80 10 client.0 _ll_register_callbacks cb 0x562441732900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x5624418823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x5624418823c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57726/0 (socket says 172.21.15.5:57726) 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 -- 172.21.15.5:0/4284394743 learned_addr learned my addr 172.21.15.5:0/4284394743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:30.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb470d67700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441882900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471d69700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56244187ed30 0x562441882e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 -- 172.21.15.5:0/4284394743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56244187ed30 msgr2=0x562441882e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56244187ed30 0x562441882e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 -- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 msgr2=0x562441882900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441882900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.437+0000 7fb471568700 1 -- 172.21.15.5:0/4284394743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562441751710 con 0x5624416fadf0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb471568700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x5624418823c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb4680a9280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4145e454220c45d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb454ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb454ff9700 1 -- 172.21.15.5:0/4284394743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680a9f10 con 0x5624416fadf0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb454ff9700 1 -- 172.21.15.5:0/4284394743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fb4680a9c80 con 0x5624416fadf0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45c005580 con 0x5624416fadf0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb454ff9700 1 -- 172.21.15.5:0/4284394743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fb4680a9810 con 0x5624416fadf0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb4821e0b80 10 client.14718 fetch_fsmap learned FSMap version 16 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5624418902f0 con 0x5624416fadf0 2024-05-14T15:21:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb454ff9700 1 -- 172.21.15.5:0/4284394743 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c005580 con 0x5624416fadf0 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb454ff9700 1 -- 172.21.15.5:0/4284394743 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7fb4680b2270 con 0x5624416fadf0 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb4821e0b80 10 client.14718 fetch_fsmap finished waiting for FSMap version 16 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb4821e0b80 10 client.14718 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.438+0000 7fb4821e0b80 20 client.14718 populate_metadata read hostname 'smithi005' 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.439+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418928f0 0x562441893d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.439+0000 7fb4821e0b80 4 client.14718 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.439+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624417a82d0 con 0x5624418928f0 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.439+0000 7fb470d67700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418928f0 0x562441893d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.439+0000 7fb470d67700 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418928f0 0x562441893d60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769f63e4da32d11e server_cookie=136daffbbec3286d in_seq=0 out_seq=0 2024-05-14T15:21:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.439+0000 7fb454ff9700 10 client.14718 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:21:30.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb454ff9700 1 -- 172.21.15.5:0/4284394743 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624417a82d0 con 0x5624418928f0 2024-05-14T15:21:30.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb454ff9700 10 client.14718 handle_command_reply: tid=0 2024-05-14T15:21:30.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 client.14718 shutdown 2024-05-14T15:21:30.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 msgr2=0x5624418823c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x5624418823c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb4680a9280 comp rx=0 tx=0).stop 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 shutdown_connections 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418928f0 0x562441893d60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56244187ed30 0x562441882e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x5624418823c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/4284394743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441882900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 >> 172.21.15.5:0/4284394743 conn(0x5624417425e0 msgr2=0x562441877870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb454ff9700 0 client.14718 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.440+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 shutdown_connections 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.441+0000 7fb4821e0b80 1 -- 172.21.15.5:0/4284394743 wait complete. 2024-05-14T15:21:30.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.441+0000 7fb4821e0b80 20 client.14718 trim_cache size 0 max 16384 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 Processor -- start 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 -- start start 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x562441878880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562441878dc0 0x562441893d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56244187ed30 0x56244188eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5624418920a0 con 0x5624416fadf0 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562441892770 con 0x56244187ed30 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562441892e40 con 0x562441878dc0 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x562441878880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x562441878880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57740/0 (socket says 172.21.15.5:57740) 2024-05-14T15:21:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 -- 172.21.15.5:0/3996771480 learned_addr learned my addr 172.21.15.5:0/3996771480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb470d67700 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562441878dc0 0x562441893d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471d69700 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56244187ed30 0x56244188eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 -- 172.21.15.5:0/3996771480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562441878dc0 msgr2=0x562441893d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562441878dc0 0x562441893d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 -- 172.21.15.5:0/3996771480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56244187ed30 msgr2=0x56244188eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56244187ed30 0x56244188eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471568700 1 -- 172.21.15.5:0/3996771480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56244187e350 con 0x5624416fadf0 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.462+0000 7fb471d69700 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56244187ed30 0x56244188eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.463+0000 7fb471568700 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x562441878880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4680a5890 tx=0x7fb46800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135a67f9db5751c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.463+0000 7fb456ffd700 1 -- 172.21.15.5:0/3996771480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468087070 con 0x5624416fadf0 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.463+0000 7fb456ffd700 1 -- 172.21.15.5:0/3996771480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680adbc0 con 0x5624416fadf0 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.463+0000 7fb456ffd700 1 -- 172.21.15.5:0/3996771480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680c6070 con 0x5624416fadf0 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 -- 172.21.15.5:0/3996771480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 msgr2=0x562441878880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x562441878880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb4680a5890 tx=0x7fb46800fc60 comp rx=0 tx=0).stop 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 -- 172.21.15.5:0/3996771480 shutdown_connections 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56244187ed30 0x56244188eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624416fadf0 0x562441878880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/3996771480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562441878dc0 0x562441893d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 -- 172.21.15.5:0/3996771480 >> 172.21.15.5:0/3996771480 conn(0x5624417425e0 msgr2=0x5624418791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 -- 172.21.15.5:0/3996771480 shutdown_connections 2024-05-14T15:21:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 -- 172.21.15.5:0/3996771480 wait complete. 2024-05-14T15:21:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 -- start start 2024-05-14T15:21:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.464+0000 7fb4821e0b80 1 Processor -- start 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x562441877870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441877db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x5624418782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562441893be0 con 0x56244187ed30 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562441893510 con 0x562441878dc0 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5624418942b0 con 0x5624416fadf0 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471d69700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x5624418782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb4821e0b80 10 client.0 _ll_register_callbacks cb 0x562441732900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471d69700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x5624418782f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57752/0 (socket says 172.21.15.5:57752) 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471568700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x562441877870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471d69700 1 -- 172.21.15.5:0/1183778822 learned_addr learned my addr 172.21.15.5:0/1183778822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb470d67700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441877db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471568700 1 -- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 msgr2=0x562441877db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471568700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441877db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471568700 1 -- 172.21.15.5:0/1183778822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 msgr2=0x5624418782f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471568700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x5624418782f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471568700 1 -- 172.21.15.5:0/1183778822 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244189c820 con 0x5624416fadf0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb470d67700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441877db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.465+0000 7fb471d69700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x5624418782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb471568700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x562441877870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb46800afd0 tx=0x7fb4680e22b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5485cbc6e6311b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb454ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb454ff9700 1 -- 172.21.15.5:0/1183778822 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468087030 con 0x5624416fadf0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4680adbc0 con 0x5624416fadf0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb454ff9700 1 -- 172.21.15.5:0/1183778822 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fb4680a8b40 con 0x5624416fadf0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb454ff9700 1 -- 172.21.15.5:0/1183778822 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fb4680c6070 con 0x5624416fadf0 2024-05-14T15:21:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 10 client.24574 fetch_fsmap learned FSMap version 16 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x56244189d9f0 con 0x5624416fadf0 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb454ff9700 1 -- 172.21.15.5:0/1183778822 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680adbc0 con 0x5624416fadf0 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb454ff9700 1 -- 172.21.15.5:0/1183778822 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7fb4680caac0 con 0x5624416fadf0 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 10 client.24574 fetch_fsmap finished waiting for FSMap version 16 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 10 client.24574 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 20 client.24574 populate_metadata read hostname 'smithi005' 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418ff730 0x562441900ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 4 client.24574 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.466+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "status"}) -- 0x5624417a2370 con 0x5624418ff730 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb470d67700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418ff730 0x562441900ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb470d67700 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418ff730 0x562441900ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78597c36946f8ef2 server_cookie=36657812a5c6e7f1 in_seq=0 out_seq=0 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb454ff9700 10 client.24574 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb454ff9700 1 -- 172.21.15.5:0/1183778822 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5624417a2370 con 0x5624418ff730 2024-05-14T15:21:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb454ff9700 10 client.24574 handle_command_reply: tid=0 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb4821e0b80 1 client.24574 shutdown 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 msgr2=0x562441877870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x562441877870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb46800afd0 tx=0x7fb4680e22b0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.467+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 shutdown_connections 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5624418ff730 0x562441900ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56244187ed30 0x5624418782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624416fadf0 0x562441877870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 --2- 172.21.15.5:0/1183778822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562441878dc0 0x562441877db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb454ff9700 0 client.24574 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 >> 172.21.15.5:0/1183778822 conn(0x5624417425e0 msgr2=0x5624418791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 shutdown_connections 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 1 -- 172.21.15.5:0/1183778822 wait complete. 2024-05-14T15:21:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4821e0b80 20 client.24574 trim_cache size 0 max 16384 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24515, 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 32.548977415000003, 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 16, 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 86, 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 50.099981051 2024-05-14T15:21:30.473 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4580b6c90 msgr2=0x7fb4580b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4580b6c90 0x7fb4580b91a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb46c06c6f0 tx=0x7fb4640077e0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 msgr2=0x7fb46c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 0x7fb46c19bc30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c9b0 tx=0x7fb46800ccc0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 shutdown_connections 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb4580b6c90 0x7fb4580b91a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46c108740 0x7fb46c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c1034c0 0x7fb46c19bc30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 --2- 172.21.15.5:0/899851158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46c107df0 0x7fb46c19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 >> 172.21.15.5:0/899851158 conn(0x7fb46c0fedc0 msgr2=0x7fb46c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.468+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 shutdown_connections 2024-05-14T15:21:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:30.469+0000 7fb4737cc700 1 -- 172.21.15.5:0/899851158 wait complete. 2024-05-14T15:21:30.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:30 smithi005 ceph-mon[31559]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:31.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:30 smithi144 ceph-mon[35650]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:31.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:30 smithi193 ceph-mon[36001]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:31.283 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T15:21:31.283 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:31.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T15:21:31.535 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:32.885 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:32 smithi005 ceph-mon[31559]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:33.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:32 smithi144 ceph-mon[35650]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:33.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:32 smithi193 ceph-mon[36001]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:33.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 -- 172.21.15.5:0/2442831426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c107df0 msgr2=0x7f854c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:33.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 --2- 172.21.15.5:0/2442831426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c107df0 0x7f854c108200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f853c0099c0 tx=0x7f853c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 -- 172.21.15.5:0/2442831426 shutdown_connections 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 --2- 172.21.15.5:0/2442831426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 0x7f854c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 --2- 172.21.15.5:0/2442831426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f854c1034c0 0x7f854c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 --2- 172.21.15.5:0/2442831426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c107df0 0x7f854c108200 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 -- 172.21.15.5:0/2442831426 >> 172.21.15.5:0/2442831426 conn(0x7f854c0fedc0 msgr2=0x7f854c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 -- 172.21.15.5:0/2442831426 shutdown_connections 2024-05-14T15:21:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.268+0000 7f85522fc700 1 -- 172.21.15.5:0/2442831426 wait complete. 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 Processor -- start 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 -- start start 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 0x7f854c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f854c107df0 0x7f854c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 0x7f854c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f854c113350 con 0x7f854c1034c0 2024-05-14T15:21:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1131d0 con 0x7f854c108740 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.269+0000 7f85522fc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1134d0 con 0x7f854c107df0 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 0x7f854c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 0x7f854c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57770/0 (socket says 172.21.15.5:57770) 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 -- 172.21.15.5:0/953625565 learned_addr learned my addr 172.21.15.5:0/953625565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854b7fe700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f854c107df0 0x7f854c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f8550899700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 0x7f854c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 -- 172.21.15.5:0/953625565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f854c107df0 msgr2=0x7f854c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f854c107df0 0x7f854c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 -- 172.21.15.5:0/953625565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 msgr2=0x7f854c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 0x7f854c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 -- 172.21.15.5:0/953625565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19fcc0 con 0x7f854c1034c0 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f8550899700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 0x7f854c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:21:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f854bfff700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 0x7f854c19adf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f853400d980 tx=0x7f853400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534016070 con 0x7f854c1034c0 2024-05-14T15:21:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19ffb0 con 0x7f854c1034c0 2024-05-14T15:21:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8534004830 con 0x7f854c1034c0 2024-05-14T15:21:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340054c0 con 0x7f854c1034c0 2024-05-14T15:21:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.270+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a78d0 con 0x7f854c1034c0 2024-05-14T15:21:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.271+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c0512c0 con 0x7f854c1034c0 2024-05-14T15:21:33.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.273+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8534004220 con 0x7f854c1034c0 2024-05-14T15:21:33.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.274+0000 7f85497fa700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85380b2880 0x7f85380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:33.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.274+0000 7f854b7fe700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85380b2880 0x7f85380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:33.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.274+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f85340e27d0 con 0x7f854c1034c0 2024-05-14T15:21:33.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.274+0000 7f854b7fe700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85380b2880 0x7f85380b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f853c006010 tx=0x7f853c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.277+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f853401f020 con 0x7f854c1034c0 2024-05-14T15:21:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.539+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f854c02e5f0 con 0x7f854c1034c0 2024-05-14T15:21:33.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.540+0000 7f85497fa700 1 -- 172.21.15.5:0/953625565 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f85340aa250 con 0x7f854c1034c0 2024-05-14T15:21:33.540 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T15:21:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.542+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85380b2880 msgr2=0x7f85380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.542+0000 7f85522fc700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85380b2880 0x7f85380b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f853c006010 tx=0x7f853c000bc0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.542+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 msgr2=0x7f854c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.542+0000 7f85522fc700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 0x7f854c19adf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f853400d980 tx=0x7f853400dc90 comp rx=0 tx=0).stop 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 shutdown_connections 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85380b2880 0x7f85380b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f854c108740 0x7f854c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854c1034c0 0x7f854c19adf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 --2- 172.21.15.5:0/953625565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f854c107df0 0x7f854c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 >> 172.21.15.5:0/953625565 conn(0x7f854c0fedc0 msgr2=0x7f854c06b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 shutdown_connections 2024-05-14T15:21:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:33.543+0000 7f85522fc700 1 -- 172.21.15.5:0/953625565 wait complete. 2024-05-14T15:21:33.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:33 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/953625565' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T15:21:34.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:33 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/953625565' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T15:21:34.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:33 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/953625565' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T15:21:34.258 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T15:21:34.259 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-14T15:21:34.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:34.259 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T15:21:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-14T15:21:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-14T15:21:34.281 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T15:21:34.281 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:21:34.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:21:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:34.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:34.485 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T15:21:34.512 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-14T15:21:34.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:34.513 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-14T15:21:34.588 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-14T15:21:34.589 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T15:21:34.589 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-14T15:21:34.589 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-14T15:21:34.589 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T15:21:34.589 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:21:34.589 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:21:34.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-14T15:21:34.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-14T15:21:34.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-14T15:21:34.590 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:21:34.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:21:34.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:34 smithi005 ceph-mon[31559]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:34.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:34.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:34.929 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-14T15:21:34.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:34.946 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-14T15:21:35.001 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:21:35.001 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:21:35.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T15:21:35.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-14T15:21:35.002 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:21:35.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:34 smithi144 ceph-mon[35650]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:35.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:34 smithi193 ceph-mon[36001]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:35.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:21:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:35.124 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-14T15:21:35.124 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:21:35.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:21:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:35.256 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:21:35.256 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:21:35.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-14T15:21:35.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T15:21:35.256 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:21:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:21:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:35.336 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-14T15:21:35.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T15:21:35.337 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:35.352 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-14T15:21:35.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T15:21:35.353 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:35.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-05-14T15:21:35.475 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T15:21:35.532 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-05-14T15:21:35.532 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-05-14T15:21:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-05-14T15:21:35.534 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/aa78e767aa34f6885a0585503ebdf4ecfb64706c10ddc08cad5d4adfc8b7f14f/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6f1965a38d0ba642c7109a54ae7b125347ebd3341a05be817d07a10e2dba97d7/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/20aa1db8b63736ec8db45403c94218d4d15129f4bcfa7416debbf81a2476055d/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/41d6ea8dbb8f6373fc6c7da8c55b0c4dccfee479674cd41bf1150e1987538403/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/75371eb09afe4d8cbc1ae7de2d96d8cec7d97d850c353fb59733dd4a3e6bc782/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/42e81d88822b707b81f7bf14297df818f909ff32b3adb867518fbfc1304e7da1/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6cff9938832b60542e8e36b70f0d4e9a96343b92a3b7926085db26edfb1e9029/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/aa7bce5051af6445b184021d889d0d280f027a2e6a41950e9c00b8d8fca9813c/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b2b9f0547288b2813fa4a470d2db8099048d617616e3bd398b5d6d42922caf08/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f9786a1ca46fe2d67959773ebeda261d01d44a9fed573b4e9e1f1494fc820a2f/merged 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T15:21:35.535 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T15:21:35.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:35.536 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-05-14T15:21:35.589 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-14T15:21:35.589 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-14T15:21:35.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-05-14T15:21:35.657 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-05-14T15:21:35.714 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T15:21:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/aa78e767aa34f6885a0585503ebdf4ecfb64706c10ddc08cad5d4adfc8b7f14f/merged 2024-05-14T15:21:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6f1965a38d0ba642c7109a54ae7b125347ebd3341a05be817d07a10e2dba97d7/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/20aa1db8b63736ec8db45403c94218d4d15129f4bcfa7416debbf81a2476055d/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/41d6ea8dbb8f6373fc6c7da8c55b0c4dccfee479674cd41bf1150e1987538403/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/75371eb09afe4d8cbc1ae7de2d96d8cec7d97d850c353fb59733dd4a3e6bc782/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/42e81d88822b707b81f7bf14297df818f909ff32b3adb867518fbfc1304e7da1/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/6cff9938832b60542e8e36b70f0d4e9a96343b92a3b7926085db26edfb1e9029/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/aa7bce5051af6445b184021d889d0d280f027a2e6a41950e9c00b8d8fca9813c/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b2b9f0547288b2813fa4a470d2db8099048d617616e3bd398b5d6d42922caf08/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/f9786a1ca46fe2d67959773ebeda261d01d44a9fed573b4e9e1f1494fc820a2f/merged 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T15:21:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T15:21:35.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:35.718 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-05-14T15:21:35.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:2024-05-14T15:21:35.752+0000 7fe89800e480 -1 init, newargv = 0x7fe884004720 newargc=15 2024-05-14T15:21:35.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[77966]: starting ceph client 2024-05-14T15:21:35.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[77966]: starting fuse 2024-05-14T15:21:35.769 INFO:teuthology.orchestra.run.smithi005.stdout:119 2024-05-14T15:21:35.770 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-14T15:21:35.770 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-05-14T15:21:35.771 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-05-14T15:21:35.772 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-14T15:21:35.772 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-14T15:21:35.772 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.admin")) 2024-05-14T15:21:35.772 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-14T15:21:35.891 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.admin.77966.asok 2024-05-14T15:21:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:35.902 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.77966.asok 2024-05-14T15:21:35.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:35.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.77966.asok status 2024-05-14T15:21:36.049 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:21:36.049 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "e2b755a64f0d72176b57299d02e880cf0620d780", 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-3588-ge2b755a6 (e2b755a64f0d72176b57299d02e880cf0620d780) squid (dev)", 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "admin", 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "77966", 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 14736, 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-05-14T15:21:36.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 14736 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4064814608 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-05-14T15:21:36.051 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4064814608 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.14736 192.168.0.1:0/4064814608", 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/4064814608", 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 16, 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 86, 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-05-14T15:21:36.052 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:21:36.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:36.058 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-05-14T15:21:36.058 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:36.075 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-05-14T15:21:36.075 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:36.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:36.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:36.142 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-14T15:21:36.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T15:21:36.257 INFO:teuthology.orchestra.run.smithi005.stdout:# file: . 2024-05-14T15:21:36.257 INFO:teuthology.orchestra.run.smithi005.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-14T15:21:36.257 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:21:36.262 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:21:36.262 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T15:21:36.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T15:21:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:36.946 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:36 smithi005 ceph-mon[31559]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:37.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:36 smithi144 ceph-mon[35650]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:37.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:36 smithi193 ceph-mon[36001]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2680360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 msgr2=0x7fd9581038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2680360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd9581038d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd948009990 tx=0x7fd948009ca0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2680360420 shutdown_connections 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2680360420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd958108740 0x7fd9581100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2680360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd9581038d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2680360420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd958107df0 0x7fd958108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2680360420 >> 172.21.15.5:0/2680360420 conn(0x7fd9580fedf0 msgr2=0x7fd958101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2680360420 shutdown_connections 2024-05-14T15:21:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.204+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2680360420 wait complete. 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.205+0000 7fd95d1d5700 1 Processor -- start 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 -- start start 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd95819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd958107df0 0x7fd95819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd958108740 0x7fd95819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9581144d0 con 0x7fd9581034c0 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd958114350 con 0x7fd958108740 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95d1d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd958114650 con 0x7fd958107df0 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd95819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd95819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38190/0 (socket says 172.21.15.5:38190) 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 -- 172.21.15.5:0/2618389559 learned_addr learned my addr 172.21.15.5:0/2618389559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95659c700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd958107df0 0x7fd95819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd95759e700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd958108740 0x7fd95819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 -- 172.21.15.5:0/2618389559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd958107df0 msgr2=0x7fd95819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd958107df0 0x7fd95819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 -- 172.21.15.5:0/2618389559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd958108740 msgr2=0x7fd95819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd958108740 0x7fd95819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.206+0000 7fd956d9d700 1 -- 172.21.15.5:0/2618389559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95819fca0 con 0x7fd9581034c0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.207+0000 7fd956d9d700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd95819ae80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd948000c00 tx=0x7fd9480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.207+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948027070 con 0x7fd9581034c0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.207+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd948003f20 con 0x7fd9581034c0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.207+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95819ff30 con 0x7fd9581034c0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.207+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948022ca0 con 0x7fd9581034c0 2024-05-14T15:21:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.207+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9581a0410 con 0x7fd9581034c0 2024-05-14T15:21:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.208+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9580512c0 con 0x7fd9581034c0 2024-05-14T15:21:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.210+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd94802a070 con 0x7fd9581034c0 2024-05-14T15:21:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.210+0000 7fd93ffff700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd9400b2880 0x7fd9400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.211+0000 7fd95659c700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd9400b2880 0x7fd9400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.211+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd9480e38e0 con 0x7fd9581034c0 2024-05-14T15:21:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.211+0000 7fd95659c700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd9400b2880 0x7fd9400b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9581063e0 tx=0x7fd94c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.214+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd948031020 con 0x7fd9581034c0 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.448+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd95802e5f0 con 0x7fd9581034c0 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.449+0000 7fd93ffff700 1 -- 172.21.15.5:0/2618389559 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+582 (secure 0 0 0) 0x7fd948013070 con 0x7fd9581034c0 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2154399460 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2401136202 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2075859388 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1587175256 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4018221162 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:38.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2401136202 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:38.451 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2357267352 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:38.451 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1647912546 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:38.451 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2154399460 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:38.451 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1991676154 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd9400b2880 msgr2=0x7fd9400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd9400b2880 0x7fd9400b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9581063e0 tx=0x7fd94c00a380 comp rx=0 tx=0).stop 2024-05-14T15:21:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 msgr2=0x7fd95819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:38.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd95819ae80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd948000c00 tx=0x7fd9480047b0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 shutdown_connections 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd9400b2880 0x7fd9400b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd958108740 0x7fd95819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9581034c0 0x7fd95819ae80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 --2- 172.21.15.5:0/2618389559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd958107df0 0x7fd95819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 >> 172.21.15.5:0/2618389559 conn(0x7fd9580fedf0 msgr2=0x7fd958101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 shutdown_connections 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:38.451+0000 7fd95d1d5700 1 -- 172.21.15.5:0/2618389559 wait complete. 2024-05-14T15:21:38.453 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-05-14T15:21:38.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:38 smithi005 ceph-mon[31559]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-14T15:21:38.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:38 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2618389559' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:21:39.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:38 smithi144 ceph-mon[35650]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-14T15:21:39.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:38 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2618389559' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:21:39.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:38 smithi193 ceph-mon[36001]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-14T15:21:39.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:38 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2618389559' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:21:39.148 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:21:39.148 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T15:21:39.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T15:21:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:40.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:40 smithi005 ceph-mon[31559]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-14T15:21:41.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:40 smithi144 ceph-mon[35650]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-14T15:21:41.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:40 smithi193 ceph-mon[36001]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-14T15:21:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 -- 172.21.15.5:0/3010357354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb01034c0 msgr2=0x7fabb01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 --2- 172.21.15.5:0/3010357354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb01034c0 0x7fabb01038d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faba00099c0 tx=0x7faba0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 -- 172.21.15.5:0/3010357354 shutdown_connections 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 --2- 172.21.15.5:0/3010357354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabb0108740 0x7fabb01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 --2- 172.21.15.5:0/3010357354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb01034c0 0x7fabb01038d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 --2- 172.21.15.5:0/3010357354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0107df0 0x7fabb0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 -- 172.21.15.5:0/3010357354 >> 172.21.15.5:0/3010357354 conn(0x7fabb00fedc0 msgr2=0x7fabb01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 -- 172.21.15.5:0/3010357354 shutdown_connections 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 -- 172.21.15.5:0/3010357354 wait complete. 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.174+0000 7fabb5861700 1 Processor -- start 2024-05-14T15:21:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 -- start start 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabb01034c0 0x7fabb019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 0x7fabb019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0108740 0x7fabb019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0113370 con 0x7fabb0107df0 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01131f0 con 0x7fabb0108740 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabb5861700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01134f0 con 0x7fabb01034c0 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabae7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 0x7fabb019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabae7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 0x7fabb019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38202/0 (socket says 172.21.15.5:38202) 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabae7fc700 1 -- 172.21.15.5:0/792284915 learned_addr learned my addr 172.21.15.5:0/792284915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabaeffd700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabb01034c0 0x7fabb019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.175+0000 7fabaf7fe700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0108740 0x7fabb019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabae7fc700 1 -- 172.21.15.5:0/792284915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabb01034c0 msgr2=0x7fabb019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabae7fc700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabb01034c0 0x7fabb019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabae7fc700 1 -- 172.21.15.5:0/792284915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0108740 msgr2=0x7fabb019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabae7fc700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0108740 0x7fabb019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabae7fc700 1 -- 172.21.15.5:0/792284915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb01a6fe0 con 0x7fabb0107df0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabaf7fe700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0108740 0x7fabb019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabae7fc700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 0x7fabb019c1d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faba400e970 tx=0x7faba400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4004a30 con 0x7fabb0107df0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba400cc90 con 0x7fabb0107df0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb01a72d0 con 0x7fabb0107df0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4005610 con 0x7fabb0107df0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.176+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01a7810 con 0x7fabb0107df0 2024-05-14T15:21:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.177+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb00512c0 con 0x7fabb0107df0 2024-05-14T15:21:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.179+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7faba400c7e0 con 0x7fabb0107df0 2024-05-14T15:21:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.180+0000 7fabb485f700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fab980b2880 0x7fab980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.180+0000 7fabaeffd700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fab980b2880 0x7fab980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.180+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7faba40e2860 con 0x7fabb0107df0 2024-05-14T15:21:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.180+0000 7fabaeffd700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fab980b2880 0x7fab980b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faba0000c00 tx=0x7faba001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.184+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faba4054df0 con 0x7fabb0107df0 2024-05-14T15:21:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.420+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fabb002e5f0 con 0x7fabb0107df0 2024-05-14T15:21:41.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.420+0000 7fabb485f700 1 -- 172.21.15.5:0/792284915 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+582 (secure 0 0 0) 0x7faba4091080 con 0x7fabb0107df0 2024-05-14T15:21:41.421 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2154399460 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2401136202 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2075859388 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1587175256 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4018221162 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2401136202 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2357267352 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1647912546 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2154399460 2024-05-15T15:13:41.367574+0000 2024-05-14T15:21:41.422 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1991676154 2024-05-15T15:13:13.551868+0000 2024-05-14T15:21:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fab980b2880 msgr2=0x7fab980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fab980b2880 0x7fab980b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faba0000c00 tx=0x7faba001a040 comp rx=0 tx=0).stop 2024-05-14T15:21:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 msgr2=0x7fabb019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 0x7fabb019c1d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faba400e970 tx=0x7faba400ec80 comp rx=0 tx=0).stop 2024-05-14T15:21:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 shutdown_connections 2024-05-14T15:21:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fab980b2880 0x7fab980b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb0108740 0x7fabb019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabb01034c0 0x7fabb019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 --2- 172.21.15.5:0/792284915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb0107df0 0x7fabb019c1d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 >> 172.21.15.5:0/792284915 conn(0x7fabb00fedc0 msgr2=0x7fabb01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 shutdown_connections 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:41.423+0000 7fabb5861700 1 -- 172.21.15.5:0/792284915 wait complete. 2024-05-14T15:21:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-05-14T15:21:41.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-14T15:21:41.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-05-14T15:21:41.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:41 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/792284915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-05-14T15:21:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:41 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/792284915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:21:42.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-05-14T15:21:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:41 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/792284915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:21:42.221 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-05-14T15:21:42.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:42.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-14T15:21:42.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[77966]: fuse finished with error 0 and tester_r 0 2024-05-14T15:21:42.249 INFO:teuthology.orchestra.run:waiting for 300 2024-05-14T15:21:42.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:21:42.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:21:42.333 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-14T15:21:42.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:21:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-14T15:21:42.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-14T15:21:42.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-14T15:21:42.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-05-14T15:21:42.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-05-14T15:21:42.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:42 smithi005 ceph-mon[31559]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T15:21:43.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-05-14T15:21:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:42 smithi144 ceph-mon[35650]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-05-14T15:21:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:42 smithi193 ceph-mon[36001]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T15:21:43.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:43 smithi005 ceph-mon[31559]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T15:21:44.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:43 smithi144 ceph-mon[35650]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T15:21:44.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:43 smithi193 ceph-mon[36001]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 -- 172.21.15.5:0/501342156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 msgr2=0x7f51c8113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 --2- 172.21.15.5:0/501342156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c8113070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f51c000b0a0 tx=0x7f51c000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 -- 172.21.15.5:0/501342156 shutdown_connections 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 --2- 172.21.15.5:0/501342156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c8113070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 --2- 172.21.15.5:0/501342156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 0x7f51c81034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 --2- 172.21.15.5:0/501342156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 0x7f51c8110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 -- 172.21.15.5:0/501342156 >> 172.21.15.5:0/501342156 conn(0x7f51c80fef30 msgr2=0x7f51c8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.349+0000 7f51cfd63700 1 -- 172.21.15.5:0/501342156 shutdown_connections 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.350+0000 7f51cfd63700 1 -- 172.21.15.5:0/501342156 wait complete. 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.350+0000 7f51cfd63700 1 Processor -- start 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.350+0000 7f51cfd63700 1 -- start start 2024-05-14T15:21:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cfd63700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 0x7f51c819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cfd63700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 0x7f51c819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cfd63700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cfd63700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81166b0 con 0x7f51c8110c50 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cfd63700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8116530 con 0x7f51c81030a0 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cfd63700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8116830 con 0x7f51c8103a80 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c81a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38224/0 (socket says 172.21.15.5:38224) 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 -- 172.21.15.5:0/1055409668 learned_addr learned my addr 172.21.15.5:0/1055409668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cd2fe700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 0x7f51c819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cdaff700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 0x7f51c819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 -- 172.21.15.5:0/1055409668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 msgr2=0x7f51c819d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 0x7f51c819d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 -- 172.21.15.5:0/1055409668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 msgr2=0x7f51c819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 0x7f51c819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 -- 172.21.15.5:0/1055409668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c81a1dc0 con 0x7f51c8110c50 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cd2fe700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 0x7f51c819d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51cdaff700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 0x7f51c819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.351+0000 7f51ce300700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c81a17d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f51c00062a0 tx=0x7f51c0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.352+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0021070 con 0x7f51c8110c50 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.352+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c0007d00 con 0x7f51c8110c50 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.352+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c81a2050 con 0x7f51c8110c50 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.352+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c001d450 con 0x7f51c8110c50 2024-05-14T15:21:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.352+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81a2590 con 0x7f51c8110c50 2024-05-14T15:21:44.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.353+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c80512c0 con 0x7f51c8110c50 2024-05-14T15:21:44.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.355+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f51c0004020 con 0x7f51c8110c50 2024-05-14T15:21:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.356+0000 7f51beffd700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f51b40b2880 0x7f51b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.356+0000 7f51cdaff700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f51b40b2880 0x7f51b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.356+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f51c000f020 con 0x7f51c8110c50 2024-05-14T15:21:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.357+0000 7f51cdaff700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f51b40b2880 0x7f51b40b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.360+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51c002a020 con 0x7f51c8110c50 2024-05-14T15:21:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.574+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51c802e5f0 con 0x7f51c8110c50 2024-05-14T15:21:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.576+0000 7f51beffd700 1 -- 172.21.15.5:0/1055409668 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24528 (secure 0 0 0) 0x7f51c00ab480 con 0x7f51c8110c50 2024-05-14T15:21:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:44.577 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:42.610573+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:21:44.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f51b40b2880 msgr2=0x7f51b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f51b40b2880 0x7f51b40b4d90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005e20 comp rx=0 tx=0).stop 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 msgr2=0x7f51c81a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c81a17d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f51c00062a0 tx=0x7f51c0004ed0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 shutdown_connections 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f51b40b2880 0x7f51b40b4d90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8110c50 0x7f51c81a17d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.578+0000 7f51cfd63700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c81030a0 0x7f51c819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.579+0000 7f51cfd63700 1 --2- 172.21.15.5:0/1055409668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51c8103a80 0x7f51c819d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.579+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 >> 172.21.15.5:0/1055409668 conn(0x7f51c80fef30 msgr2=0x7f51c81125f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.579+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 shutdown_connections 2024-05-14T15:21:44.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:44.579+0000 7f51cfd63700 1 -- 172.21.15.5:0/1055409668 wait complete. 2024-05-14T15:21:44.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:44 smithi005 ceph-mon[31559]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-14T15:21:44.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:44 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1055409668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:45.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:44 smithi144 ceph-mon[35650]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-14T15:21:45.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:44 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1055409668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:45.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:44 smithi193 ceph-mon[36001]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-14T15:21:45.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:44 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1055409668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:45.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:21:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:46.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:46 smithi005 ceph-mon[31559]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-14T15:21:47.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:46 smithi144 ceph-mon[35650]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-14T15:21:47.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:46 smithi193 ceph-mon[36001]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-14T15:21:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 -- 172.21.15.5:0/1115977820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530107df0 msgr2=0x7f3530108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 --2- 172.21.15.5:0/1115977820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530107df0 0x7f3530108200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35240099c0 tx=0x7f3524009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 -- 172.21.15.5:0/1115977820 shutdown_connections 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 --2- 172.21.15.5:0/1115977820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530108740 0x7f35301100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 --2- 172.21.15.5:0/1115977820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35301034c0 0x7f35301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 --2- 172.21.15.5:0/1115977820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530107df0 0x7f3530108200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 -- 172.21.15.5:0/1115977820 >> 172.21.15.5:0/1115977820 conn(0x7f35300fedf0 msgr2=0x7f3530101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 -- 172.21.15.5:0/1115977820 shutdown_connections 2024-05-14T15:21:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.318+0000 7f3535eec700 1 -- 172.21.15.5:0/1115977820 wait complete. 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.319+0000 7f3535eec700 1 Processor -- start 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.319+0000 7f3535eec700 1 -- start start 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f3535eec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35301034c0 0x7f353019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f3535eec700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 0x7f353019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f3535eec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 0x7f3530199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f3535eec700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3530113350 con 0x7f3530108740 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f3535eec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35301131d0 con 0x7f35301034c0 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f3535eec700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35301134d0 con 0x7f3530107df0 2024-05-14T15:21:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 0x7f353019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 0x7f3530199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 0x7f353019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:55368/0 (socket says 172.21.15.5:55368) 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352effd700 1 -- 172.21.15.5:0/3575852957 learned_addr learned my addr 172.21.15.5:0/3575852957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 0x7f3530199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45992/0 (socket says 172.21.15.5:45992) 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352f7fe700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35301034c0 0x7f353019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 -- 172.21.15.5:0/3575852957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 msgr2=0x7f353019ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 0x7f353019ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 -- 172.21.15.5:0/3575852957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35301034c0 msgr2=0x7f353019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35301034c0 0x7f353019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352ffff700 1 -- 172.21.15.5:0/3575852957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353019a1f0 con 0x7f3530108740 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.320+0000 7f352effd700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 0x7f353019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.321+0000 7f352ffff700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 0x7f3530199b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3528007f10 tx=0x7f352800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.321+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004db0 con 0x7f3530108740 2024-05-14T15:21:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.321+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353019a4e0 con 0x7f3530108740 2024-05-14T15:21:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.321+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528009e20 con 0x7f3530108740 2024-05-14T15:21:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.321+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528015400 con 0x7f3530108740 2024-05-14T15:21:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.321+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35300759f0 con 0x7f3530108740 2024-05-14T15:21:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.322+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353019aa20 con 0x7f3530108740 2024-05-14T15:21:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.324+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3528005260 con 0x7f3530108740 2024-05-14T15:21:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.324+0000 7f352cff9700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f35180b6d10 0x7f35180b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.324+0000 7f352f7fe700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f35180b6d10 0x7f35180b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.325+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f35280e1dd0 con 0x7f3530108740 2024-05-14T15:21:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.325+0000 7f352f7fe700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f35180b6d10 0x7f35180b9220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f352000ac50 tx=0x7f352000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:47.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.329+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3528010070 con 0x7f3530108740 2024-05-14T15:21:47.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.543+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3530076420 con 0x7f3530108740 2024-05-14T15:21:47.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.545+0000 7f352cff9700 1 -- 172.21.15.5:0/3575852957 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24528 (secure 0 0 0) 0x7f35280e3090 con 0x7f3530108740 2024-05-14T15:21:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:21:47.545 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:42.610573+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:21:47.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f35180b6d10 msgr2=0x7f35180b9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:47.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f35180b6d10 0x7f35180b9220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f352000ac50 tx=0x7f352000a380 comp rx=0 tx=0).stop 2024-05-14T15:21:47.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 msgr2=0x7f3530199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:47.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 0x7f3530199b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3528007f10 tx=0x7f352800d3b0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 shutdown_connections 2024-05-14T15:21:47.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f35180b6d10 0x7f35180b9220 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3530108740 0x7f3530199b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35301034c0 0x7f353019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 --2- 172.21.15.5:0/3575852957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3530107df0 0x7f353019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 >> 172.21.15.5:0/3575852957 conn(0x7f35300fedf0 msgr2=0x7f35301096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 shutdown_connections 2024-05-14T15:21:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:47.547+0000 7f3535eec700 1 -- 172.21.15.5:0/3575852957 wait complete. 2024-05-14T15:21:47.958 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:47 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3575852957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:48.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:47 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3575852957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:48.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:47 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3575852957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:21:48.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-14T15:21:48.482 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:48.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:48 smithi005 ceph-mon[31559]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-05-14T15:21:49.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:48 smithi144 ceph-mon[35650]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-05-14T15:21:49.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:48 smithi193 ceph-mon[36001]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-05-14T15:21:49.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:49 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:21:50.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:49 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:21:50.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:49 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:21:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 -- 172.21.15.5:0/711065992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114107e20 msgr2=0x7f5114108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 --2- 172.21.15.5:0/711065992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114107e20 0x7f5114108230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f51080099c0 tx=0x7f5108009cd0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 -- 172.21.15.5:0/711065992 shutdown_connections 2024-05-14T15:21:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 --2- 172.21.15.5:0/711065992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 0x7f51141100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 --2- 172.21.15.5:0/711065992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5114103c80 0x7f5114104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 --2- 172.21.15.5:0/711065992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114107e20 0x7f5114108230 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 -- 172.21.15.5:0/711065992 >> 172.21.15.5:0/711065992 conn(0x7f51140fe6f0 msgr2=0x7f5114100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:50.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 -- 172.21.15.5:0/711065992 shutdown_connections 2024-05-14T15:21:50.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.213+0000 7f5118c4b700 1 -- 172.21.15.5:0/711065992 wait complete. 2024-05-14T15:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.214+0000 7f5118c4b700 1 Processor -- start 2024-05-14T15:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.214+0000 7f5118c4b700 1 -- start start 2024-05-14T15:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5118c4b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 0x7f511419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5118c4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5114107e20 0x7f511419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5118c4b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 0x7f511419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5118c4b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5114113390 con 0x7f5114103c80 2024-05-14T15:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5118c4b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5114113210 con 0x7f5114107e20 2024-05-14T15:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5118c4b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5114113510 con 0x7f5114108770 2024-05-14T15:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f511259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 0x7f511419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f511259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 0x7f511419bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46014/0 (socket says 172.21.15.5:46014) 2024-05-14T15:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f511259c700 1 -- 172.21.15.5:0/3073130064 learned_addr learned my addr 172.21.15.5:0/3073130064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.215+0000 7f5111d9b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5114107e20 0x7f511419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:50.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f511259c700 1 -- 172.21.15.5:0/3073130064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 msgr2=0x7f511419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:50.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f5112d9d700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 0x7f511419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:50.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f511259c700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 0x7f511419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f511259c700 1 -- 172.21.15.5:0/3073130064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5114107e20 msgr2=0x7f511419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f511259c700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5114107e20 0x7f511419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f511259c700 1 -- 172.21.15.5:0/3073130064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5114199b60 con 0x7f5114103c80 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f5112d9d700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 0x7f511419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f511259c700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 0x7f511419bd70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f510400d950 tx=0x7f510400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104016070 con 0x7f5114103c80 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5114199e50 con 0x7f5114103c80 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5104004830 con 0x7f5114103c80 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104005420 con 0x7f5114103c80 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.216+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511419a3b0 con 0x7f5114103c80 2024-05-14T15:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.217+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51140512c0 con 0x7f5114103c80 2024-05-14T15:21:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.219+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f51040055e0 con 0x7f5114103c80 2024-05-14T15:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.220+0000 7f51037fe700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f50fc0b2880 0x7f50fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.220+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f51040e27d0 con 0x7f5114103c80 2024-05-14T15:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.220+0000 7f5111d9b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f50fc0b2880 0x7f50fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.221+0000 7f5111d9b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f50fc0b2880 0x7f50fc0b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5108006010 tx=0x7f5108000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:50.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.224+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f510401f020 con 0x7f5114103c80 2024-05-14T15:21:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.500+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5114102ed0 con 0x7f5114103c80 2024-05-14T15:21:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.647+0000 7f51037fe700 1 -- 172.21.15.5:0/3073130064 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f51040aa1b0 con 0x7f5114103c80 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f50fc0b2880 msgr2=0x7f50fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f50fc0b2880 0x7f50fc0b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5108006010 tx=0x7f5108000bc0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 msgr2=0x7f511419bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 0x7f511419bd70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f510400d950 tx=0x7f510400dc60 comp rx=0 tx=0).stop 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 shutdown_connections 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f50fc0b2880 0x7f50fc0b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5114108770 0x7f511419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5114103c80 0x7f511419bd70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 --2- 172.21.15.5:0/3073130064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5114107e20 0x7f511419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 >> 172.21.15.5:0/3073130064 conn(0x7f51140fe6f0 msgr2=0x7f5114109680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 shutdown_connections 2024-05-14T15:21:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:50.650+0000 7f5118c4b700 1 -- 172.21.15.5:0/3073130064 wait complete. 2024-05-14T15:21:50.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:50 smithi005 ceph-mon[31559]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-05-14T15:21:50.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:50 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3073130064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T15:21:51.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:50 smithi144 ceph-mon[35650]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-05-14T15:21:51.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:50 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3073130064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T15:21:51.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:50 smithi193 ceph-mon[36001]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-05-14T15:21:51.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:50 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3073130064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T15:21:51.637 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-14T15:21:51.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-14T15:21:51.915 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:52.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:51 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3073130064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T15:21:52.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:51 smithi005 ceph-mon[31559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:52.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:51 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:51 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:51 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:51 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:51 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3073130064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T15:21:52.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:51 smithi144 ceph-mon[35650]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:52.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:51 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:51 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:51 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:51 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:51 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3073130064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T15:21:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:51 smithi193 ceph-mon[36001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:51 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:51 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:51 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:51 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:53.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:52 smithi005 ceph-mon[31559]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s 2024-05-14T15:21:53.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:52 smithi144 ceph-mon[35650]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s 2024-05-14T15:21:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:52 smithi193 ceph-mon[36001]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s 2024-05-14T15:21:53.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/4211225288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac102790 msgr2=0x7ff5ac102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/4211225288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac102790 0x7ff5ac102ba0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009990 tx=0x7ff5a0009ca0 comp rx=0 tx=0).stop 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/4211225288 shutdown_connections 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/4211225288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1030e0 0x7ff5ac111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/4211225288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac109300 0x7ff5ac109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/4211225288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac102790 0x7ff5ac102ba0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/4211225288 >> 172.21.15.5:0/4211225288 conn(0x7ff5ac0fe4e0 msgr2=0x7ff5ac100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/4211225288 shutdown_connections 2024-05-14T15:21:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.594+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/4211225288 wait complete. 2024-05-14T15:21:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.595+0000 7ff5b2bd3700 1 Processor -- start 2024-05-14T15:21:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.595+0000 7ff5b2bd3700 1 -- start start 2024-05-14T15:21:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.595+0000 7ff5b2bd3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac102790 0x7ff5ac19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.595+0000 7ff5b2bd3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1030e0 0x7ff5ac19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:53.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b2bd3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 0x7ff5ac19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b2bd3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1145a0 con 0x7ff5ac109300 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b2bd3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac114420 con 0x7ff5ac102790 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b2bd3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac114720 con 0x7ff5ac1030e0 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 0x7ff5ac19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 0x7ff5ac19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46034/0 (socket says 172.21.15.5:46034) 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 -- 172.21.15.5:0/3286227696 learned_addr learned my addr 172.21.15.5:0/3286227696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5abfff700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1030e0 0x7ff5ac19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b096f700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac102790 0x7ff5ac19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 -- 172.21.15.5:0/3286227696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1030e0 msgr2=0x7ff5ac19b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1030e0 0x7ff5ac19b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 -- 172.21.15.5:0/3286227696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac102790 msgr2=0x7ff5ac19af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac102790 0x7ff5ac19af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 -- 172.21.15.5:0/3286227696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac19fd70 con 0x7ff5ac109300 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.596+0000 7ff5b1170700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 0x7ff5ac19f780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4007ee0 tx=0x7ff5a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.597+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004bb0 con 0x7ff5ac109300 2024-05-14T15:21:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.597+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a4004d50 con 0x7ff5ac109300 2024-05-14T15:21:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.597+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac1a0060 con 0x7ff5ac109300 2024-05-14T15:21:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.597+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4005920 con 0x7ff5ac109300 2024-05-14T15:21:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.597+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1a05c0 con 0x7ff5ac109300 2024-05-14T15:21:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.598+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ac19c2b0 con 0x7ff5ac109300 2024-05-14T15:21:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.599+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff5a401a020 con 0x7ff5ac109300 2024-05-14T15:21:53.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.600+0000 7ff5a9ffb700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff5940b2900 0x7ff5940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:53.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.600+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff5a40e1b80 con 0x7ff5ac109300 2024-05-14T15:21:53.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.600+0000 7ff5b096f700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff5940b2900 0x7ff5940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:53.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.601+0000 7ff5b096f700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff5940b2900 0x7ff5940b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac06ca90 tx=0x7ff59c00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.605+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5a4010070 con 0x7ff5ac109300 2024-05-14T15:21:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:53.867+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff5ac02e7b0 con 0x7ff5ac109300 2024-05-14T15:21:53.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:53.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:53.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:21:53.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:21:53.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:53 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:21:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:21:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:53 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:21:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:21:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:53 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:21:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.742+0000 7ff5a9ffb700 1 -- 172.21.15.5:0/3286227696 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff5a40a9500 con 0x7ff5ac109300 2024-05-14T15:21:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff5940b2900 msgr2=0x7ff5940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff5940b2900 0x7ff5940b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac06ca90 tx=0x7ff59c00a3b0 comp rx=0 tx=0).stop 2024-05-14T15:21:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 msgr2=0x7ff5ac19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 0x7ff5ac19f780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4007ee0 tx=0x7ff5a400d3b0 comp rx=0 tx=0).stop 2024-05-14T15:21:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 shutdown_connections 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff5940b2900 0x7ff5940b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5ac109300 0x7ff5ac19f780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5ac102790 0x7ff5ac19af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 --2- 172.21.15.5:0/3286227696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5ac1030e0 0x7ff5ac19b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 >> 172.21.15.5:0/3286227696 conn(0x7ff5ac0fe4e0 msgr2=0x7ff5ac107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 shutdown_connections 2024-05-14T15:21:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:54.746+0000 7ff5b2bd3700 1 -- 172.21.15.5:0/3286227696 wait complete. 2024-05-14T15:21:55.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:54 smithi144 ceph-mon[35650]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 0 op/s 2024-05-14T15:21:55.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:54 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3286227696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T15:21:55.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:54 smithi193 ceph-mon[36001]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 0 op/s 2024-05-14T15:21:55.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:54 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3286227696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T15:21:55.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:54 smithi005 ceph-mon[31559]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s wr, 0 op/s 2024-05-14T15:21:55.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:54 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3286227696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T15:21:55.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-14T15:21:55.631 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:55.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:55 smithi005 ceph-mon[31559]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T15:21:55.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:55 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3286227696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T15:21:55.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:55 smithi005 ceph-mon[31559]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T15:21:55.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:55 smithi005 ceph-mon[31559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:55.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:55 smithi005 ceph-mon[31559]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T15:21:55.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:55 smithi005 ceph-mon[31559]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T15:21:56.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:55 smithi144 ceph-mon[35650]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T15:21:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:55 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3286227696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T15:21:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:55 smithi144 ceph-mon[35650]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T15:21:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:55 smithi144 ceph-mon[35650]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:55 smithi144 ceph-mon[35650]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T15:21:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:55 smithi144 ceph-mon[35650]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T15:21:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:55 smithi193 ceph-mon[36001]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T15:21:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:55 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3286227696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T15:21:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:55 smithi193 ceph-mon[36001]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T15:21:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:55 smithi193 ceph-mon[36001]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T15:21:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:55 smithi193 ceph-mon[36001]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T15:21:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:55 smithi193 ceph-mon[36001]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T15:21:57.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] up:active 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-14T15:21:57.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:56 smithi005 ceph-mon[31559]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-14T15:21:57.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-14T15:21:57.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] up:active 2024-05-14T15:21:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T15:21:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T15:21:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:21:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T15:21:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-14T15:21:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:56 smithi144 ceph-mon[35650]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-14T15:21:57.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] up:active 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-14T15:21:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:56 smithi193 ceph-mon[36001]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 -- 172.21.15.5:0/2825599909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4108740 msgr2=0x7f3ed41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 --2- 172.21.15.5:0/2825599909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4108740 0x7f3ed41100a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b0d0 tx=0x7f3ecc00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 -- 172.21.15.5:0/2825599909 shutdown_connections 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 --2- 172.21.15.5:0/2825599909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4108740 0x7f3ed41100a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 --2- 172.21.15.5:0/2825599909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed41034c0 0x7f3ed41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 --2- 172.21.15.5:0/2825599909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4107df0 0x7f3ed4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 -- 172.21.15.5:0/2825599909 >> 172.21.15.5:0/2825599909 conn(0x7f3ed40fedf0 msgr2=0x7f3ed4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 -- 172.21.15.5:0/2825599909 shutdown_connections 2024-05-14T15:21:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.283+0000 7f3edb412700 1 -- 172.21.15.5:0/2825599909 wait complete. 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.284+0000 7f3edb412700 1 Processor -- start 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.284+0000 7f3edb412700 1 -- start start 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3edb412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 0x7f3ed419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3edb412700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed4107df0 0x7f3ed419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3edb412700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4108740 0x7f3ed419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3edb412700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41144c0 con 0x7f3ed41034c0 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3edb412700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4114340 con 0x7f3ed4108740 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3edb412700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4114640 con 0x7f3ed4107df0 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed91ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 0x7f3ed419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed91ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 0x7f3ed419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32980/0 (socket says 172.21.15.5:32980) 2024-05-14T15:21:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed91ae700 1 -- 172.21.15.5:0/2267156595 learned_addr learned my addr 172.21.15.5:0/2267156595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed89ad700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed4107df0 0x7f3ed419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed99af700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4108740 0x7f3ed419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed89ad700 1 -- 172.21.15.5:0/2267156595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4108740 msgr2=0x7f3ed419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed89ad700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4108740 0x7f3ed419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed89ad700 1 -- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 msgr2=0x7f3ed419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed89ad700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 0x7f3ed419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed89ad700 1 -- 172.21.15.5:0/2267156595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed419fcc0 con 0x7f3ed4107df0 2024-05-14T15:21:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed91ae700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 0x7f3ed419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:21:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.285+0000 7f3ed99af700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4108740 0x7f3ed419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:21:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.286+0000 7f3ed89ad700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed4107df0 0x7f3ed419b390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e9c0 tx=0x7f3ec400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.286+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400c9c0 con 0x7f3ed4107df0 2024-05-14T15:21:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.286+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed419ffb0 con 0x7f3ed4107df0 2024-05-14T15:21:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.286+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec400cb60 con 0x7f3ed4107df0 2024-05-14T15:21:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.286+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4018680 con 0x7f3ed4107df0 2024-05-14T15:21:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.286+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed41a78d0 con 0x7f3ed4107df0 2024-05-14T15:21:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.287+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed40512c0 con 0x7f3ed4107df0 2024-05-14T15:21:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.289+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3ec4010430 con 0x7f3ed4107df0 2024-05-14T15:21:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.290+0000 7f3eca7fc700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec00b2880 0x7f3ec00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:21:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.290+0000 7f3ed91ae700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec00b2880 0x7f3ec00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:21:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.290+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3ec40e29b0 con 0x7f3ed4107df0 2024-05-14T15:21:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.290+0000 7f3ed91ae700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec00b2880 0x7f3ec00b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0005950 tx=0x7f3ed000b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:21:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.293+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ec40aa390 con 0x7f3ed4107df0 2024-05-14T15:21:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.555+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3ed419c150 con 0x7f3ed4107df0 2024-05-14T15:21:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.774+0000 7f3eca7fc700 1 -- 172.21.15.5:0/2267156595 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f3ec40b2410 con 0x7f3ed4107df0 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.776+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec00b2880 msgr2=0x7f3ec00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.776+0000 7f3edb412700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec00b2880 0x7f3ec00b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0005950 tx=0x7f3ed000b500 comp rx=0 tx=0).stop 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.776+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed4107df0 msgr2=0x7f3ed419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.776+0000 7f3edb412700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed4107df0 0x7f3ed419b390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e9c0 tx=0x7f3ec400ecd0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 shutdown_connections 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec00b2880 0x7f3ec00b4d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed4108740 0x7f3ed419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed41034c0 0x7f3ed419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 --2- 172.21.15.5:0/2267156595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed4107df0 0x7f3ed419b390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 >> 172.21.15.5:0/2267156595 conn(0x7f3ed40fedf0 msgr2=0x7f3ed41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 shutdown_connections 2024-05-14T15:21:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:21:57.777+0000 7f3edb412700 1 -- 172.21.15.5:0/2267156595 wait complete. 2024-05-14T15:21:57.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:57 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] up:active 2024-05-14T15:21:57.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:57 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T15:21:57.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:57 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2267156595' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T15:21:57.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:57 smithi005 ceph-mon[31559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T15:21:58.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:57 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] up:active 2024-05-14T15:21:58.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:57 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T15:21:58.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:57 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2267156595' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T15:21:58.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:57 smithi144 ceph-mon[35650]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T15:21:58.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:57 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] up:active 2024-05-14T15:21:58.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:57 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T15:21:58.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:57 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2267156595' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T15:21:58.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:57 smithi193 ceph-mon[36001]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T15:21:58.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-14T15:21:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:21:58.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:58 smithi005 ceph-mon[31559]: pgmap v326: 129 pgs: 129 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T15:21:58.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:58 smithi005 ceph-mon[31559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T15:21:58.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:58 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T15:21:58.961 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:21:58 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:21:59.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:58 smithi144 ceph-mon[35650]: pgmap v326: 129 pgs: 129 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T15:21:59.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:58 smithi144 ceph-mon[35650]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T15:21:59.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:58 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T15:21:59.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:21:58 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:21:59.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:58 smithi193 ceph-mon[36001]: pgmap v326: 129 pgs: 129 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T15:21:59.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:58 smithi193 ceph-mon[36001]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T15:21:59.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:58 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-14T15:21:59.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:21:58 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:22:00.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/2557502508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 msgr2=0x7f4eb0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:00.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/2557502508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb0108200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00099c0 tx=0x7f4ea0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/2557502508 shutdown_connections 2024-05-14T15:22:00.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/2557502508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4eb0108740 0x7f4eb01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/2557502508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eb01034c0 0x7f4eb01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/2557502508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb0108200 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.407+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/2557502508 >> 172.21.15.5:0/2557502508 conn(0x7f4eb00fedc0 msgr2=0x7f4eb01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:00.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.408+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/2557502508 shutdown_connections 2024-05-14T15:22:00.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.408+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/2557502508 wait complete. 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.408+0000 7f4eb7b6d700 1 Processor -- start 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 -- start start 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eb01034c0 0x7f4eb019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4eb0108740 0x7f4eb019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb01133c0 con 0x7f4eb0107df0 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0113240 con 0x7f4eb01034c0 2024-05-14T15:22:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb7b6d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb0113540 con 0x7f4eb0108740 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb019c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33000/0 (socket says 172.21.15.5:33000) 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 -- 172.21.15.5:0/558962077 learned_addr learned my addr 172.21.15.5:0/558962077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5909700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eb01034c0 0x7f4eb019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb610a700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4eb0108740 0x7f4eb019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 -- 172.21.15.5:0/558962077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4eb0108740 msgr2=0x7f4eb019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4eb0108740 0x7f4eb019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 -- 172.21.15.5:0/558962077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eb01034c0 msgr2=0x7f4eb019bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eb01034c0 0x7f4eb019bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.409+0000 7f4eb5108700 1 -- 172.21.15.5:0/558962077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb01a6fc0 con 0x7f4eb0107df0 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.410+0000 7f4eb5108700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb019c220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009f50 tx=0x7f4ea00037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.410+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0027070 con 0x7f4eb0107df0 2024-05-14T15:22:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.410+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb01a7250 con 0x7f4eb0107df0 2024-05-14T15:22:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.410+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea0004090 con 0x7f4eb0107df0 2024-05-14T15:22:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.410+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0022e60 con 0x7f4eb0107df0 2024-05-14T15:22:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.410+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb01a7700 con 0x7f4eb0107df0 2024-05-14T15:22:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.411+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb00512c0 con 0x7f4eb0107df0 2024-05-14T15:22:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.413+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4ea002a070 con 0x7f4eb0107df0 2024-05-14T15:22:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.413+0000 7f4ea6ffd700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.413+0000 7f4eb5909700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.413+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f4ea00e34c0 con 0x7f4eb0107df0 2024-05-14T15:22:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.414+0000 7f4eb5909700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009a10 tx=0x7f4eac009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.417+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ea0013070 con 0x7f4eb0107df0 2024-05-14T15:22:00.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.686+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f4eb002e5f0 con 0x7f4eb0107df0 2024-05-14T15:22:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.771+0000 7f4ea6ffd700 1 -- 172.21.15.5:0/558962077 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f4ea00aaf30 con 0x7f4eb0107df0 2024-05-14T15:22:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e9c0b2880 msgr2=0x7f4e9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009a10 tx=0x7f4eac009190 comp rx=0 tx=0).stop 2024-05-14T15:22:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 msgr2=0x7f4eb019c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb019c220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009f50 tx=0x7f4ea00037c0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 shutdown_connections 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e9c0b2880 0x7f4e9c0b4d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4eb0108740 0x7f4eb019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eb01034c0 0x7f4eb019bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 --2- 172.21.15.5:0/558962077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eb0107df0 0x7f4eb019c220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 >> 172.21.15.5:0/558962077 conn(0x7f4eb00fedc0 msgr2=0x7f4eb01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 shutdown_connections 2024-05-14T15:22:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:00.775+0000 7f4eb7b6d700 1 -- 172.21.15.5:0/558962077 wait complete. 2024-05-14T15:22:01.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:00 smithi144 ceph-mon[35650]: pgmap v327: 129 pgs: 129 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T15:22:01.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:00 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/558962077' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T15:22:01.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:00 smithi193 ceph-mon[36001]: pgmap v327: 129 pgs: 129 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T15:22:01.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:00 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/558962077' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T15:22:01.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:00 smithi005 ceph-mon[31559]: pgmap v327: 129 pgs: 129 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T15:22:01.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:00 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/558962077' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T15:22:01.431 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-14T15:22:01.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-14T15:22:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:02.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:01 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/558962077' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T15:22:02.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:01 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:22:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:01 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/558962077' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T15:22:02.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:01 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:22:02.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:01 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/558962077' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T15:22:02.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:01 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:22:03.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:02 smithi005 ceph-mon[31559]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:03.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:02 smithi144 ceph-mon[35650]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:03.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:02 smithi193 ceph-mon[36001]: pgmap v328: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.240+0000 7f33207aa700 1 -- 172.21.15.5:0/3486918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318108740 msgr2=0x7f33181100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 --2- 172.21.15.5:0/3486918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318108740 0x7f33181100a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f331000b0d0 tx=0x7f331000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 -- 172.21.15.5:0/3486918126 shutdown_connections 2024-05-14T15:22:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 --2- 172.21.15.5:0/3486918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318108740 0x7f33181100a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 --2- 172.21.15.5:0/3486918126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33181034c0 0x7f33181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 --2- 172.21.15.5:0/3486918126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3318107df0 0x7f3318108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 -- 172.21.15.5:0/3486918126 >> 172.21.15.5:0/3486918126 conn(0x7f33180fedc0 msgr2=0x7f33181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 -- 172.21.15.5:0/3486918126 shutdown_connections 2024-05-14T15:22:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 -- 172.21.15.5:0/3486918126 wait complete. 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 Processor -- start 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.241+0000 7f33207aa700 1 -- start start 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f33207aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33181034c0 0x7f33180ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f33207aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 0x7f33180ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f33207aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3318108740 0x7f33180ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f33207aa700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3318113370 con 0x7f3318107df0 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f33207aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33181131f0 con 0x7f3318108740 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f33207aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33181134f0 con 0x7f33181034c0 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 0x7f33180ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 0x7f33180ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33018/0 (socket says 172.21.15.5:33018) 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 -- 172.21.15.5:0/3286293238 learned_addr learned my addr 172.21.15.5:0/3286293238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331e546700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33181034c0 0x7f33180ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331ed47700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3318108740 0x7f33180ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 -- 172.21.15.5:0/3286293238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33181034c0 msgr2=0x7f33180ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33181034c0 0x7f33180ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 -- 172.21.15.5:0/3286293238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3318108740 msgr2=0x7f33180ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3318108740 0x7f33180ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 -- 172.21.15.5:0/3286293238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33181a9170 con 0x7f3318107df0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.242+0000 7f331dd45700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 0x7f33180ff7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f330800d980 tx=0x7f330800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.243+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308017070 con 0x7f3318107df0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.243+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f330800fd40 con 0x7f3318107df0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.243+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33181a9460 con 0x7f3318107df0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.243+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330800e3b0 con 0x7f3318107df0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.243+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33181a99f0 con 0x7f3318107df0 2024-05-14T15:22:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.244+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33180512c0 con 0x7f3318107df0 2024-05-14T15:22:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.245+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3308010610 con 0x7f3318107df0 2024-05-14T15:22:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.246+0000 7f330f7fe700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f33040b2900 0x7f33040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.246+0000 7f331e546700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f33040b2900 0x7f33040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.246+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f33080e1eb0 con 0x7f3318107df0 2024-05-14T15:22:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.247+0000 7f331e546700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f33040b2900 0x7f33040b4e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3314005950 tx=0x7f331400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:03.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.251+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33080a9890 con 0x7f3318107df0 2024-05-14T15:22:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.501+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f33181007d0 con 0x7f33040b2900 2024-05-14T15:22:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.530+0000 7f330f7fe700 1 -- 172.21.15.5:0/3286293238 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33181007d0 con 0x7f33040b2900 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.532+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f33040b2900 msgr2=0x7f33040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.532+0000 7f33207aa700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f33040b2900 0x7f33040b4e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3314005950 tx=0x7f331400b410 comp rx=0 tx=0).stop 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.532+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 msgr2=0x7f33180ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.532+0000 7f33207aa700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 0x7f33180ff7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f330800d980 tx=0x7f330800dc90 comp rx=0 tx=0).stop 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 shutdown_connections 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f33040b2900 0x7f33040b4e10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3318108740 0x7f33180ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33181034c0 0x7f33180ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 --2- 172.21.15.5:0/3286293238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3318107df0 0x7f33180ff7e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 >> 172.21.15.5:0/3286293238 conn(0x7f33180fedc0 msgr2=0x7f331806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 shutdown_connections 2024-05-14T15:22:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:03.533+0000 7f33207aa700 1 -- 172.21.15.5:0/3286293238 wait complete. 2024-05-14T15:22:03.815 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:22:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:22:03.507+0000 7f05106e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:22:03.815 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:22:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:22:03.507+0000 7f05106e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:22:03.815 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:22:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:22:03.507+0000 7f05106e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:22:03.815 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:22:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:22:03.507+0000 7f05106e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:22:03.815 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:22:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:22:03.507+0000 7f05106e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:22:04.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-14T15:22:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:04.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:04 smithi005 ceph-mon[31559]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:04.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:04 smithi005 ceph-mon[31559]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:04.956 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:04 smithi005 ceph-mon[31559]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-14T15:22:05.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:04 smithi144 ceph-mon[35650]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:05.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:04 smithi144 ceph-mon[35650]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:05.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:04 smithi144 ceph-mon[35650]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-14T15:22:05.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:04 smithi193 ceph-mon[36001]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:05.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:04 smithi193 ceph-mon[36001]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:05.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:04 smithi193 ceph-mon[36001]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-14T15:22:06.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 -- 172.21.15.5:0/951013454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4103550 msgr2=0x7f68e4103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:06.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 --2- 172.21.15.5:0/951013454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4103550 0x7f68e4103960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f68d80099c0 tx=0x7f68d8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 -- 172.21.15.5:0/951013454 shutdown_connections 2024-05-14T15:22:06.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 --2- 172.21.15.5:0/951013454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 0x7f68e41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 --2- 172.21.15.5:0/951013454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4103550 0x7f68e4103960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 --2- 172.21.15.5:0/951013454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e4107e30 0x7f68e4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 -- 172.21.15.5:0/951013454 >> 172.21.15.5:0/951013454 conn(0x7f68e4078e00 msgr2=0x7f68e4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:06.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 -- 172.21.15.5:0/951013454 shutdown_connections 2024-05-14T15:22:06.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.130+0000 7f68eac7f700 1 -- 172.21.15.5:0/951013454 wait complete. 2024-05-14T15:22:06.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 Processor -- start 2024-05-14T15:22:06.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 -- start start 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e4103550 0x7f68e4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 0x7f68e419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 0x7f68e419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68e41133b0 con 0x7f68e4107e30 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4113230 con 0x7f68e4108780 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68eac7f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68e4113530 con 0x7f68e4103550 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 0x7f68e419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 0x7f68e419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45780/0 (socket says 172.21.15.5:45780) 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68e3fff700 1 -- 172.21.15.5:0/3939515673 learned_addr learned my addr 172.21.15.5:0/3939515673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68e921c700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 0x7f68e419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.131+0000 7f68e8a1b700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e4103550 0x7f68e4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e3fff700 1 -- 172.21.15.5:0/3939515673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e4103550 msgr2=0x7f68e4199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e3fff700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e4103550 0x7f68e4199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e3fff700 1 -- 172.21.15.5:0/3939515673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 msgr2=0x7f68e419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e3fff700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 0x7f68e419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e3fff700 1 -- 172.21.15.5:0/3939515673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e419eb60 con 0x7f68e4107e30 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e921c700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 0x7f68e419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e3fff700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 0x7f68e419a2e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f68d00098f0 tx=0x7f68d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000d020 con 0x7f68e4107e30 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d000d1c0 con 0x7f68e4107e30 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0007860 con 0x7f68e4107e30 2024-05-14T15:22:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.132+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e419ee50 con 0x7f68e4107e30 2024-05-14T15:22:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.133+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e40759f0 con 0x7f68e4107e30 2024-05-14T15:22:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.134+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e41049b0 con 0x7f68e4107e30 2024-05-14T15:22:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.135+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f68d0007a00 con 0x7f68e4107e30 2024-05-14T15:22:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.136+0000 7f68e1ffb700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f68d40b2990 0x7f68d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.136+0000 7f68e8a1b700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f68d40b2990 0x7f68d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.136+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f68d00e3d20 con 0x7f68e4107e30 2024-05-14T15:22:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.137+0000 7f68e8a1b700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f68d40b2990 0x7f68d40b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009990 tx=0x7f68d80036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.140+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f68d00ab5a0 con 0x7f68e4107e30 2024-05-14T15:22:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.388+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f68e4075c50 con 0x7f68d40b2990 2024-05-14T15:22:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.417+0000 7f68e1ffb700 1 -- 172.21.15.5:0/3939515673 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68e4075c50 con 0x7f68d40b2990 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f68d40b2990 msgr2=0x7f68d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f68d40b2990 0x7f68d40b4ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009990 tx=0x7f68d80036e0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 msgr2=0x7f68e419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 0x7f68e419a2e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f68d00098f0 tx=0x7f68d0009c00 comp rx=0 tx=0).stop 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 shutdown_connections 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f68d40b2990 0x7f68d40b4ea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68e4108780 0x7f68e419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e4103550 0x7f68e4199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 --2- 172.21.15.5:0/3939515673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68e4107e30 0x7f68e419a2e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 >> 172.21.15.5:0/3939515673 conn(0x7f68e4078e00 msgr2=0x7f68e4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 shutdown_connections 2024-05-14T15:22:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:06.420+0000 7f68eac7f700 1 -- 172.21.15.5:0/3939515673 wait complete. 2024-05-14T15:22:07.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:06 smithi144 ceph-mon[35650]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:07.052 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:06 smithi193 ceph-mon[36001]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:07.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:06 smithi005 ceph-mon[31559]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-14T15:22:07.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-14T15:22:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:08.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:07 smithi193 ceph-mon[36001]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:08.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:07 smithi005 ceph-mon[31559]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:08.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:07 smithi144 ceph-mon[35650]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 -- 172.21.15.5:0/239001652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec1034c0 msgr2=0x7f45ec1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/239001652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec1034c0 0x7f45ec1038d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0099c0 tx=0x7f45dc009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 -- 172.21.15.5:0/239001652 shutdown_connections 2024-05-14T15:22:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/239001652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec108740 0x7f45ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/239001652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec1034c0 0x7f45ec1038d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/239001652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec107df0 0x7f45ec108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 -- 172.21.15.5:0/239001652 >> 172.21.15.5:0/239001652 conn(0x7f45ec0fedf0 msgr2=0x7f45ec101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 -- 172.21.15.5:0/239001652 shutdown_connections 2024-05-14T15:22:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.951+0000 7f45f1f1d700 1 -- 172.21.15.5:0/239001652 wait complete. 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 Processor -- start 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 -- start start 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec1034c0 0x7f45ec19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 0x7f45ec19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec108740 0x7f45ec199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec113370 con 0x7f45ec107df0 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec1131f0 con 0x7f45ec1034c0 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.952+0000 7f45f1f1d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec1134f0 con 0x7f45ec108740 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 0x7f45ec19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 0x7f45ec19fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45802/0 (socket says 172.21.15.5:45802) 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 -- 172.21.15.5:0/948620883 learned_addr learned my addr 172.21.15.5:0/948620883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eb7fe700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec1034c0 0x7f45ec19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45ebfff700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec108740 0x7f45ec199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 -- 172.21.15.5:0/948620883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec108740 msgr2=0x7f45ec199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec108740 0x7f45ec199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 -- 172.21.15.5:0/948620883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec1034c0 msgr2=0x7f45ec19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec1034c0 0x7f45ec19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 -- 172.21.15.5:0/948620883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec19a160 con 0x7f45ec107df0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eb7fe700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec1034c0 0x7f45ec19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45eaffd700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 0x7f45ec19fe10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c9e0 tx=0x7f45e000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0007aa0 con 0x7f45ec107df0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e0007c40 con 0x7f45ec107df0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec19a450 con 0x7f45ec107df0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0018660 con 0x7f45ec107df0 2024-05-14T15:22:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.953+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec1a6760 con 0x7f45ec107df0 2024-05-14T15:22:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.954+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ec0512c0 con 0x7f45ec107df0 2024-05-14T15:22:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.956+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f45e0018800 con 0x7f45ec107df0 2024-05-14T15:22:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.957+0000 7f45e8ff9700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f45d40b2990 0x7f45d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.957+0000 7f45eb7fe700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f45d40b2990 0x7f45d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.957+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f45e00e2880 con 0x7f45ec107df0 2024-05-14T15:22:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.957+0000 7f45eb7fe700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f45d40b2990 0x7f45d40b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00b580 tx=0x7f45dc0096f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:08.961+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45e0054df0 con 0x7f45ec107df0 2024-05-14T15:22:09.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:08 smithi005 ceph-mon[31559]: pgmap v331: 129 pgs: 129 active+clean; 591 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-14T15:22:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.210+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f45ec19afe0 con 0x7f45d40b2990 2024-05-14T15:22:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.215+0000 7f45e8ff9700 1 -- 172.21.15.5:0/948620883 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f45ec19afe0 con 0x7f45d40b2990 2024-05-14T15:22:09.215 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/d5fee1b7-0af3-4866-bcfd-0d8684813c3c 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f45d40b2990 msgr2=0x7f45d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f45d40b2990 0x7f45d40b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00b580 tx=0x7f45dc0096f0 comp rx=0 tx=0).stop 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 msgr2=0x7f45ec19fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 0x7f45ec19fe10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c9e0 tx=0x7f45e000cda0 comp rx=0 tx=0).stop 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 shutdown_connections 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f45d40b2990 0x7f45d40b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45ec108740 0x7f45ec199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45ec1034c0 0x7f45ec19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 --2- 172.21.15.5:0/948620883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45ec107df0 0x7f45ec19fe10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.217+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 >> 172.21.15.5:0/948620883 conn(0x7f45ec0fedf0 msgr2=0x7f45ec101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.218+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 shutdown_connections 2024-05-14T15:22:09.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:09.218+0000 7f45f1f1d700 1 -- 172.21.15.5:0/948620883 wait complete. 2024-05-14T15:22:09.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:08 smithi144 ceph-mon[35650]: pgmap v331: 129 pgs: 129 active+clean; 591 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-14T15:22:09.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:08 smithi193 ceph-mon[36001]: pgmap v331: 129 pgs: 129 active+clean; 591 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-14T15:22:09.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-14T15:22:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:11.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:10 smithi005 ceph-mon[31559]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:11.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:10 smithi005 ceph-mon[31559]: pgmap v332: 129 pgs: 129 active+clean; 591 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T15:22:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:10 smithi144 ceph-mon[35650]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:11.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:10 smithi144 ceph-mon[35650]: pgmap v332: 129 pgs: 129 active+clean; 591 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T15:22:11.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:10 smithi193 ceph-mon[36001]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:11.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:10 smithi193 ceph-mon[36001]: pgmap v332: 129 pgs: 129 active+clean; 591 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T15:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 -- 172.21.15.5:0/1863537347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560107df0 msgr2=0x7f6560108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 --2- 172.21.15.5:0/1863537347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560107df0 0x7f6560108200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65540099c0 tx=0x7f6554009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 -- 172.21.15.5:0/1863537347 shutdown_connections 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 --2- 172.21.15.5:0/1863537347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6560108740 0x7f65601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 --2- 172.21.15.5:0/1863537347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65601034c0 0x7f65601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 --2- 172.21.15.5:0/1863537347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560107df0 0x7f6560108200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 -- 172.21.15.5:0/1863537347 >> 172.21.15.5:0/1863537347 conn(0x7f65600fedf0 msgr2=0x7f6560101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 -- 172.21.15.5:0/1863537347 shutdown_connections 2024-05-14T15:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.744+0000 7f656543a700 1 -- 172.21.15.5:0/1863537347 wait complete. 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.745+0000 7f656543a700 1 Processor -- start 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.745+0000 7f656543a700 1 -- start start 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f656543a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65601034c0 0x7f656019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f656543a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560107df0 0x7f656019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f656543a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 0x7f65601a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f656543a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6560113350 con 0x7f6560108740 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f656543a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65601131d0 con 0x7f6560107df0 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f656543a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65601134d0 con 0x7f65601034c0 2024-05-14T15:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 0x7f65601a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 0x7f65601a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45824/0 (socket says 172.21.15.5:45824) 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 -- 172.21.15.5:0/361491519 learned_addr learned my addr 172.21.15.5:0/361491519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655effd700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65601034c0 0x7f656019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655e7fc700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560107df0 0x7f656019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 -- 172.21.15.5:0/361491519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65601034c0 msgr2=0x7f656019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65601034c0 0x7f656019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 -- 172.21.15.5:0/361491519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560107df0 msgr2=0x7f656019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560107df0 0x7f656019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 -- 172.21.15.5:0/361491519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65601a1f10 con 0x7f6560108740 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655e7fc700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560107df0 0x7f656019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.746+0000 7f655f7fe700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 0x7f65601a1870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f655800ada0 tx=0x7f655800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.747+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558017020 con 0x7f6560108740 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.747+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65601a2200 con 0x7f6560108740 2024-05-14T15:22:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.747+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655800ce30 con 0x7f6560108740 2024-05-14T15:22:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.747+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601a9b20 con 0x7f6560108740 2024-05-14T15:22:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.747+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558005350 con 0x7f6560108740 2024-05-14T15:22:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.748+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65600512c0 con 0x7f6560108740 2024-05-14T15:22:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.749+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f655800c980 con 0x7f6560108740 2024-05-14T15:22:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.750+0000 7f6547fff700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f65480b2990 0x7f65480b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.750+0000 7f655effd700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f65480b2990 0x7f65480b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.750+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f65580e2250 con 0x7f6560108740 2024-05-14T15:22:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.751+0000 7f655effd700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f65480b2990 0x7f65480b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6550005950 tx=0x7f655000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:11.754+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6558021020 con 0x7f6560108740 2024-05-14T15:22:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.002+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6560063c00 con 0x7f65480b2990 2024-05-14T15:22:12.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.035+0000 7f6547fff700 1 -- 172.21.15.5:0/361491519 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6560063c00 con 0x7f65480b2990 2024-05-14T15:22:12.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f65480b2990 msgr2=0x7f65480b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:12.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f65480b2990 0x7f65480b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6550005950 tx=0x7f655000a300 comp rx=0 tx=0).stop 2024-05-14T15:22:12.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 msgr2=0x7f65601a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:12.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 0x7f65601a1870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f655800ada0 tx=0x7f655800c330 comp rx=0 tx=0).stop 2024-05-14T15:22:12.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 shutdown_connections 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f65480b2990 0x7f65480b4ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560108740 0x7f65601a1870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65601034c0 0x7f656019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 --2- 172.21.15.5:0/361491519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6560107df0 0x7f656019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 >> 172.21.15.5:0/361491519 conn(0x7f65600fedf0 msgr2=0x7f656010ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 shutdown_connections 2024-05-14T15:22:12.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:12.037+0000 7f656543a700 1 -- 172.21.15.5:0/361491519 wait complete. 2024-05-14T15:22:12.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-14T15:22:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:13.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:12 smithi005 ceph-mon[31559]: pgmap v333: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-14T15:22:13.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:12 smithi144 ceph-mon[35650]: pgmap v333: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-14T15:22:13.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:12 smithi193 ceph-mon[36001]: pgmap v333: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-14T15:22:14.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:13 smithi005 ceph-mon[31559]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:14.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:13 smithi144 ceph-mon[35650]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:14.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:13 smithi193 ceph-mon[36001]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:14.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 -- 172.21.15.5:0/2453725081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe414107e10 msgr2=0x7fe414108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:14.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2453725081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe414107e10 0x7fe414108220 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4040099c0 tx=0x7fe404009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 -- 172.21.15.5:0/2453725081 shutdown_connections 2024-05-14T15:22:14.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2453725081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414108760 0x7fe4141100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2453725081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4141034e0 0x7fe4141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2453725081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe414107e10 0x7fe414108220 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 -- 172.21.15.5:0/2453725081 >> 172.21.15.5:0/2453725081 conn(0x7fe4140fef30 msgr2=0x7fe414101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:14.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 -- 172.21.15.5:0/2453725081 shutdown_connections 2024-05-14T15:22:14.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.561+0000 7fe41b50e700 1 -- 172.21.15.5:0/2453725081 wait complete. 2024-05-14T15:22:14.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 Processor -- start 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 -- start start 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 0x7fe414199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414107e10 0x7fe41419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe414108760 0x7fe41419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe414113370 con 0x7fe4141034e0 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141131f0 con 0x7fe414108760 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.562+0000 7fe41b50e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141134f0 con 0x7fe414107e10 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 0x7fe414199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 0x7fe414199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45842/0 (socket says 172.21.15.5:45842) 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 -- 172.21.15.5:0/2925294563 learned_addr learned my addr 172.21.15.5:0/2925294563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe418aa9700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414107e10 0x7fe41419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 -- 172.21.15.5:0/2925294563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414107e10 msgr2=0x7fe41419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414107e10 0x7fe41419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 -- 172.21.15.5:0/2925294563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe414108760 msgr2=0x7fe41419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:14.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe414108760 0x7fe41419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 -- 172.21.15.5:0/2925294563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41419ec10 con 0x7fe4141034e0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe4192aa700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 0x7fe414199d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe41000e9a0 tx=0x7fe41000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41000cbd0 con 0x7fe4141034e0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41419ef00 con 0x7fe4141034e0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41000cd70 con 0x7fe4141034e0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410018680 con 0x7fe4141034e0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.563+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141a6820 con 0x7fe4141034e0 2024-05-14T15:22:14.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.564+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4140512c0 con 0x7fe4141034e0 2024-05-14T15:22:14.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.566+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe410010430 con 0x7fe4141034e0 2024-05-14T15:22:14.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.567+0000 7fe40a7fc700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe4000b2910 0x7fe4000b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:14.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.567+0000 7fe418aa9700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe4000b2910 0x7fe4000b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:14.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.567+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fe4100e2c10 con 0x7fe4141034e0 2024-05-14T15:22:14.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.567+0000 7fe418aa9700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe4000b2910 0x7fe4000b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe40401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:14.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.570+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4100aa430 con 0x7fe4141034e0 2024-05-14T15:22:14.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.823+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe414063c00 con 0x7fe4000b2910 2024-05-14T15:22:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.828+0000 7fe40a7fc700 1 -- 172.21.15.5:0/2925294563 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe414063c00 con 0x7fe4000b2910 2024-05-14T15:22:14.828 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/a7c2309d-a418-4b06-b5dd-d00a7f41b7c3 2024-05-14T15:22:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe4000b2910 msgr2=0x7fe4000b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe4000b2910 0x7fe4000b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe40401a040 comp rx=0 tx=0).stop 2024-05-14T15:22:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 msgr2=0x7fe414199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 0x7fe414199d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe41000e9a0 tx=0x7fe41000ecb0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 shutdown_connections 2024-05-14T15:22:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fe4000b2910 0x7fe4000b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe414108760 0x7fe41419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4141034e0 0x7fe414199d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 --2- 172.21.15.5:0/2925294563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414107e10 0x7fe41419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 >> 172.21.15.5:0/2925294563 conn(0x7fe4140fef30 msgr2=0x7fe4141096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 shutdown_connections 2024-05-14T15:22:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:14.831+0000 7fe41b50e700 1 -- 172.21.15.5:0/2925294563 wait complete. 2024-05-14T15:22:15.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:14 smithi005 ceph-mon[31559]: pgmap v334: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-14T15:22:15.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:14 smithi144 ceph-mon[35650]: pgmap v334: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-14T15:22:15.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:14 smithi193 ceph-mon[36001]: pgmap v334: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-14T15:22:15.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:22:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:16.111 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:15 smithi005 ceph-mon[31559]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:16.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:15 smithi144 ceph-mon[35650]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:16.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:15 smithi193 ceph-mon[36001]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:22:16.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:16 smithi005 ceph-mon[31559]: pgmap v335: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T15:22:17.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:16 smithi144 ceph-mon[35650]: pgmap v335: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T15:22:17.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:16 smithi193 ceph-mon[36001]: pgmap v335: 129 pgs: 129 active+clean; 610 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-14T15:22:17.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/858029383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 msgr2=0x7f0c9c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/858029383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c109310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c840099c0 tx=0x7f0c84009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/858029383 shutdown_connections 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/858029383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c109850 0x7f0c9c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/858029383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c9c1045d0 0x7f0c9c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/858029383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c109310 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/858029383 >> 172.21.15.5:0/858029383 conn(0x7f0c9c0ffea0 msgr2=0x7f0c9c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/858029383 shutdown_connections 2024-05-14T15:22:17.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.447+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/858029383 wait complete. 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.448+0000 7f0ca0ba3700 1 Processor -- start 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 -- start start 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c9c1045d0 0x7f0c9c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c109850 0x7f0c9c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c114460 con 0x7f0c9c108f00 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c1142e0 con 0x7f0c9c109850 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0ca0ba3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c1145e0 con 0x7f0c9c1045d0 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37080/0 (socket says 172.21.15.5:37080) 2024-05-14T15:22:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 -- 172.21.15.5:0/3996227923 learned_addr learned my addr 172.21.15.5:0/3996227923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c9a59c700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c9c1045d0 0x7f0c9c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c9ad9d700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c109850 0x7f0c9c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 -- 172.21.15.5:0/3996227923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c9c1045d0 msgr2=0x7f0c9c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c9c1045d0 0x7f0c9c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 -- 172.21.15.5:0/3996227923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c109850 msgr2=0x7f0c9c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c109850 0x7f0c9c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.449+0000 7f0c91d9b700 1 -- 172.21.15.5:0/3996227923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c19fcf0 con 0x7f0c9c108f00 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.450+0000 7f0c91d9b700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c19b470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c84006010 tx=0x7f0c840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.450+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84027070 con 0x7f0c9c108f00 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.450+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c19ff80 con 0x7f0c9c108f00 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.450+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c84003f20 con 0x7f0c9c108f00 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.450+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84022c60 con 0x7f0c9c108f00 2024-05-14T15:22:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.450+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c1a0460 con 0x7f0c9c108f00 2024-05-14T15:22:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.451+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c9c0512c0 con 0x7f0c9c108f00 2024-05-14T15:22:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.452+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0c8402a070 con 0x7f0c9c108f00 2024-05-14T15:22:17.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.454+0000 7f0c93fff700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0c880b2910 0x7f0c880b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:17.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.454+0000 7f0c9a59c700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0c880b2910 0x7f0c880b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:17.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.454+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f0c84022e00 con 0x7f0c9c108f00 2024-05-14T15:22:17.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.454+0000 7f0c9a59c700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0c880b2910 0x7f0c880b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009550 tx=0x7f0c8c0091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:17.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.457+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c84031080 con 0x7f0c9c108f00 2024-05-14T15:22:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.719+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c9c02e5f0 con 0x7f0c9c108f00 2024-05-14T15:22:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.720+0000 7f0c93fff700 1 -- 172.21.15.5:0/3996227923 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11864 (secure 0 0 0) 0x7f0c84013070 con 0x7f0c9c108f00 2024-05-14T15:22:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:17.720 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:22:00.688182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24487,24515]},"id":1}]} 2024-05-14T15:22:17.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0c880b2910 msgr2=0x7f0c880b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:17.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0c880b2910 0x7f0c880b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009550 tx=0x7f0c8c0091b0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 msgr2=0x7f0c9c19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c19b470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c84006010 tx=0x7f0c840047b0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 shutdown_connections 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0c880b2910 0x7f0c880b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c109850 0x7f0c9c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c9c1045d0 0x7f0c9c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 --2- 172.21.15.5:0/3996227923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c108f00 0x7f0c9c19b470 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.722+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 >> 172.21.15.5:0/3996227923 conn(0x7f0c9c0ffea0 msgr2=0x7f0c9c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.723+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 shutdown_connections 2024-05-14T15:22:17.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:17.723+0000 7f0ca0ba3700 1 -- 172.21.15.5:0/3996227923 wait complete. 2024-05-14T15:22:17.724 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:22:18.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:17 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3996227923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:22:18.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:17 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3996227923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:22:18.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:17 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3996227923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:22:18.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:22:18.764 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:19.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:18 smithi005 ceph-mon[31559]: pgmap v336: 129 pgs: 129 active+clean; 637 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-14T15:22:19.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:18 smithi144 ceph-mon[35650]: pgmap v336: 129 pgs: 129 active+clean; 637 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-14T15:22:19.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:18 smithi193 ceph-mon[36001]: pgmap v336: 129 pgs: 129 active+clean; 637 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-05-14T15:22:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 -- 172.21.15.5:0/4020497212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 msgr2=0x7f8c501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 --2- 172.21.15.5:0/4020497212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c501100a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b0d0 tx=0x7f8c4800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 -- 172.21.15.5:0/4020497212 shutdown_connections 2024-05-14T15:22:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 --2- 172.21.15.5:0/4020497212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c501100a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 --2- 172.21.15.5:0/4020497212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c501034c0 0x7f8c501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 --2- 172.21.15.5:0/4020497212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c50107df0 0x7f8c50108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 -- 172.21.15.5:0/4020497212 >> 172.21.15.5:0/4020497212 conn(0x7f8c500fedf0 msgr2=0x7f8c50101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 -- 172.21.15.5:0/4020497212 shutdown_connections 2024-05-14T15:22:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 -- 172.21.15.5:0/4020497212 wait complete. 2024-05-14T15:22:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.465+0000 7f8c5600e700 1 Processor -- start 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 -- start start 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c501034c0 0x7f8c5019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50107df0 0x7f8c5019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c5019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50113370 con 0x7f8c50108740 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501131f0 con 0x7f8c501034c0 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c5600e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c501134f0 con 0x7f8c50107df0 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c5019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c5019c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37090/0 (socket says 172.21.15.5:37090) 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 -- 172.21.15.5:0/3892229694 learned_addr learned my addr 172.21.15.5:0/3892229694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4f7fe700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c501034c0 0x7f8c5019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4effd700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50107df0 0x7f8c5019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 -- 172.21.15.5:0/3892229694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50107df0 msgr2=0x7f8c5019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50107df0 0x7f8c5019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 -- 172.21.15.5:0/3892229694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c501034c0 msgr2=0x7f8c5019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.466+0000 7f8c4ffff700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c501034c0 0x7f8c5019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c4ffff700 1 -- 172.21.15.5:0/3892229694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c501a6fe0 con 0x7f8c50108740 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c4ffff700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c5019c730 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c480095c0 tx=0x7f8c48007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4801f070 con 0x7f8c50108740 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c48003c00 con 0x7f8c50108740 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4801b6a0 con 0x7f8c50108740 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c501a7270 con 0x7f8c50108740 2024-05-14T15:22:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.467+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501a76c0 con 0x7f8c50108740 2024-05-14T15:22:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.468+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c500512c0 con 0x7f8c50108740 2024-05-14T15:22:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.470+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f8c48022070 con 0x7f8c50108740 2024-05-14T15:22:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.470+0000 7f8c4cff9700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8c380b2910 0x7f8c380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.470+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f8c4800f020 con 0x7f8c50108740 2024-05-14T15:22:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.470+0000 7f8c4f7fe700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8c380b2910 0x7f8c380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:20.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.471+0000 7f8c4f7fe700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8c380b2910 0x7f8c380b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c40007be0 tx=0x7f8c400073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:20.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.475+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c48029020 con 0x7f8c50108740 2024-05-14T15:22:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.697+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c5002e7b0 con 0x7f8c50108740 2024-05-14T15:22:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.699+0000 7f8c4cff9700 1 -- 172.21.15.5:0/3892229694 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24528 (secure 0 0 0) 0x7f8c480ab290 con 0x7f8c50108740 2024-05-14T15:22:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:20.699 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:42.610573+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:22:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8c380b2910 msgr2=0x7f8c380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8c380b2910 0x7f8c380b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c40007be0 tx=0x7f8c400073d0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 msgr2=0x7f8c5019c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c5019c730 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c480095c0 tx=0x7f8c48007a20 comp rx=0 tx=0).stop 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 shutdown_connections 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8c380b2910 0x7f8c380b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c50108740 0x7f8c5019c730 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c501034c0 0x7f8c5019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 --2- 172.21.15.5:0/3892229694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50107df0 0x7f8c5019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 >> 172.21.15.5:0/3892229694 conn(0x7f8c500fedf0 msgr2=0x7f8c50107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 shutdown_connections 2024-05-14T15:22:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:20.701+0000 7f8c5600e700 1 -- 172.21.15.5:0/3892229694 wait complete. 2024-05-14T15:22:21.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:20 smithi005 ceph-mon[31559]: pgmap v337: 129 pgs: 129 active+clean; 637 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T15:22:21.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:20 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3892229694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:21.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:20 smithi144 ceph-mon[35650]: pgmap v337: 129 pgs: 129 active+clean; 637 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T15:22:21.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:20 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3892229694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:21.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:20 smithi193 ceph-mon[36001]: pgmap v337: 129 pgs: 129 active+clean; 637 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T15:22:21.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:20 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3892229694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:21.452 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T15:22:21.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T15:22:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:23.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:22 smithi005 ceph-mon[31559]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-14T15:22:23.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:22 smithi144 ceph-mon[35650]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-14T15:22:23.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:22 smithi193 ceph-mon[36001]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 -- 172.21.15.5:0/1266839718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38141032f0 msgr2=0x7f3814103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 --2- 172.21.15.5:0/1266839718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38141032f0 0x7f3814103700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38040099c0 tx=0x7f3804009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 -- 172.21.15.5:0/1266839718 shutdown_connections 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 --2- 172.21.15.5:0/1266839718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814108570 0x7f381410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 --2- 172.21.15.5:0/1266839718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38141032f0 0x7f3814103700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 --2- 172.21.15.5:0/1266839718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814107c20 0x7f3814108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 -- 172.21.15.5:0/1266839718 >> 172.21.15.5:0/1266839718 conn(0x7f3814078b00 msgr2=0x7f3814078f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 -- 172.21.15.5:0/1266839718 shutdown_connections 2024-05-14T15:22:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.486+0000 7f381359e700 1 -- 172.21.15.5:0/1266839718 wait complete. 2024-05-14T15:22:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 Processor -- start 2024-05-14T15:22:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 -- start start 2024-05-14T15:22:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38141032f0 0x7f38141a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 0x7f38141a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814108570 0x7f381419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3814114310 con 0x7f3814107c20 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3814114190 con 0x7f3814108570 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f381359e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3814114490 con 0x7f38141032f0 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 0x7f38141a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 0x7f38141a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37106/0 (socket says 172.21.15.5:37106) 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.487+0000 7f3811d9b700 1 -- 172.21.15.5:0/4245997142 learned_addr learned my addr 172.21.15.5:0/4245997142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3812d9d700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814108570 0x7f381419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f381259c700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38141032f0 0x7f38141a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3811d9b700 1 -- 172.21.15.5:0/4245997142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38141032f0 msgr2=0x7f38141a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3811d9b700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38141032f0 0x7f38141a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3811d9b700 1 -- 172.21.15.5:0/4245997142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814108570 msgr2=0x7f381419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3811d9b700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814108570 0x7f381419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3811d9b700 1 -- 172.21.15.5:0/4245997142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381419b1d0 con 0x7f3814107c20 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f381259c700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38141032f0 0x7f38141a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f3811d9b700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 0x7f38141a0ee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f380800b500 tx=0x7f380800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808015020 con 0x7f3814107c20 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3808011ce0 con 0x7f3814107c20 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808010d40 con 0x7f3814107c20 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381419b4c0 con 0x7f3814107c20 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.488+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a7750 con 0x7f3814107c20 2024-05-14T15:22:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.489+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38141047a0 con 0x7f3814107c20 2024-05-14T15:22:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.491+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3808018020 con 0x7f3814107c20 2024-05-14T15:22:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.492+0000 7f38037fe700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37fc0b2990 0x7f37fc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.492+0000 7f381259c700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37fc0b2990 0x7f37fc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.492+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f38080e20c0 con 0x7f3814107c20 2024-05-14T15:22:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.492+0000 7f381259c700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37fc0b2990 0x7f37fc0b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3804003800 tx=0x7f3804005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.495+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f380801f020 con 0x7f3814107c20 2024-05-14T15:22:23.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.705+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3814063c00 con 0x7f37fc0b2990 2024-05-14T15:22:23.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.717+0000 7f38037fe700 1 -- 172.21.15.5:0/4245997142 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340615 (secure 0 0 0) 0x7f3814063c00 con 0x7f37fc0b2990 2024-05-14T15:22:23.722 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37fc0b2990 msgr2=0x7f37fc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37fc0b2990 0x7f37fc0b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3804003800 tx=0x7f3804005e80 comp rx=0 tx=0).stop 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 msgr2=0x7f38141a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 0x7f38141a0ee0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f380800b500 tx=0x7f380800b8c0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 shutdown_connections 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37fc0b2990 0x7f37fc0b4ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814108570 0x7f381419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38141032f0 0x7f38141a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 --2- 172.21.15.5:0/4245997142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814107c20 0x7f38141a0ee0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 >> 172.21.15.5:0/4245997142 conn(0x7f3814078b00 msgr2=0x7f3814100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 shutdown_connections 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:23.720+0000 7f381359e700 1 -- 172.21.15.5:0/4245997142 wait complete. 2024-05-14T15:22:23.725 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:22:24.392 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-05-14T15:22:23.484523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652995,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":250,"num_read_kb":220,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377552,"kb_used_data":11760,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564272,"statfs":{"total":1151940427776,"available":1151553814528,"internally_reserved":0,"allocated":12042240,"data_stored":7923279,"data_compressed":425394,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28164,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003796"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.615259+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:21:42.615259+0000","last_peered":"2024-05-14T15:21:42.615259+0000","last_clean":"2024-05-14T15:21:42.615259+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:21:42.615259+0000","last_undegraded":"2024-05-14T15:21:42.615259+0000","last_fullsized":"2024-05-14T15:21:42.615259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619055+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:21:42.619055+0000","last_peered":"2024-05-14T15:21:42.619055+0000","last_clean":"2024-05-14T15:21:42.619055+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:21:42.619055+0000","last_undegraded":"2024-05-14T15:21:42.619055+0000","last_fullsized":"2024-05-14T15:21:42.619055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618843+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:21:42.618843+0000","last_peered":"2024-05-14T15:21:42.618843+0000","last_clean":"2024-05-14T15:21:42.618843+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:21:42.618843+0000","last_undegraded":"2024-05-14T15:21:42.618843+0000","last_fullsized":"2024-05-14T15:21:42.618843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617553+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:21:42.617553+0000","last_peered":"2024-05-14T15:21:42.617553+0000","last_clean":"2024-05-14T15:21:42.617553+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:21:42.617553+0000","last_undegraded":"2024-05-14T15:21:42.617553+0000","last_fullsized":"2024-05-14T15:21:42.617553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665926+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:21:42.665926+0000","last_peered":"2024-05-14T15:21:42.665926+0000","last_clean":"2024-05-14T15:21:42.665926+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:21:42.665926+0000","last_undegraded":"2024-05-14T15:21:42.665926+0000","last_fullsized":"2024-05-14T15:21:42.665926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620752+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:21:42.620752+0000","last_peered":"2024-05-14T15:21:42.620752+0000","last_clean":"2024-05-14T15:21:42.620752+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:21:42.620752+0000","last_undegraded":"2024-05-14T15:21:42.620752+0000","last_fullsized":"2024-05-14T15:21:42.620752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618062+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:21:42.618062+0000","last_peered":"2024-05-14T15:21:42.618062+0000","last_clean":"2024-05-14T15:21:42.618062+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:21:42.618062+0000","last_undegraded":"2024-05-14T15:21:42.618062+0000","last_fullsized":"2024-05-14T15:21:42.618062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663954+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:21:42.663954+0000","last_peered":"2024-05-14T15:21:42.663954+0000","last_clean":"2024-05-14T15:21:42.663954+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:21:42.663954+0000","last_undegraded":"2024-05-14T15:21:42.663954+0000","last_fullsized":"2024-05-14T15:21:42.663954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664291+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:21:42.664291+0000","last_peered":"2024-05-14T15:21:42.664291+0000","last_clean":"2024-05-14T15:21:42.664291+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:21:42.664291+0000","last_undegraded":"2024-05-14T15:21:42.664291+0000","last_fullsized":"2024-05-14T15:21:42.664291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.622265+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:21:42.622265+0000","last_peered":"2024-05-14T15:21:42.622265+0000","last_clean":"2024-05-14T15:21:42.622265+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:21:42.622265+0000","last_undegraded":"2024-05-14T15:21:42.622265+0000","last_fullsized":"2024-05-14T15:21:42.622265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618463+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:21:42.618463+0000","last_peered":"2024-05-14T15:21:42.618463+0000","last_clean":"2024-05-14T15:21:42.618463+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:21:42.618463+0000","last_undegraded":"2024-05-14T15:21:42.618463+0000","last_fullsized":"2024-05-14T15:21:42.618463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617517+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:21:42.617517+0000","last_peered":"2024-05-14T15:21:42.617517+0000","last_clean":"2024-05-14T15:21:42.617517+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:21:42.617517+0000","last_undegraded":"2024-05-14T15:21:42.617517+0000","last_fullsized":"2024-05-14T15:21:42.617517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717018+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:21:42.717018+0000","last_peered":"2024-05-14T15:21:42.717018+0000","last_clean":"2024-05-14T15:21:42.717018+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:21:42.717018+0000","last_undegraded":"2024-05-14T15:21:42.717018+0000","last_fullsized":"2024-05-14T15:21:42.717018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618478+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:21:42.618478+0000","last_peered":"2024-05-14T15:21:42.618478+0000","last_clean":"2024-05-14T15:21:42.618478+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:21:42.618478+0000","last_undegraded":"2024-05-14T15:21:42.618478+0000","last_fullsized":"2024-05-14T15:21:42.618478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618436+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:21:42.618436+0000","last_peered":"2024-05-14T15:21:42.618436+0000","last_clean":"2024-05-14T15:21:42.618436+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:21:42.618436+0000","last_undegraded":"2024-05-14T15:21:42.618436+0000","last_fullsized":"2024-05-14T15:21:42.618436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663598+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:21:42.663598+0000","last_peered":"2024-05-14T15:21:42.663598+0000","last_clean":"2024-05-14T15:21:42.663598+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:21:42.663598+0000","last_undegraded":"2024-05-14T15:21:42.663598+0000","last_fullsized":"2024-05-14T15:21:42.663598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614906+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:21:42.614906+0000","last_peered":"2024-05-14T15:21:42.614906+0000","last_clean":"2024-05-14T15:21:42.614906+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:21:42.614906+0000","last_undegraded":"2024-05-14T15:21:42.614906+0000","last_fullsized":"2024-05-14T15:21:42.614906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614168+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:21:42.614168+0000","last_peered":"2024-05-14T15:21:42.614168+0000","last_clean":"2024-05-14T15:21:42.614168+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:21:42.614168+0000","last_undegraded":"2024-05-14T15:21:42.614168+0000","last_fullsized":"2024-05-14T15:21:42.614168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619728+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:21:42.619728+0000","last_peered":"2024-05-14T15:21:42.619728+0000","last_clean":"2024-05-14T15:21:42.619728+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:21:42.619728+0000","last_undegraded":"2024-05-14T15:21:42.619728+0000","last_fullsized":"2024-05-14T15:21:42.619728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618618+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:21:42.618618+0000","last_peered":"2024-05-14T15:21:42.618618+0000","last_clean":"2024-05-14T15:21:42.618618+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:21:42.618618+0000","last_undegraded":"2024-05-14T15:21:42.618618+0000","last_fullsized":"2024-05-14T15:21:42.618618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770373+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:21:55.770373+0000","last_peered":"2024-05-14T15:21:55.770373+0000","last_clean":"2024-05-14T15:21:55.770373+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:21:55.770373+0000","last_undegraded":"2024-05-14T15:21:55.770373+0000","last_fullsized":"2024-05-14T15:21:55.770373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664268+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:21:42.664268+0000","last_peered":"2024-05-14T15:21:42.664268+0000","last_clean":"2024-05-14T15:21:42.664268+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:21:42.664268+0000","last_undegraded":"2024-05-14T15:21:42.664268+0000","last_fullsized":"2024-05-14T15:21:42.664268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717433+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:21:42.717433+0000","last_peered":"2024-05-14T15:21:42.717433+0000","last_clean":"2024-05-14T15:21:42.717433+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:21:42.717433+0000","last_undegraded":"2024-05-14T15:21:42.717433+0000","last_fullsized":"2024-05-14T15:21:42.717433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664751+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:21:42.664751+0000","last_peered":"2024-05-14T15:21:42.664751+0000","last_clean":"2024-05-14T15:21:42.664751+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:21:42.664751+0000","last_undegraded":"2024-05-14T15:21:42.664751+0000","last_fullsized":"2024-05-14T15:21:42.664751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618666+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:21:42.618666+0000","last_peered":"2024-05-14T15:21:42.618666+0000","last_clean":"2024-05-14T15:21:42.618666+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:21:42.618666+0000","last_undegraded":"2024-05-14T15:21:42.618666+0000","last_fullsized":"2024-05-14T15:21:42.618666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.667773+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:21:42.667773+0000","last_peered":"2024-05-14T15:21:42.667773+0000","last_clean":"2024-05-14T15:21:42.667773+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:21:42.667773+0000","last_undegraded":"2024-05-14T15:21:42.667773+0000","last_fullsized":"2024-05-14T15:21:42.667773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.767496+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:21:55.767496+0000","last_peered":"2024-05-14T15:21:55.767496+0000","last_clean":"2024-05-14T15:21:55.767496+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:21:55.767496+0000","last_undegraded":"2024-05-14T15:21:55.767496+0000","last_fullsized":"2024-05-14T15:21:55.767496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663977+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:21:42.663977+0000","last_peered":"2024-05-14T15:21:42.663977+0000","last_clean":"2024-05-14T15:21:42.663977+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:21:42.663977+0000","last_undegraded":"2024-05-14T15:21:42.663977+0000","last_fullsized":"2024-05-14T15:21:42.663977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.616939+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:21:42.616939+0000","last_peered":"2024-05-14T15:21:42.616939+0000","last_clean":"2024-05-14T15:21:42.616939+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:21:42.616939+0000","last_undegraded":"2024-05-14T15:21:42.616939+0000","last_fullsized":"2024-05-14T15:21:42.616939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.666406+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:21:42.666406+0000","last_peered":"2024-05-14T15:21:42.666406+0000","last_clean":"2024-05-14T15:21:42.666406+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:21:42.666406+0000","last_undegraded":"2024-05-14T15:21:42.666406+0000","last_fullsized":"2024-05-14T15:21:42.666406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664373+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:21:42.664373+0000","last_peered":"2024-05-14T15:21:42.664373+0000","last_clean":"2024-05-14T15:21:42.664373+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:21:42.664373+0000","last_undegraded":"2024-05-14T15:21:42.664373+0000","last_fullsized":"2024-05-14T15:21:42.664373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619449+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:21:42.619449+0000","last_peered":"2024-05-14T15:21:42.619449+0000","last_clean":"2024-05-14T15:21:42.619449+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:21:42.619449+0000","last_undegraded":"2024-05-14T15:21:42.619449+0000","last_fullsized":"2024-05-14T15:21:42.619449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:18.816999+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:22:18.816999+0000","last_peered":"2024-05-14T15:22:18.816999+0000","last_clean":"2024-05-14T15:22:18.816999+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:22:18.816999+0000","last_undegraded":"2024-05-14T15:22:18.816999+0000","last_fullsized":"2024-05-14T15:22:18.816999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665694+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:21:42.665694+0000","last_peered":"2024-05-14T15:21:42.665694+0000","last_clean":"2024-05-14T15:21:42.665694+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:21:42.665694+0000","last_undegraded":"2024-05-14T15:21:42.665694+0000","last_fullsized":"2024-05-14T15:21:42.665694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619209+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:21:42.619209+0000","last_peered":"2024-05-14T15:21:42.619209+0000","last_clean":"2024-05-14T15:21:42.619209+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:21:42.619209+0000","last_undegraded":"2024-05-14T15:21:42.619209+0000","last_fullsized":"2024-05-14T15:21:42.619209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770133+0000","last_change":"2024-05-14T15:20:52.901518+0000","last_active":"2024-05-14T15:21:55.770133+0000","last_peered":"2024-05-14T15:21:55.770133+0000","last_clean":"2024-05-14T15:21:55.770133+0000","last_became_active":"2024-05-14T15:20:52.900367+0000","last_became_peered":"2024-05-14T15:20:52.900367+0000","last_unstale":"2024-05-14T15:21:55.770133+0000","last_undegraded":"2024-05-14T15:21:55.770133+0000","last_fullsized":"2024-05-14T15:21:55.770133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:33.475589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664343+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:21:42.664343+0000","last_peered":"2024-05-14T15:21:42.664343+0000","last_clean":"2024-05-14T15:21:42.664343+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:21:42.664343+0000","last_undegraded":"2024-05-14T15:21:42.664343+0000","last_fullsized":"2024-05-14T15:21:42.664343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.622805+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:21:42.622805+0000","last_peered":"2024-05-14T15:21:42.622805+0000","last_clean":"2024-05-14T15:21:42.622805+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:21:42.622805+0000","last_undegraded":"2024-05-14T15:21:42.622805+0000","last_fullsized":"2024-05-14T15:21:42.622805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.769171+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:21:55.769171+0000","last_peered":"2024-05-14T15:21:55.769171+0000","last_clean":"2024-05-14T15:21:55.769171+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:21:55.769171+0000","last_undegraded":"2024-05-14T15:21:55.769171+0000","last_fullsized":"2024-05-14T15:21:55.769171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664090+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:21:42.664090+0000","last_peered":"2024-05-14T15:21:42.664090+0000","last_clean":"2024-05-14T15:21:42.664090+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:21:42.664090+0000","last_undegraded":"2024-05-14T15:21:42.664090+0000","last_fullsized":"2024-05-14T15:21:42.664090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618819+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:21:42.618819+0000","last_peered":"2024-05-14T15:21:42.618819+0000","last_clean":"2024-05-14T15:21:42.618819+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:21:42.618819+0000","last_undegraded":"2024-05-14T15:21:42.618819+0000","last_fullsized":"2024-05-14T15:21:42.618819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664037+0000","last_change":"2024-05-14T15:20:52.899873+0000","last_active":"2024-05-14T15:21:42.664037+0000","last_peered":"2024-05-14T15:21:42.664037+0000","last_clean":"2024-05-14T15:21:42.664037+0000","last_became_active":"2024-05-14T15:20:52.898667+0000","last_became_peered":"2024-05-14T15:20:52.898667+0000","last_unstale":"2024-05-14T15:21:42.664037+0000","last_undegraded":"2024-05-14T15:21:42.664037+0000","last_fullsized":"2024-05-14T15:21:42.664037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:31.182449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614034+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:21:42.614034+0000","last_peered":"2024-05-14T15:21:42.614034+0000","last_clean":"2024-05-14T15:21:42.614034+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:21:42.614034+0000","last_undegraded":"2024-05-14T15:21:42.614034+0000","last_fullsized":"2024-05-14T15:21:42.614034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663924+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:21:42.663924+0000","last_peered":"2024-05-14T15:21:42.663924+0000","last_clean":"2024-05-14T15:21:42.663924+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:21:42.663924+0000","last_undegraded":"2024-05-14T15:21:42.663924+0000","last_fullsized":"2024-05-14T15:21:42.663924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:19.817746+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:22:19.817746+0000","last_peered":"2024-05-14T15:22:19.817746+0000","last_clean":"2024-05-14T15:22:19.817746+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:22:19.817746+0000","last_undegraded":"2024-05-14T15:22:19.817746+0000","last_fullsized":"2024-05-14T15:22:19.817746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664239+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:21:42.664239+0000","last_peered":"2024-05-14T15:21:42.664239+0000","last_clean":"2024-05-14T15:21:42.664239+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:21:42.664239+0000","last_undegraded":"2024-05-14T15:21:42.664239+0000","last_fullsized":"2024-05-14T15:21:42.664239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.613912+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:21:42.613912+0000","last_peered":"2024-05-14T15:21:42.613912+0000","last_clean":"2024-05-14T15:21:42.613912+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:21:42.613912+0000","last_undegraded":"2024-05-14T15:21:42.613912+0000","last_fullsized":"2024-05-14T15:21:42.613912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.761184+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:21:54.761184+0000","last_peered":"2024-05-14T15:21:54.761184+0000","last_clean":"2024-05-14T15:21:54.761184+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:21:54.761184+0000","last_undegraded":"2024-05-14T15:21:54.761184+0000","last_fullsized":"2024-05-14T15:21:54.761184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.667723+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:21:42.667723+0000","last_peered":"2024-05-14T15:21:42.667723+0000","last_clean":"2024-05-14T15:21:42.667723+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:21:42.667723+0000","last_undegraded":"2024-05-14T15:21:42.667723+0000","last_fullsized":"2024-05-14T15:21:42.667723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617136+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:21:42.617136+0000","last_peered":"2024-05-14T15:21:42.617136+0000","last_clean":"2024-05-14T15:21:42.617136+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:21:42.617136+0000","last_undegraded":"2024-05-14T15:21:42.617136+0000","last_fullsized":"2024-05-14T15:21:42.617136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.763249+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:21:54.763249+0000","last_peered":"2024-05-14T15:21:54.763249+0000","last_clean":"2024-05-14T15:21:54.763249+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:21:54.763249+0000","last_undegraded":"2024-05-14T15:21:54.763249+0000","last_fullsized":"2024-05-14T15:21:54.763249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664111+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:21:42.664111+0000","last_peered":"2024-05-14T15:21:42.664111+0000","last_clean":"2024-05-14T15:21:42.664111+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:21:42.664111+0000","last_undegraded":"2024-05-14T15:21:42.664111+0000","last_fullsized":"2024-05-14T15:21:42.664111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614084+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:21:42.614084+0000","last_peered":"2024-05-14T15:21:42.614084+0000","last_clean":"2024-05-14T15:21:42.614084+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:21:42.614084+0000","last_undegraded":"2024-05-14T15:21:42.614084+0000","last_fullsized":"2024-05-14T15:21:42.614084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:20.819517+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:22:20.819517+0000","last_peered":"2024-05-14T15:22:20.819517+0000","last_clean":"2024-05-14T15:22:20.819517+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:22:20.819517+0000","last_undegraded":"2024-05-14T15:22:20.819517+0000","last_fullsized":"2024-05-14T15:22:20.819517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58392,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":85,"num_write":32,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665971+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:21:42.665971+0000","last_peered":"2024-05-14T15:21:42.665971+0000","last_clean":"2024-05-14T15:21:42.665971+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:21:42.665971+0000","last_undegraded":"2024-05-14T15:21:42.665971+0000","last_fullsized":"2024-05-14T15:21:42.665971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665072+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:21:42.665072+0000","last_peered":"2024-05-14T15:21:42.665072+0000","last_clean":"2024-05-14T15:21:42.665072+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:21:42.665072+0000","last_undegraded":"2024-05-14T15:21:42.665072+0000","last_fullsized":"2024-05-14T15:21:42.665072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.769278+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:21:55.769278+0000","last_peered":"2024-05-14T15:21:55.769278+0000","last_clean":"2024-05-14T15:21:55.769278+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:21:55.769278+0000","last_undegraded":"2024-05-14T15:21:55.769278+0000","last_fullsized":"2024-05-14T15:21:55.769278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716720+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:21:42.716720+0000","last_peered":"2024-05-14T15:21:42.716720+0000","last_clean":"2024-05-14T15:21:42.716720+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:21:42.716720+0000","last_undegraded":"2024-05-14T15:21:42.716720+0000","last_fullsized":"2024-05-14T15:21:42.716720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614582+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:21:42.614582+0000","last_peered":"2024-05-14T15:21:42.614582+0000","last_clean":"2024-05-14T15:21:42.614582+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:21:42.614582+0000","last_undegraded":"2024-05-14T15:21:42.614582+0000","last_fullsized":"2024-05-14T15:21:42.614582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.763074+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:21:54.763074+0000","last_peered":"2024-05-14T15:21:54.763074+0000","last_clean":"2024-05-14T15:21:54.763074+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:21:54.763074+0000","last_undegraded":"2024-05-14T15:21:54.763074+0000","last_fullsized":"2024-05-14T15:21:54.763074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617077+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:21:42.617077+0000","last_peered":"2024-05-14T15:21:42.617077+0000","last_clean":"2024-05-14T15:21:42.617077+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:21:42.617077+0000","last_undegraded":"2024-05-14T15:21:42.617077+0000","last_fullsized":"2024-05-14T15:21:42.617077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614673+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:21:42.614673+0000","last_peered":"2024-05-14T15:21:42.614673+0000","last_clean":"2024-05-14T15:21:42.614673+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:21:42.614673+0000","last_undegraded":"2024-05-14T15:21:42.614673+0000","last_fullsized":"2024-05-14T15:21:42.614673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:19.818896+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:22:19.818896+0000","last_peered":"2024-05-14T15:22:19.818896+0000","last_clean":"2024-05-14T15:22:19.818896+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:22:19.818896+0000","last_undegraded":"2024-05-14T15:22:19.818896+0000","last_fullsized":"2024-05-14T15:22:19.818896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618218+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:21:42.618218+0000","last_peered":"2024-05-14T15:21:42.618218+0000","last_clean":"2024-05-14T15:21:42.618218+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:21:42.618218+0000","last_undegraded":"2024-05-14T15:21:42.618218+0000","last_fullsized":"2024-05-14T15:21:42.618218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717136+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:21:42.717136+0000","last_peered":"2024-05-14T15:21:42.717136+0000","last_clean":"2024-05-14T15:21:42.717136+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:21:42.717136+0000","last_undegraded":"2024-05-14T15:21:42.717136+0000","last_fullsized":"2024-05-14T15:21:42.717136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770005+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:21:55.770005+0000","last_peered":"2024-05-14T15:21:55.770005+0000","last_clean":"2024-05-14T15:21:55.770005+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:21:55.770005+0000","last_undegraded":"2024-05-14T15:21:55.770005+0000","last_fullsized":"2024-05-14T15:21:55.770005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664663+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:21:42.664663+0000","last_peered":"2024-05-14T15:21:42.664663+0000","last_clean":"2024-05-14T15:21:42.664663+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:21:42.664663+0000","last_undegraded":"2024-05-14T15:21:42.664663+0000","last_fullsized":"2024-05-14T15:21:42.664663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664204+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:21:42.664204+0000","last_peered":"2024-05-14T15:21:42.664204+0000","last_clean":"2024-05-14T15:21:42.664204+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:21:42.664204+0000","last_undegraded":"2024-05-14T15:21:42.664204+0000","last_fullsized":"2024-05-14T15:21:42.664204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:57.765127+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:21:57.765127+0000","last_peered":"2024-05-14T15:21:57.765127+0000","last_clean":"2024-05-14T15:21:57.765127+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:21:57.765127+0000","last_undegraded":"2024-05-14T15:21:57.765127+0000","last_fullsized":"2024-05-14T15:21:57.765127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614723+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:21:42.614723+0000","last_peered":"2024-05-14T15:21:42.614723+0000","last_clean":"2024-05-14T15:21:42.614723+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:21:42.614723+0000","last_undegraded":"2024-05-14T15:21:42.614723+0000","last_fullsized":"2024-05-14T15:21:42.614723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665662+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:21:42.665662+0000","last_peered":"2024-05-14T15:21:42.665662+0000","last_clean":"2024-05-14T15:21:42.665662+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:21:42.665662+0000","last_undegraded":"2024-05-14T15:21:42.665662+0000","last_fullsized":"2024-05-14T15:21:42.665662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620211+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:21:42.620211+0000","last_peered":"2024-05-14T15:21:42.620211+0000","last_clean":"2024-05-14T15:21:42.620211+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:21:42.620211+0000","last_undegraded":"2024-05-14T15:21:42.620211+0000","last_fullsized":"2024-05-14T15:21:42.620211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620903+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:21:42.620903+0000","last_peered":"2024-05-14T15:21:42.620903+0000","last_clean":"2024-05-14T15:21:42.620903+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:21:42.620903+0000","last_undegraded":"2024-05-14T15:21:42.620903+0000","last_fullsized":"2024-05-14T15:21:42.620903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716869+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:21:42.716869+0000","last_peered":"2024-05-14T15:21:42.716869+0000","last_clean":"2024-05-14T15:21:42.716869+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:21:42.716869+0000","last_undegraded":"2024-05-14T15:21:42.716869+0000","last_fullsized":"2024-05-14T15:21:42.716869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618811+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:21:42.618811+0000","last_peered":"2024-05-14T15:21:42.618811+0000","last_clean":"2024-05-14T15:21:42.618811+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:21:42.618811+0000","last_undegraded":"2024-05-14T15:21:42.618811+0000","last_fullsized":"2024-05-14T15:21:42.618811+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620723+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:21:42.620723+0000","last_peered":"2024-05-14T15:21:42.620723+0000","last_clean":"2024-05-14T15:21:42.620723+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:21:42.620723+0000","last_undegraded":"2024-05-14T15:21:42.620723+0000","last_fullsized":"2024-05-14T15:21:42.620723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717002+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:21:42.717002+0000","last_peered":"2024-05-14T15:21:42.717002+0000","last_clean":"2024-05-14T15:21:42.717002+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:21:42.717002+0000","last_undegraded":"2024-05-14T15:21:42.717002+0000","last_fullsized":"2024-05-14T15:21:42.717002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664483+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:21:42.664483+0000","last_peered":"2024-05-14T15:21:42.664483+0000","last_clean":"2024-05-14T15:21:42.664483+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:21:42.664483+0000","last_undegraded":"2024-05-14T15:21:42.664483+0000","last_fullsized":"2024-05-14T15:21:42.664483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:18.816382+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:22:18.816382+0000","last_peered":"2024-05-14T15:22:18.816382+0000","last_clean":"2024-05-14T15:22:18.816382+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:22:18.816382+0000","last_undegraded":"2024-05-14T15:22:18.816382+0000","last_fullsized":"2024-05-14T15:22:18.816382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618980+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:21:42.618980+0000","last_peered":"2024-05-14T15:21:42.618980+0000","last_clean":"2024-05-14T15:21:42.618980+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:21:42.618980+0000","last_undegraded":"2024-05-14T15:21:42.618980+0000","last_fullsized":"2024-05-14T15:21:42.618980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664143+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:21:42.664143+0000","last_peered":"2024-05-14T15:21:42.664143+0000","last_clean":"2024-05-14T15:21:42.664143+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:21:42.664143+0000","last_undegraded":"2024-05-14T15:21:42.664143+0000","last_fullsized":"2024-05-14T15:21:42.664143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:57.782976+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:21:57.782976+0000","last_peered":"2024-05-14T15:21:57.782976+0000","last_clean":"2024-05-14T15:21:57.782976+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:21:57.782976+0000","last_undegraded":"2024-05-14T15:21:57.782976+0000","last_fullsized":"2024-05-14T15:21:57.782976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618018+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:21:42.618018+0000","last_peered":"2024-05-14T15:21:42.618018+0000","last_clean":"2024-05-14T15:21:42.618018+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:21:42.618018+0000","last_undegraded":"2024-05-14T15:21:42.618018+0000","last_fullsized":"2024-05-14T15:21:42.618018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664968+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:21:42.664968+0000","last_peered":"2024-05-14T15:21:42.664968+0000","last_clean":"2024-05-14T15:21:42.664968+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:21:42.664968+0000","last_undegraded":"2024-05-14T15:21:42.664968+0000","last_fullsized":"2024-05-14T15:21:42.664968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716440+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:21:42.716440+0000","last_peered":"2024-05-14T15:21:42.716440+0000","last_clean":"2024-05-14T15:21:42.716440+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:21:42.716440+0000","last_undegraded":"2024-05-14T15:21:42.716440+0000","last_fullsized":"2024-05-14T15:21:42.716440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770222+0000","last_change":"2024-05-14T15:20:52.895934+0000","last_active":"2024-05-14T15:21:55.770222+0000","last_peered":"2024-05-14T15:21:55.770222+0000","last_clean":"2024-05-14T15:21:55.770222+0000","last_became_active":"2024-05-14T15:20:52.895438+0000","last_became_peered":"2024-05-14T15:20:52.895438+0000","last_unstale":"2024-05-14T15:21:55.770222+0000","last_undegraded":"2024-05-14T15:21:55.770222+0000","last_fullsized":"2024-05-14T15:21:55.770222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:33:23.453484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614530+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:21:42.614530+0000","last_peered":"2024-05-14T15:21:42.614530+0000","last_clean":"2024-05-14T15:21:42.614530+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:21:42.614530+0000","last_undegraded":"2024-05-14T15:21:42.614530+0000","last_fullsized":"2024-05-14T15:21:42.614530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.760392+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:21:54.760392+0000","last_peered":"2024-05-14T15:21:54.760392+0000","last_clean":"2024-05-14T15:21:54.760392+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:21:54.760392+0000","last_undegraded":"2024-05-14T15:21:54.760392+0000","last_fullsized":"2024-05-14T15:21:54.760392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717022+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:21:42.717022+0000","last_peered":"2024-05-14T15:21:42.717022+0000","last_clean":"2024-05-14T15:21:42.717022+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:21:42.717022+0000","last_undegraded":"2024-05-14T15:21:42.717022+0000","last_fullsized":"2024-05-14T15:21:42.717022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663962+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:21:42.663962+0000","last_peered":"2024-05-14T15:21:42.663962+0000","last_clean":"2024-05-14T15:21:42.663962+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:21:42.663962+0000","last_undegraded":"2024-05-14T15:21:42.663962+0000","last_fullsized":"2024-05-14T15:21:42.663962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:15.937065+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:22:15.937065+0000","last_peered":"2024-05-14T15:22:15.937065+0000","last_clean":"2024-05-14T15:22:15.937065+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:22:15.937065+0000","last_undegraded":"2024-05-14T15:22:15.937065+0000","last_fullsized":"2024-05-14T15:22:15.937065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664871+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:21:42.664871+0000","last_peered":"2024-05-14T15:21:42.664871+0000","last_clean":"2024-05-14T15:21:42.664871+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:21:42.664871+0000","last_undegraded":"2024-05-14T15:21:42.664871+0000","last_fullsized":"2024-05-14T15:21:42.664871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618618+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:21:42.618618+0000","last_peered":"2024-05-14T15:21:42.618618+0000","last_clean":"2024-05-14T15:21:42.618618+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:21:42.618618+0000","last_undegraded":"2024-05-14T15:21:42.618618+0000","last_fullsized":"2024-05-14T15:21:42.618618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.759260+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:21:54.759260+0000","last_peered":"2024-05-14T15:21:54.759260+0000","last_clean":"2024-05-14T15:21:54.759260+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:21:54.759260+0000","last_undegraded":"2024-05-14T15:21:54.759260+0000","last_fullsized":"2024-05-14T15:21:54.759260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.718180+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:21:42.718180+0000","last_peered":"2024-05-14T15:21:42.718180+0000","last_clean":"2024-05-14T15:21:42.718180+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:21:42.718180+0000","last_undegraded":"2024-05-14T15:21:42.718180+0000","last_fullsized":"2024-05-14T15:21:42.718180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617095+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:21:42.617095+0000","last_peered":"2024-05-14T15:21:42.617095+0000","last_clean":"2024-05-14T15:21:42.617095+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:21:42.617095+0000","last_undegraded":"2024-05-14T15:21:42.617095+0000","last_fullsized":"2024-05-14T15:21:42.617095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.760917+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:21:54.760917+0000","last_peered":"2024-05-14T15:21:54.760917+0000","last_clean":"2024-05-14T15:21:54.760917+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:21:54.760917+0000","last_undegraded":"2024-05-14T15:21:54.760917+0000","last_fullsized":"2024-05-14T15:21:54.760917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665704+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:21:42.665704+0000","last_peered":"2024-05-14T15:21:42.665704+0000","last_clean":"2024-05-14T15:21:42.665704+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:21:42.665704+0000","last_undegraded":"2024-05-14T15:21:42.665704+0000","last_fullsized":"2024-05-14T15:21:42.665704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614800+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:21:42.614800+0000","last_peered":"2024-05-14T15:21:42.614800+0000","last_clean":"2024-05-14T15:21:42.614800+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:21:42.614800+0000","last_undegraded":"2024-05-14T15:21:42.614800+0000","last_fullsized":"2024-05-14T15:21:42.614800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.763413+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:21:54.763413+0000","last_peered":"2024-05-14T15:21:54.763413+0000","last_clean":"2024-05-14T15:21:54.763413+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:21:54.763413+0000","last_undegraded":"2024-05-14T15:21:54.763413+0000","last_fullsized":"2024-05-14T15:21:54.763413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614376+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:21:42.614376+0000","last_peered":"2024-05-14T15:21:42.614376+0000","last_clean":"2024-05-14T15:21:42.614376+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:21:42.614376+0000","last_undegraded":"2024-05-14T15:21:42.614376+0000","last_fullsized":"2024-05-14T15:21:42.614376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617169+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:21:42.617169+0000","last_peered":"2024-05-14T15:21:42.617169+0000","last_clean":"2024-05-14T15:21:42.617169+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:21:42.617169+0000","last_undegraded":"2024-05-14T15:21:42.617169+0000","last_fullsized":"2024-05-14T15:21:42.617169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618811+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:21:42.618811+0000","last_peered":"2024-05-14T15:21:42.618811+0000","last_clean":"2024-05-14T15:21:42.618811+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:21:42.618811+0000","last_undegraded":"2024-05-14T15:21:42.618811+0000","last_fullsized":"2024-05-14T15:21:42.618811+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664412+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:21:42.664412+0000","last_peered":"2024-05-14T15:21:42.664412+0000","last_clean":"2024-05-14T15:21:42.664412+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:21:42.664412+0000","last_undegraded":"2024-05-14T15:21:42.664412+0000","last_fullsized":"2024-05-14T15:21:42.664412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.616931+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:21:42.616931+0000","last_peered":"2024-05-14T15:21:42.616931+0000","last_clean":"2024-05-14T15:21:42.616931+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:21:42.616931+0000","last_undegraded":"2024-05-14T15:21:42.616931+0000","last_fullsized":"2024-05-14T15:21:42.616931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.762051+0000","last_change":"2024-05-14T15:20:52.898099+0000","last_active":"2024-05-14T15:21:54.762051+0000","last_peered":"2024-05-14T15:21:54.762051+0000","last_clean":"2024-05-14T15:21:54.762051+0000","last_became_active":"2024-05-14T15:20:52.897670+0000","last_became_peered":"2024-05-14T15:20:52.897670+0000","last_unstale":"2024-05-14T15:21:54.762051+0000","last_undegraded":"2024-05-14T15:21:54.762051+0000","last_fullsized":"2024-05-14T15:21:54.762051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:05.332248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618751+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:21:42.618751+0000","last_peered":"2024-05-14T15:21:42.618751+0000","last_clean":"2024-05-14T15:21:42.618751+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:21:42.618751+0000","last_undegraded":"2024-05-14T15:21:42.618751+0000","last_fullsized":"2024-05-14T15:21:42.618751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663723+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:21:42.663723+0000","last_peered":"2024-05-14T15:21:42.663723+0000","last_clean":"2024-05-14T15:21:42.663723+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:21:42.663723+0000","last_undegraded":"2024-05-14T15:21:42.663723+0000","last_fullsized":"2024-05-14T15:21:42.663723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.769193+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:21:55.769193+0000","last_peered":"2024-05-14T15:21:55.769193+0000","last_clean":"2024-05-14T15:21:55.769193+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:21:55.769193+0000","last_undegraded":"2024-05-14T15:21:55.769193+0000","last_fullsized":"2024-05-14T15:21:55.769193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619094+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:21:42.619094+0000","last_peered":"2024-05-14T15:21:42.619094+0000","last_clean":"2024-05-14T15:21:42.619094+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:21:42.619094+0000","last_undegraded":"2024-05-14T15:21:42.619094+0000","last_fullsized":"2024-05-14T15:21:42.619094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618603+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:21:42.618603+0000","last_peered":"2024-05-14T15:21:42.618603+0000","last_clean":"2024-05-14T15:21:42.618603+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:21:42.618603+0000","last_undegraded":"2024-05-14T15:21:42.618603+0000","last_fullsized":"2024-05-14T15:21:42.618603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664764+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:21:42.664764+0000","last_peered":"2024-05-14T15:21:42.664764+0000","last_clean":"2024-05-14T15:21:42.664764+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:21:42.664764+0000","last_undegraded":"2024-05-14T15:21:42.664764+0000","last_fullsized":"2024-05-14T15:21:42.664764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618102+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:21:42.618102+0000","last_peered":"2024-05-14T15:21:42.618102+0000","last_clean":"2024-05-14T15:21:42.618102+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:21:42.618102+0000","last_undegraded":"2024-05-14T15:21:42.618102+0000","last_fullsized":"2024-05-14T15:21:42.618102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663709+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:21:42.663709+0000","last_peered":"2024-05-14T15:21:42.663709+0000","last_clean":"2024-05-14T15:21:42.663709+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:21:42.663709+0000","last_undegraded":"2024-05-14T15:21:42.663709+0000","last_fullsized":"2024-05-14T15:21:42.663709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717242+0000","last_change":"2024-05-14T15:20:52.892665+0000","last_active":"2024-05-14T15:21:42.717242+0000","last_peered":"2024-05-14T15:21:42.717242+0000","last_clean":"2024-05-14T15:21:42.717242+0000","last_became_active":"2024-05-14T15:20:52.892225+0000","last_became_peered":"2024-05-14T15:20:52.892225+0000","last_unstale":"2024-05-14T15:21:42.717242+0000","last_undegraded":"2024-05-14T15:21:42.717242+0000","last_fullsized":"2024-05-14T15:21:42.717242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:50.934857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617566+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:21:42.617566+0000","last_peered":"2024-05-14T15:21:42.617566+0000","last_clean":"2024-05-14T15:21:42.617566+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:21:42.617566+0000","last_undegraded":"2024-05-14T15:21:42.617566+0000","last_fullsized":"2024-05-14T15:21:42.617566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614544+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:21:42.614544+0000","last_peered":"2024-05-14T15:21:42.614544+0000","last_clean":"2024-05-14T15:21:42.614544+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:21:42.614544+0000","last_undegraded":"2024-05-14T15:21:42.614544+0000","last_fullsized":"2024-05-14T15:21:42.614544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619097+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:21:42.619097+0000","last_peered":"2024-05-14T15:21:42.619097+0000","last_clean":"2024-05-14T15:21:42.619097+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:21:42.619097+0000","last_undegraded":"2024-05-14T15:21:42.619097+0000","last_fullsized":"2024-05-14T15:21:42.619097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.621443+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:21:42.621443+0000","last_peered":"2024-05-14T15:21:42.621443+0000","last_clean":"2024-05-14T15:21:42.621443+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:21:42.621443+0000","last_undegraded":"2024-05-14T15:21:42.621443+0000","last_fullsized":"2024-05-14T15:21:42.621443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618880+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:21:42.618880+0000","last_peered":"2024-05-14T15:21:42.618880+0000","last_clean":"2024-05-14T15:21:42.618880+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:21:42.618880+0000","last_undegraded":"2024-05-14T15:21:42.618880+0000","last_fullsized":"2024-05-14T15:21:42.618880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717039+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:21:42.717039+0000","last_peered":"2024-05-14T15:21:42.717039+0000","last_clean":"2024-05-14T15:21:42.717039+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:21:42.717039+0000","last_undegraded":"2024-05-14T15:21:42.717039+0000","last_fullsized":"2024-05-14T15:21:42.717039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.667394+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:21:42.667394+0000","last_peered":"2024-05-14T15:21:42.667394+0000","last_clean":"2024-05-14T15:21:42.667394+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:21:42.667394+0000","last_undegraded":"2024-05-14T15:21:42.667394+0000","last_fullsized":"2024-05-14T15:21:42.667394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619717+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:21:42.619717+0000","last_peered":"2024-05-14T15:21:42.619717+0000","last_clean":"2024-05-14T15:21:42.619717+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:21:42.619717+0000","last_undegraded":"2024-05-14T15:21:42.619717+0000","last_fullsized":"2024-05-14T15:21:42.619717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664958+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:21:42.664958+0000","last_peered":"2024-05-14T15:21:42.664958+0000","last_clean":"2024-05-14T15:21:42.664958+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:21:42.664958+0000","last_undegraded":"2024-05-14T15:21:42.664958+0000","last_fullsized":"2024-05-14T15:21:42.664958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664109+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:21:42.664109+0000","last_peered":"2024-05-14T15:21:42.664109+0000","last_clean":"2024-05-14T15:21:42.664109+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:21:42.664109+0000","last_undegraded":"2024-05-14T15:21:42.664109+0000","last_fullsized":"2024-05-14T15:21:42.664109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664071+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:21:42.664071+0000","last_peered":"2024-05-14T15:21:42.664071+0000","last_clean":"2024-05-14T15:21:42.664071+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:21:42.664071+0000","last_undegraded":"2024-05-14T15:21:42.664071+0000","last_fullsized":"2024-05-14T15:21:42.664071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664066+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:21:42.664066+0000","last_peered":"2024-05-14T15:21:42.664066+0000","last_clean":"2024-05-14T15:21:42.664066+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:21:42.664066+0000","last_undegraded":"2024-05-14T15:21:42.664066+0000","last_fullsized":"2024-05-14T15:21:42.664066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716911+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:21:42.716911+0000","last_peered":"2024-05-14T15:21:42.716911+0000","last_clean":"2024-05-14T15:21:42.716911+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:21:42.716911+0000","last_undegraded":"2024-05-14T15:21:42.716911+0000","last_fullsized":"2024-05-14T15:21:42.716911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614696+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:21:42.614696+0000","last_peered":"2024-05-14T15:21:42.614696+0000","last_clean":"2024-05-14T15:21:42.614696+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:21:42.614696+0000","last_undegraded":"2024-05-14T15:21:42.614696+0000","last_fullsized":"2024-05-14T15:21:42.614696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283950,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62393,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":169,"num_read_kb":151,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":499220,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874277,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1143742,"data_compressed":62526,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037803,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":503552,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":499376,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1095285,"data_compressed":62526,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":494632,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":552934,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":496020,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":553537,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":494826,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":499282,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1090873,"data_compressed":62526,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":59970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:22:24.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T15:22:24.649 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:25.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:24 smithi005 ceph-mon[31559]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T15:22:25.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:24 smithi005 ceph-mon[31559]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:22:25.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:24 smithi144 ceph-mon[35650]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T15:22:25.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:24 smithi144 ceph-mon[35650]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:22:25.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:24 smithi193 ceph-mon[36001]: pgmap v339: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T15:22:25.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:24 smithi193 ceph-mon[36001]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 -- 172.21.15.5:0/2040124140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 msgr2=0x7fba641038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 --2- 172.21.15.5:0/2040124140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba641038d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fba540099c0 tx=0x7fba54009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 -- 172.21.15.5:0/2040124140 shutdown_connections 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 --2- 172.21.15.5:0/2040124140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba64108740 0x7fba641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 --2- 172.21.15.5:0/2040124140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba641038d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 --2- 172.21.15.5:0/2040124140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba64107df0 0x7fba64108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 -- 172.21.15.5:0/2040124140 >> 172.21.15.5:0/2040124140 conn(0x7fba640fedc0 msgr2=0x7fba641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 -- 172.21.15.5:0/2040124140 shutdown_connections 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 -- 172.21.15.5:0/2040124140 wait complete. 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 Processor -- start 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.343+0000 7fba6c793700 1 -- start start 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6c793700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba6419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6c793700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba64107df0 0x7fba6419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6c793700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba64108740 0x7fba641a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6c793700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba641144c0 con 0x7fba641034c0 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6c793700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba64114340 con 0x7fba64107df0 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6c793700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba64114640 con 0x7fba64108740 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba6419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba6419d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38038/0 (socket says 172.21.15.5:38038) 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 -- 172.21.15.5:0/1216574953 learned_addr learned my addr 172.21.15.5:0/1216574953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6ad30700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba64108740 0x7fba641a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba69d2e700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba64107df0 0x7fba6419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 -- 172.21.15.5:0/1216574953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba64108740 msgr2=0x7fba641a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba64108740 0x7fba641a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 -- 172.21.15.5:0/1216574953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba64107df0 msgr2=0x7fba6419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba64107df0 0x7fba6419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.344+0000 7fba6a52f700 1 -- 172.21.15.5:0/1216574953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba641a1fc0 con 0x7fba641034c0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.345+0000 7fba6a52f700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba6419d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fba5400b580 tx=0x7fba540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.345+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54027070 con 0x7fba641034c0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.345+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba54003f20 con 0x7fba641034c0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.345+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54022d70 con 0x7fba641034c0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.345+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba641a2250 con 0x7fba641034c0 2024-05-14T15:22:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.345+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba641a26e0 con 0x7fba641034c0 2024-05-14T15:22:26.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.346+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba640512c0 con 0x7fba641034c0 2024-05-14T15:22:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.347+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fba54022500 con 0x7fba641034c0 2024-05-14T15:22:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.348+0000 7fba637fe700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fba4c0b2910 0x7fba4c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.348+0000 7fba69d2e700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fba4c0b2910 0x7fba4c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.348+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fba540e3710 con 0x7fba641034c0 2024-05-14T15:22:26.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.349+0000 7fba69d2e700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fba4c0b2910 0x7fba4c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba58005950 tx=0x7fba580058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:26.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.352+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba54013070 con 0x7fba641034c0 2024-05-14T15:22:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.562+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba6410f370 con 0x7fba4c0b2910 2024-05-14T15:22:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.572+0000 7fba637fe700 1 -- 172.21.15.5:0/1216574953 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340615 (secure 0 0 0) 0x7fba6410f370 con 0x7fba4c0b2910 2024-05-14T15:22:26.577 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fba4c0b2910 msgr2=0x7fba4c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fba4c0b2910 0x7fba4c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba58005950 tx=0x7fba580058e0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 msgr2=0x7fba6419d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba6419d0f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fba5400b580 tx=0x7fba540047b0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 shutdown_connections 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fba4c0b2910 0x7fba4c0b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba64108740 0x7fba641a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba641034c0 0x7fba6419d0f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 --2- 172.21.15.5:0/1216574953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba64107df0 0x7fba6419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.576+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 >> 172.21.15.5:0/1216574953 conn(0x7fba640fedc0 msgr2=0x7fba64106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.577+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 shutdown_connections 2024-05-14T15:22:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:26.577+0000 7fba6c793700 1 -- 172.21.15.5:0/1216574953 wait complete. 2024-05-14T15:22:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:22:27.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:26 smithi005 ceph-mon[31559]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T15:22:27.277 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-05-14T15:22:25.485266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652995,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":255,"num_read_kb":225,"num_write":290,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377556,"kb_used_data":11764,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564268,"statfs":{"total":1151940427776,"available":1151553810432,"internally_reserved":0,"allocated":12046336,"data_stored":7924417,"data_compressed":425394,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28164,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":64,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003758"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.615259+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:21:42.615259+0000","last_peered":"2024-05-14T15:21:42.615259+0000","last_clean":"2024-05-14T15:21:42.615259+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:21:42.615259+0000","last_undegraded":"2024-05-14T15:21:42.615259+0000","last_fullsized":"2024-05-14T15:21:42.615259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619055+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:21:42.619055+0000","last_peered":"2024-05-14T15:21:42.619055+0000","last_clean":"2024-05-14T15:21:42.619055+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:21:42.619055+0000","last_undegraded":"2024-05-14T15:21:42.619055+0000","last_fullsized":"2024-05-14T15:21:42.619055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618843+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:21:42.618843+0000","last_peered":"2024-05-14T15:21:42.618843+0000","last_clean":"2024-05-14T15:21:42.618843+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:21:42.618843+0000","last_undegraded":"2024-05-14T15:21:42.618843+0000","last_fullsized":"2024-05-14T15:21:42.618843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617553+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:21:42.617553+0000","last_peered":"2024-05-14T15:21:42.617553+0000","last_clean":"2024-05-14T15:21:42.617553+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:21:42.617553+0000","last_undegraded":"2024-05-14T15:21:42.617553+0000","last_fullsized":"2024-05-14T15:21:42.617553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665926+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:21:42.665926+0000","last_peered":"2024-05-14T15:21:42.665926+0000","last_clean":"2024-05-14T15:21:42.665926+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:21:42.665926+0000","last_undegraded":"2024-05-14T15:21:42.665926+0000","last_fullsized":"2024-05-14T15:21:42.665926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620752+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:21:42.620752+0000","last_peered":"2024-05-14T15:21:42.620752+0000","last_clean":"2024-05-14T15:21:42.620752+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:21:42.620752+0000","last_undegraded":"2024-05-14T15:21:42.620752+0000","last_fullsized":"2024-05-14T15:21:42.620752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618062+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:21:42.618062+0000","last_peered":"2024-05-14T15:21:42.618062+0000","last_clean":"2024-05-14T15:21:42.618062+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:21:42.618062+0000","last_undegraded":"2024-05-14T15:21:42.618062+0000","last_fullsized":"2024-05-14T15:21:42.618062+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663954+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:21:42.663954+0000","last_peered":"2024-05-14T15:21:42.663954+0000","last_clean":"2024-05-14T15:21:42.663954+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:21:42.663954+0000","last_undegraded":"2024-05-14T15:21:42.663954+0000","last_fullsized":"2024-05-14T15:21:42.663954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664291+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:21:42.664291+0000","last_peered":"2024-05-14T15:21:42.664291+0000","last_clean":"2024-05-14T15:21:42.664291+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:21:42.664291+0000","last_undegraded":"2024-05-14T15:21:42.664291+0000","last_fullsized":"2024-05-14T15:21:42.664291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.622265+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:21:42.622265+0000","last_peered":"2024-05-14T15:21:42.622265+0000","last_clean":"2024-05-14T15:21:42.622265+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:21:42.622265+0000","last_undegraded":"2024-05-14T15:21:42.622265+0000","last_fullsized":"2024-05-14T15:21:42.622265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618463+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:21:42.618463+0000","last_peered":"2024-05-14T15:21:42.618463+0000","last_clean":"2024-05-14T15:21:42.618463+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:21:42.618463+0000","last_undegraded":"2024-05-14T15:21:42.618463+0000","last_fullsized":"2024-05-14T15:21:42.618463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617517+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:21:42.617517+0000","last_peered":"2024-05-14T15:21:42.617517+0000","last_clean":"2024-05-14T15:21:42.617517+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:21:42.617517+0000","last_undegraded":"2024-05-14T15:21:42.617517+0000","last_fullsized":"2024-05-14T15:21:42.617517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717018+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:21:42.717018+0000","last_peered":"2024-05-14T15:21:42.717018+0000","last_clean":"2024-05-14T15:21:42.717018+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:21:42.717018+0000","last_undegraded":"2024-05-14T15:21:42.717018+0000","last_fullsized":"2024-05-14T15:21:42.717018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618478+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:21:42.618478+0000","last_peered":"2024-05-14T15:21:42.618478+0000","last_clean":"2024-05-14T15:21:42.618478+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:21:42.618478+0000","last_undegraded":"2024-05-14T15:21:42.618478+0000","last_fullsized":"2024-05-14T15:21:42.618478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618436+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:21:42.618436+0000","last_peered":"2024-05-14T15:21:42.618436+0000","last_clean":"2024-05-14T15:21:42.618436+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:21:42.618436+0000","last_undegraded":"2024-05-14T15:21:42.618436+0000","last_fullsized":"2024-05-14T15:21:42.618436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663598+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:21:42.663598+0000","last_peered":"2024-05-14T15:21:42.663598+0000","last_clean":"2024-05-14T15:21:42.663598+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:21:42.663598+0000","last_undegraded":"2024-05-14T15:21:42.663598+0000","last_fullsized":"2024-05-14T15:21:42.663598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614906+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:21:42.614906+0000","last_peered":"2024-05-14T15:21:42.614906+0000","last_clean":"2024-05-14T15:21:42.614906+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:21:42.614906+0000","last_undegraded":"2024-05-14T15:21:42.614906+0000","last_fullsized":"2024-05-14T15:21:42.614906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614168+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:21:42.614168+0000","last_peered":"2024-05-14T15:21:42.614168+0000","last_clean":"2024-05-14T15:21:42.614168+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:21:42.614168+0000","last_undegraded":"2024-05-14T15:21:42.614168+0000","last_fullsized":"2024-05-14T15:21:42.614168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619728+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:21:42.619728+0000","last_peered":"2024-05-14T15:21:42.619728+0000","last_clean":"2024-05-14T15:21:42.619728+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:21:42.619728+0000","last_undegraded":"2024-05-14T15:21:42.619728+0000","last_fullsized":"2024-05-14T15:21:42.619728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618618+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:21:42.618618+0000","last_peered":"2024-05-14T15:21:42.618618+0000","last_clean":"2024-05-14T15:21:42.618618+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:21:42.618618+0000","last_undegraded":"2024-05-14T15:21:42.618618+0000","last_fullsized":"2024-05-14T15:21:42.618618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770373+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:21:55.770373+0000","last_peered":"2024-05-14T15:21:55.770373+0000","last_clean":"2024-05-14T15:21:55.770373+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:21:55.770373+0000","last_undegraded":"2024-05-14T15:21:55.770373+0000","last_fullsized":"2024-05-14T15:21:55.770373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664268+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:21:42.664268+0000","last_peered":"2024-05-14T15:21:42.664268+0000","last_clean":"2024-05-14T15:21:42.664268+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:21:42.664268+0000","last_undegraded":"2024-05-14T15:21:42.664268+0000","last_fullsized":"2024-05-14T15:21:42.664268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717433+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:21:42.717433+0000","last_peered":"2024-05-14T15:21:42.717433+0000","last_clean":"2024-05-14T15:21:42.717433+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:21:42.717433+0000","last_undegraded":"2024-05-14T15:21:42.717433+0000","last_fullsized":"2024-05-14T15:21:42.717433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664751+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:21:42.664751+0000","last_peered":"2024-05-14T15:21:42.664751+0000","last_clean":"2024-05-14T15:21:42.664751+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:21:42.664751+0000","last_undegraded":"2024-05-14T15:21:42.664751+0000","last_fullsized":"2024-05-14T15:21:42.664751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618666+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:21:42.618666+0000","last_peered":"2024-05-14T15:21:42.618666+0000","last_clean":"2024-05-14T15:21:42.618666+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:21:42.618666+0000","last_undegraded":"2024-05-14T15:21:42.618666+0000","last_fullsized":"2024-05-14T15:21:42.618666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.667773+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:21:42.667773+0000","last_peered":"2024-05-14T15:21:42.667773+0000","last_clean":"2024-05-14T15:21:42.667773+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:21:42.667773+0000","last_undegraded":"2024-05-14T15:21:42.667773+0000","last_fullsized":"2024-05-14T15:21:42.667773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.767496+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:21:55.767496+0000","last_peered":"2024-05-14T15:21:55.767496+0000","last_clean":"2024-05-14T15:21:55.767496+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:21:55.767496+0000","last_undegraded":"2024-05-14T15:21:55.767496+0000","last_fullsized":"2024-05-14T15:21:55.767496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663977+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:21:42.663977+0000","last_peered":"2024-05-14T15:21:42.663977+0000","last_clean":"2024-05-14T15:21:42.663977+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:21:42.663977+0000","last_undegraded":"2024-05-14T15:21:42.663977+0000","last_fullsized":"2024-05-14T15:21:42.663977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.616939+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:21:42.616939+0000","last_peered":"2024-05-14T15:21:42.616939+0000","last_clean":"2024-05-14T15:21:42.616939+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:21:42.616939+0000","last_undegraded":"2024-05-14T15:21:42.616939+0000","last_fullsized":"2024-05-14T15:21:42.616939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.666406+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:21:42.666406+0000","last_peered":"2024-05-14T15:21:42.666406+0000","last_clean":"2024-05-14T15:21:42.666406+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:21:42.666406+0000","last_undegraded":"2024-05-14T15:21:42.666406+0000","last_fullsized":"2024-05-14T15:21:42.666406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664373+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:21:42.664373+0000","last_peered":"2024-05-14T15:21:42.664373+0000","last_clean":"2024-05-14T15:21:42.664373+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:21:42.664373+0000","last_undegraded":"2024-05-14T15:21:42.664373+0000","last_fullsized":"2024-05-14T15:21:42.664373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619449+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:21:42.619449+0000","last_peered":"2024-05-14T15:21:42.619449+0000","last_clean":"2024-05-14T15:21:42.619449+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:21:42.619449+0000","last_undegraded":"2024-05-14T15:21:42.619449+0000","last_fullsized":"2024-05-14T15:21:42.619449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:18.816999+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:22:18.816999+0000","last_peered":"2024-05-14T15:22:18.816999+0000","last_clean":"2024-05-14T15:22:18.816999+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:22:18.816999+0000","last_undegraded":"2024-05-14T15:22:18.816999+0000","last_fullsized":"2024-05-14T15:22:18.816999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665694+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:21:42.665694+0000","last_peered":"2024-05-14T15:21:42.665694+0000","last_clean":"2024-05-14T15:21:42.665694+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:21:42.665694+0000","last_undegraded":"2024-05-14T15:21:42.665694+0000","last_fullsized":"2024-05-14T15:21:42.665694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619209+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:21:42.619209+0000","last_peered":"2024-05-14T15:21:42.619209+0000","last_clean":"2024-05-14T15:21:42.619209+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:21:42.619209+0000","last_undegraded":"2024-05-14T15:21:42.619209+0000","last_fullsized":"2024-05-14T15:21:42.619209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770133+0000","last_change":"2024-05-14T15:20:52.901518+0000","last_active":"2024-05-14T15:21:55.770133+0000","last_peered":"2024-05-14T15:21:55.770133+0000","last_clean":"2024-05-14T15:21:55.770133+0000","last_became_active":"2024-05-14T15:20:52.900367+0000","last_became_peered":"2024-05-14T15:20:52.900367+0000","last_unstale":"2024-05-14T15:21:55.770133+0000","last_undegraded":"2024-05-14T15:21:55.770133+0000","last_fullsized":"2024-05-14T15:21:55.770133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:33.475589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664343+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:21:42.664343+0000","last_peered":"2024-05-14T15:21:42.664343+0000","last_clean":"2024-05-14T15:21:42.664343+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:21:42.664343+0000","last_undegraded":"2024-05-14T15:21:42.664343+0000","last_fullsized":"2024-05-14T15:21:42.664343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.622805+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:21:42.622805+0000","last_peered":"2024-05-14T15:21:42.622805+0000","last_clean":"2024-05-14T15:21:42.622805+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:21:42.622805+0000","last_undegraded":"2024-05-14T15:21:42.622805+0000","last_fullsized":"2024-05-14T15:21:42.622805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.769171+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:21:55.769171+0000","last_peered":"2024-05-14T15:21:55.769171+0000","last_clean":"2024-05-14T15:21:55.769171+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:21:55.769171+0000","last_undegraded":"2024-05-14T15:21:55.769171+0000","last_fullsized":"2024-05-14T15:21:55.769171+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664090+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:21:42.664090+0000","last_peered":"2024-05-14T15:21:42.664090+0000","last_clean":"2024-05-14T15:21:42.664090+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:21:42.664090+0000","last_undegraded":"2024-05-14T15:21:42.664090+0000","last_fullsized":"2024-05-14T15:21:42.664090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618819+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:21:42.618819+0000","last_peered":"2024-05-14T15:21:42.618819+0000","last_clean":"2024-05-14T15:21:42.618819+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:21:42.618819+0000","last_undegraded":"2024-05-14T15:21:42.618819+0000","last_fullsized":"2024-05-14T15:21:42.618819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664037+0000","last_change":"2024-05-14T15:20:52.899873+0000","last_active":"2024-05-14T15:21:42.664037+0000","last_peered":"2024-05-14T15:21:42.664037+0000","last_clean":"2024-05-14T15:21:42.664037+0000","last_became_active":"2024-05-14T15:20:52.898667+0000","last_became_peered":"2024-05-14T15:20:52.898667+0000","last_unstale":"2024-05-14T15:21:42.664037+0000","last_undegraded":"2024-05-14T15:21:42.664037+0000","last_fullsized":"2024-05-14T15:21:42.664037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:31.182449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614034+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:21:42.614034+0000","last_peered":"2024-05-14T15:21:42.614034+0000","last_clean":"2024-05-14T15:21:42.614034+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:21:42.614034+0000","last_undegraded":"2024-05-14T15:21:42.614034+0000","last_fullsized":"2024-05-14T15:21:42.614034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663924+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:21:42.663924+0000","last_peered":"2024-05-14T15:21:42.663924+0000","last_clean":"2024-05-14T15:21:42.663924+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:21:42.663924+0000","last_undegraded":"2024-05-14T15:21:42.663924+0000","last_fullsized":"2024-05-14T15:21:42.663924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:19.817746+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:22:19.817746+0000","last_peered":"2024-05-14T15:22:19.817746+0000","last_clean":"2024-05-14T15:22:19.817746+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:22:19.817746+0000","last_undegraded":"2024-05-14T15:22:19.817746+0000","last_fullsized":"2024-05-14T15:22:19.817746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664239+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:21:42.664239+0000","last_peered":"2024-05-14T15:21:42.664239+0000","last_clean":"2024-05-14T15:21:42.664239+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:21:42.664239+0000","last_undegraded":"2024-05-14T15:21:42.664239+0000","last_fullsized":"2024-05-14T15:21:42.664239+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.613912+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:21:42.613912+0000","last_peered":"2024-05-14T15:21:42.613912+0000","last_clean":"2024-05-14T15:21:42.613912+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:21:42.613912+0000","last_undegraded":"2024-05-14T15:21:42.613912+0000","last_fullsized":"2024-05-14T15:21:42.613912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.761184+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:21:54.761184+0000","last_peered":"2024-05-14T15:21:54.761184+0000","last_clean":"2024-05-14T15:21:54.761184+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:21:54.761184+0000","last_undegraded":"2024-05-14T15:21:54.761184+0000","last_fullsized":"2024-05-14T15:21:54.761184+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.667723+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:21:42.667723+0000","last_peered":"2024-05-14T15:21:42.667723+0000","last_clean":"2024-05-14T15:21:42.667723+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:21:42.667723+0000","last_undegraded":"2024-05-14T15:21:42.667723+0000","last_fullsized":"2024-05-14T15:21:42.667723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617136+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:21:42.617136+0000","last_peered":"2024-05-14T15:21:42.617136+0000","last_clean":"2024-05-14T15:21:42.617136+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:21:42.617136+0000","last_undegraded":"2024-05-14T15:21:42.617136+0000","last_fullsized":"2024-05-14T15:21:42.617136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.763249+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:21:54.763249+0000","last_peered":"2024-05-14T15:21:54.763249+0000","last_clean":"2024-05-14T15:21:54.763249+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:21:54.763249+0000","last_undegraded":"2024-05-14T15:21:54.763249+0000","last_fullsized":"2024-05-14T15:21:54.763249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664111+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:21:42.664111+0000","last_peered":"2024-05-14T15:21:42.664111+0000","last_clean":"2024-05-14T15:21:42.664111+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:21:42.664111+0000","last_undegraded":"2024-05-14T15:21:42.664111+0000","last_fullsized":"2024-05-14T15:21:42.664111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614084+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:21:42.614084+0000","last_peered":"2024-05-14T15:21:42.614084+0000","last_clean":"2024-05-14T15:21:42.614084+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:21:42.614084+0000","last_undegraded":"2024-05-14T15:21:42.614084+0000","last_fullsized":"2024-05-14T15:21:42.614084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:20.819517+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:22:20.819517+0000","last_peered":"2024-05-14T15:22:20.819517+0000","last_clean":"2024-05-14T15:22:20.819517+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:22:20.819517+0000","last_undegraded":"2024-05-14T15:22:20.819517+0000","last_fullsized":"2024-05-14T15:22:20.819517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58392,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":85,"num_write":32,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665971+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:21:42.665971+0000","last_peered":"2024-05-14T15:21:42.665971+0000","last_clean":"2024-05-14T15:21:42.665971+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:21:42.665971+0000","last_undegraded":"2024-05-14T15:21:42.665971+0000","last_fullsized":"2024-05-14T15:21:42.665971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665072+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:21:42.665072+0000","last_peered":"2024-05-14T15:21:42.665072+0000","last_clean":"2024-05-14T15:21:42.665072+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:21:42.665072+0000","last_undegraded":"2024-05-14T15:21:42.665072+0000","last_fullsized":"2024-05-14T15:21:42.665072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.769278+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:21:55.769278+0000","last_peered":"2024-05-14T15:21:55.769278+0000","last_clean":"2024-05-14T15:21:55.769278+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:21:55.769278+0000","last_undegraded":"2024-05-14T15:21:55.769278+0000","last_fullsized":"2024-05-14T15:21:55.769278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716720+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:21:42.716720+0000","last_peered":"2024-05-14T15:21:42.716720+0000","last_clean":"2024-05-14T15:21:42.716720+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:21:42.716720+0000","last_undegraded":"2024-05-14T15:21:42.716720+0000","last_fullsized":"2024-05-14T15:21:42.716720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614582+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:21:42.614582+0000","last_peered":"2024-05-14T15:21:42.614582+0000","last_clean":"2024-05-14T15:21:42.614582+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:21:42.614582+0000","last_undegraded":"2024-05-14T15:21:42.614582+0000","last_fullsized":"2024-05-14T15:21:42.614582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.763074+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:21:54.763074+0000","last_peered":"2024-05-14T15:21:54.763074+0000","last_clean":"2024-05-14T15:21:54.763074+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:21:54.763074+0000","last_undegraded":"2024-05-14T15:21:54.763074+0000","last_fullsized":"2024-05-14T15:21:54.763074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617077+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:21:42.617077+0000","last_peered":"2024-05-14T15:21:42.617077+0000","last_clean":"2024-05-14T15:21:42.617077+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:21:42.617077+0000","last_undegraded":"2024-05-14T15:21:42.617077+0000","last_fullsized":"2024-05-14T15:21:42.617077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614673+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:21:42.614673+0000","last_peered":"2024-05-14T15:21:42.614673+0000","last_clean":"2024-05-14T15:21:42.614673+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:21:42.614673+0000","last_undegraded":"2024-05-14T15:21:42.614673+0000","last_fullsized":"2024-05-14T15:21:42.614673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:19.818896+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:22:19.818896+0000","last_peered":"2024-05-14T15:22:19.818896+0000","last_clean":"2024-05-14T15:22:19.818896+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:22:19.818896+0000","last_undegraded":"2024-05-14T15:22:19.818896+0000","last_fullsized":"2024-05-14T15:22:19.818896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618218+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:21:42.618218+0000","last_peered":"2024-05-14T15:21:42.618218+0000","last_clean":"2024-05-14T15:21:42.618218+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:21:42.618218+0000","last_undegraded":"2024-05-14T15:21:42.618218+0000","last_fullsized":"2024-05-14T15:21:42.618218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717136+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:21:42.717136+0000","last_peered":"2024-05-14T15:21:42.717136+0000","last_clean":"2024-05-14T15:21:42.717136+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:21:42.717136+0000","last_undegraded":"2024-05-14T15:21:42.717136+0000","last_fullsized":"2024-05-14T15:21:42.717136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770005+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:21:55.770005+0000","last_peered":"2024-05-14T15:21:55.770005+0000","last_clean":"2024-05-14T15:21:55.770005+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:21:55.770005+0000","last_undegraded":"2024-05-14T15:21:55.770005+0000","last_fullsized":"2024-05-14T15:21:55.770005+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664663+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:21:42.664663+0000","last_peered":"2024-05-14T15:21:42.664663+0000","last_clean":"2024-05-14T15:21:42.664663+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:21:42.664663+0000","last_undegraded":"2024-05-14T15:21:42.664663+0000","last_fullsized":"2024-05-14T15:21:42.664663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664204+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:21:42.664204+0000","last_peered":"2024-05-14T15:21:42.664204+0000","last_clean":"2024-05-14T15:21:42.664204+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:21:42.664204+0000","last_undegraded":"2024-05-14T15:21:42.664204+0000","last_fullsized":"2024-05-14T15:21:42.664204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:57.765127+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:21:57.765127+0000","last_peered":"2024-05-14T15:21:57.765127+0000","last_clean":"2024-05-14T15:21:57.765127+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:21:57.765127+0000","last_undegraded":"2024-05-14T15:21:57.765127+0000","last_fullsized":"2024-05-14T15:21:57.765127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614723+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:21:42.614723+0000","last_peered":"2024-05-14T15:21:42.614723+0000","last_clean":"2024-05-14T15:21:42.614723+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:21:42.614723+0000","last_undegraded":"2024-05-14T15:21:42.614723+0000","last_fullsized":"2024-05-14T15:21:42.614723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665662+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:21:42.665662+0000","last_peered":"2024-05-14T15:21:42.665662+0000","last_clean":"2024-05-14T15:21:42.665662+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:21:42.665662+0000","last_undegraded":"2024-05-14T15:21:42.665662+0000","last_fullsized":"2024-05-14T15:21:42.665662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620211+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:21:42.620211+0000","last_peered":"2024-05-14T15:21:42.620211+0000","last_clean":"2024-05-14T15:21:42.620211+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:21:42.620211+0000","last_undegraded":"2024-05-14T15:21:42.620211+0000","last_fullsized":"2024-05-14T15:21:42.620211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620903+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:21:42.620903+0000","last_peered":"2024-05-14T15:21:42.620903+0000","last_clean":"2024-05-14T15:21:42.620903+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:21:42.620903+0000","last_undegraded":"2024-05-14T15:21:42.620903+0000","last_fullsized":"2024-05-14T15:21:42.620903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716869+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:21:42.716869+0000","last_peered":"2024-05-14T15:21:42.716869+0000","last_clean":"2024-05-14T15:21:42.716869+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:21:42.716869+0000","last_undegraded":"2024-05-14T15:21:42.716869+0000","last_fullsized":"2024-05-14T15:21:42.716869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618811+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:21:42.618811+0000","last_peered":"2024-05-14T15:21:42.618811+0000","last_clean":"2024-05-14T15:21:42.618811+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:21:42.618811+0000","last_undegraded":"2024-05-14T15:21:42.618811+0000","last_fullsized":"2024-05-14T15:21:42.618811+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.620723+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:21:42.620723+0000","last_peered":"2024-05-14T15:21:42.620723+0000","last_clean":"2024-05-14T15:21:42.620723+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:21:42.620723+0000","last_undegraded":"2024-05-14T15:21:42.620723+0000","last_fullsized":"2024-05-14T15:21:42.620723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717002+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:21:42.717002+0000","last_peered":"2024-05-14T15:21:42.717002+0000","last_clean":"2024-05-14T15:21:42.717002+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:21:42.717002+0000","last_undegraded":"2024-05-14T15:21:42.717002+0000","last_fullsized":"2024-05-14T15:21:42.717002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664483+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:21:42.664483+0000","last_peered":"2024-05-14T15:21:42.664483+0000","last_clean":"2024-05-14T15:21:42.664483+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:21:42.664483+0000","last_undegraded":"2024-05-14T15:21:42.664483+0000","last_fullsized":"2024-05-14T15:21:42.664483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:23.823834+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:22:23.823834+0000","last_peered":"2024-05-14T15:22:23.823834+0000","last_clean":"2024-05-14T15:22:23.823834+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:22:23.823834+0000","last_undegraded":"2024-05-14T15:22:23.823834+0000","last_fullsized":"2024-05-14T15:22:23.823834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618980+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:21:42.618980+0000","last_peered":"2024-05-14T15:21:42.618980+0000","last_clean":"2024-05-14T15:21:42.618980+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:21:42.618980+0000","last_undegraded":"2024-05-14T15:21:42.618980+0000","last_fullsized":"2024-05-14T15:21:42.618980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664143+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:21:42.664143+0000","last_peered":"2024-05-14T15:21:42.664143+0000","last_clean":"2024-05-14T15:21:42.664143+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:21:42.664143+0000","last_undegraded":"2024-05-14T15:21:42.664143+0000","last_fullsized":"2024-05-14T15:21:42.664143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:57.782976+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:21:57.782976+0000","last_peered":"2024-05-14T15:21:57.782976+0000","last_clean":"2024-05-14T15:21:57.782976+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:21:57.782976+0000","last_undegraded":"2024-05-14T15:21:57.782976+0000","last_fullsized":"2024-05-14T15:21:57.782976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618018+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:21:42.618018+0000","last_peered":"2024-05-14T15:21:42.618018+0000","last_clean":"2024-05-14T15:21:42.618018+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:21:42.618018+0000","last_undegraded":"2024-05-14T15:21:42.618018+0000","last_fullsized":"2024-05-14T15:21:42.618018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664968+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:21:42.664968+0000","last_peered":"2024-05-14T15:21:42.664968+0000","last_clean":"2024-05-14T15:21:42.664968+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:21:42.664968+0000","last_undegraded":"2024-05-14T15:21:42.664968+0000","last_fullsized":"2024-05-14T15:21:42.664968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716440+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:21:42.716440+0000","last_peered":"2024-05-14T15:21:42.716440+0000","last_clean":"2024-05-14T15:21:42.716440+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:21:42.716440+0000","last_undegraded":"2024-05-14T15:21:42.716440+0000","last_fullsized":"2024-05-14T15:21:42.716440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.770222+0000","last_change":"2024-05-14T15:20:52.895934+0000","last_active":"2024-05-14T15:21:55.770222+0000","last_peered":"2024-05-14T15:21:55.770222+0000","last_clean":"2024-05-14T15:21:55.770222+0000","last_became_active":"2024-05-14T15:20:52.895438+0000","last_became_peered":"2024-05-14T15:20:52.895438+0000","last_unstale":"2024-05-14T15:21:55.770222+0000","last_undegraded":"2024-05-14T15:21:55.770222+0000","last_fullsized":"2024-05-14T15:21:55.770222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:33:23.453484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614530+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:21:42.614530+0000","last_peered":"2024-05-14T15:21:42.614530+0000","last_clean":"2024-05-14T15:21:42.614530+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:21:42.614530+0000","last_undegraded":"2024-05-14T15:21:42.614530+0000","last_fullsized":"2024-05-14T15:21:42.614530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.760392+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:21:54.760392+0000","last_peered":"2024-05-14T15:21:54.760392+0000","last_clean":"2024-05-14T15:21:54.760392+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:21:54.760392+0000","last_undegraded":"2024-05-14T15:21:54.760392+0000","last_fullsized":"2024-05-14T15:21:54.760392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717022+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:21:42.717022+0000","last_peered":"2024-05-14T15:21:42.717022+0000","last_clean":"2024-05-14T15:21:42.717022+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:21:42.717022+0000","last_undegraded":"2024-05-14T15:21:42.717022+0000","last_fullsized":"2024-05-14T15:21:42.717022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663962+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:21:42.663962+0000","last_peered":"2024-05-14T15:21:42.663962+0000","last_clean":"2024-05-14T15:21:42.663962+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:21:42.663962+0000","last_undegraded":"2024-05-14T15:21:42.663962+0000","last_fullsized":"2024-05-14T15:21:42.663962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:22:15.937065+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:22:15.937065+0000","last_peered":"2024-05-14T15:22:15.937065+0000","last_clean":"2024-05-14T15:22:15.937065+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:22:15.937065+0000","last_undegraded":"2024-05-14T15:22:15.937065+0000","last_fullsized":"2024-05-14T15:22:15.937065+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664871+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:21:42.664871+0000","last_peered":"2024-05-14T15:21:42.664871+0000","last_clean":"2024-05-14T15:21:42.664871+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:21:42.664871+0000","last_undegraded":"2024-05-14T15:21:42.664871+0000","last_fullsized":"2024-05-14T15:21:42.664871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618618+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:21:42.618618+0000","last_peered":"2024-05-14T15:21:42.618618+0000","last_clean":"2024-05-14T15:21:42.618618+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:21:42.618618+0000","last_undegraded":"2024-05-14T15:21:42.618618+0000","last_fullsized":"2024-05-14T15:21:42.618618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.759260+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:21:54.759260+0000","last_peered":"2024-05-14T15:21:54.759260+0000","last_clean":"2024-05-14T15:21:54.759260+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:21:54.759260+0000","last_undegraded":"2024-05-14T15:21:54.759260+0000","last_fullsized":"2024-05-14T15:21:54.759260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.718180+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:21:42.718180+0000","last_peered":"2024-05-14T15:21:42.718180+0000","last_clean":"2024-05-14T15:21:42.718180+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:21:42.718180+0000","last_undegraded":"2024-05-14T15:21:42.718180+0000","last_fullsized":"2024-05-14T15:21:42.718180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617095+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:21:42.617095+0000","last_peered":"2024-05-14T15:21:42.617095+0000","last_clean":"2024-05-14T15:21:42.617095+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:21:42.617095+0000","last_undegraded":"2024-05-14T15:21:42.617095+0000","last_fullsized":"2024-05-14T15:21:42.617095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.760917+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:21:54.760917+0000","last_peered":"2024-05-14T15:21:54.760917+0000","last_clean":"2024-05-14T15:21:54.760917+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:21:54.760917+0000","last_undegraded":"2024-05-14T15:21:54.760917+0000","last_fullsized":"2024-05-14T15:21:54.760917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.665704+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:21:42.665704+0000","last_peered":"2024-05-14T15:21:42.665704+0000","last_clean":"2024-05-14T15:21:42.665704+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:21:42.665704+0000","last_undegraded":"2024-05-14T15:21:42.665704+0000","last_fullsized":"2024-05-14T15:21:42.665704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614800+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:21:42.614800+0000","last_peered":"2024-05-14T15:21:42.614800+0000","last_clean":"2024-05-14T15:21:42.614800+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:21:42.614800+0000","last_undegraded":"2024-05-14T15:21:42.614800+0000","last_fullsized":"2024-05-14T15:21:42.614800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.763413+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:21:54.763413+0000","last_peered":"2024-05-14T15:21:54.763413+0000","last_clean":"2024-05-14T15:21:54.763413+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:21:54.763413+0000","last_undegraded":"2024-05-14T15:21:54.763413+0000","last_fullsized":"2024-05-14T15:21:54.763413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614376+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:21:42.614376+0000","last_peered":"2024-05-14T15:21:42.614376+0000","last_clean":"2024-05-14T15:21:42.614376+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:21:42.614376+0000","last_undegraded":"2024-05-14T15:21:42.614376+0000","last_fullsized":"2024-05-14T15:21:42.614376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617169+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:21:42.617169+0000","last_peered":"2024-05-14T15:21:42.617169+0000","last_clean":"2024-05-14T15:21:42.617169+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:21:42.617169+0000","last_undegraded":"2024-05-14T15:21:42.617169+0000","last_fullsized":"2024-05-14T15:21:42.617169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618811+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:21:42.618811+0000","last_peered":"2024-05-14T15:21:42.618811+0000","last_clean":"2024-05-14T15:21:42.618811+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:21:42.618811+0000","last_undegraded":"2024-05-14T15:21:42.618811+0000","last_fullsized":"2024-05-14T15:21:42.618811+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664412+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:21:42.664412+0000","last_peered":"2024-05-14T15:21:42.664412+0000","last_clean":"2024-05-14T15:21:42.664412+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:21:42.664412+0000","last_undegraded":"2024-05-14T15:21:42.664412+0000","last_fullsized":"2024-05-14T15:21:42.664412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.616931+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:21:42.616931+0000","last_peered":"2024-05-14T15:21:42.616931+0000","last_clean":"2024-05-14T15:21:42.616931+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:21:42.616931+0000","last_undegraded":"2024-05-14T15:21:42.616931+0000","last_fullsized":"2024-05-14T15:21:42.616931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:54.762051+0000","last_change":"2024-05-14T15:20:52.898099+0000","last_active":"2024-05-14T15:21:54.762051+0000","last_peered":"2024-05-14T15:21:54.762051+0000","last_clean":"2024-05-14T15:21:54.762051+0000","last_became_active":"2024-05-14T15:20:52.897670+0000","last_became_peered":"2024-05-14T15:20:52.897670+0000","last_unstale":"2024-05-14T15:21:54.762051+0000","last_undegraded":"2024-05-14T15:21:54.762051+0000","last_fullsized":"2024-05-14T15:21:54.762051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:05.332248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618751+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:21:42.618751+0000","last_peered":"2024-05-14T15:21:42.618751+0000","last_clean":"2024-05-14T15:21:42.618751+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:21:42.618751+0000","last_undegraded":"2024-05-14T15:21:42.618751+0000","last_fullsized":"2024-05-14T15:21:42.618751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663723+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:21:42.663723+0000","last_peered":"2024-05-14T15:21:42.663723+0000","last_clean":"2024-05-14T15:21:42.663723+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:21:42.663723+0000","last_undegraded":"2024-05-14T15:21:42.663723+0000","last_fullsized":"2024-05-14T15:21:42.663723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:55.769193+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:21:55.769193+0000","last_peered":"2024-05-14T15:21:55.769193+0000","last_clean":"2024-05-14T15:21:55.769193+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:21:55.769193+0000","last_undegraded":"2024-05-14T15:21:55.769193+0000","last_fullsized":"2024-05-14T15:21:55.769193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619094+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:21:42.619094+0000","last_peered":"2024-05-14T15:21:42.619094+0000","last_clean":"2024-05-14T15:21:42.619094+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:21:42.619094+0000","last_undegraded":"2024-05-14T15:21:42.619094+0000","last_fullsized":"2024-05-14T15:21:42.619094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618603+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:21:42.618603+0000","last_peered":"2024-05-14T15:21:42.618603+0000","last_clean":"2024-05-14T15:21:42.618603+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:21:42.618603+0000","last_undegraded":"2024-05-14T15:21:42.618603+0000","last_fullsized":"2024-05-14T15:21:42.618603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664764+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:21:42.664764+0000","last_peered":"2024-05-14T15:21:42.664764+0000","last_clean":"2024-05-14T15:21:42.664764+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:21:42.664764+0000","last_undegraded":"2024-05-14T15:21:42.664764+0000","last_fullsized":"2024-05-14T15:21:42.664764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618102+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:21:42.618102+0000","last_peered":"2024-05-14T15:21:42.618102+0000","last_clean":"2024-05-14T15:21:42.618102+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:21:42.618102+0000","last_undegraded":"2024-05-14T15:21:42.618102+0000","last_fullsized":"2024-05-14T15:21:42.618102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.663709+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:21:42.663709+0000","last_peered":"2024-05-14T15:21:42.663709+0000","last_clean":"2024-05-14T15:21:42.663709+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:21:42.663709+0000","last_undegraded":"2024-05-14T15:21:42.663709+0000","last_fullsized":"2024-05-14T15:21:42.663709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717242+0000","last_change":"2024-05-14T15:20:52.892665+0000","last_active":"2024-05-14T15:21:42.717242+0000","last_peered":"2024-05-14T15:21:42.717242+0000","last_clean":"2024-05-14T15:21:42.717242+0000","last_became_active":"2024-05-14T15:20:52.892225+0000","last_became_peered":"2024-05-14T15:20:52.892225+0000","last_unstale":"2024-05-14T15:21:42.717242+0000","last_undegraded":"2024-05-14T15:21:42.717242+0000","last_fullsized":"2024-05-14T15:21:42.717242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:50.934857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.617566+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:21:42.617566+0000","last_peered":"2024-05-14T15:21:42.617566+0000","last_clean":"2024-05-14T15:21:42.617566+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:21:42.617566+0000","last_undegraded":"2024-05-14T15:21:42.617566+0000","last_fullsized":"2024-05-14T15:21:42.617566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614544+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:21:42.614544+0000","last_peered":"2024-05-14T15:21:42.614544+0000","last_clean":"2024-05-14T15:21:42.614544+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:21:42.614544+0000","last_undegraded":"2024-05-14T15:21:42.614544+0000","last_fullsized":"2024-05-14T15:21:42.614544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619097+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:21:42.619097+0000","last_peered":"2024-05-14T15:21:42.619097+0000","last_clean":"2024-05-14T15:21:42.619097+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:21:42.619097+0000","last_undegraded":"2024-05-14T15:21:42.619097+0000","last_fullsized":"2024-05-14T15:21:42.619097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.621443+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:21:42.621443+0000","last_peered":"2024-05-14T15:21:42.621443+0000","last_clean":"2024-05-14T15:21:42.621443+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:21:42.621443+0000","last_undegraded":"2024-05-14T15:21:42.621443+0000","last_fullsized":"2024-05-14T15:21:42.621443+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.618880+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:21:42.618880+0000","last_peered":"2024-05-14T15:21:42.618880+0000","last_clean":"2024-05-14T15:21:42.618880+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:21:42.618880+0000","last_undegraded":"2024-05-14T15:21:42.618880+0000","last_fullsized":"2024-05-14T15:21:42.618880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.717039+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:21:42.717039+0000","last_peered":"2024-05-14T15:21:42.717039+0000","last_clean":"2024-05-14T15:21:42.717039+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:21:42.717039+0000","last_undegraded":"2024-05-14T15:21:42.717039+0000","last_fullsized":"2024-05-14T15:21:42.717039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.667394+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:21:42.667394+0000","last_peered":"2024-05-14T15:21:42.667394+0000","last_clean":"2024-05-14T15:21:42.667394+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:21:42.667394+0000","last_undegraded":"2024-05-14T15:21:42.667394+0000","last_fullsized":"2024-05-14T15:21:42.667394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.619717+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:21:42.619717+0000","last_peered":"2024-05-14T15:21:42.619717+0000","last_clean":"2024-05-14T15:21:42.619717+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:21:42.619717+0000","last_undegraded":"2024-05-14T15:21:42.619717+0000","last_fullsized":"2024-05-14T15:21:42.619717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664958+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:21:42.664958+0000","last_peered":"2024-05-14T15:21:42.664958+0000","last_clean":"2024-05-14T15:21:42.664958+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:21:42.664958+0000","last_undegraded":"2024-05-14T15:21:42.664958+0000","last_fullsized":"2024-05-14T15:21:42.664958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664109+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:21:42.664109+0000","last_peered":"2024-05-14T15:21:42.664109+0000","last_clean":"2024-05-14T15:21:42.664109+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:21:42.664109+0000","last_undegraded":"2024-05-14T15:21:42.664109+0000","last_fullsized":"2024-05-14T15:21:42.664109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664071+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:21:42.664071+0000","last_peered":"2024-05-14T15:21:42.664071+0000","last_clean":"2024-05-14T15:21:42.664071+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:21:42.664071+0000","last_undegraded":"2024-05-14T15:21:42.664071+0000","last_fullsized":"2024-05-14T15:21:42.664071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.664066+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:21:42.664066+0000","last_peered":"2024-05-14T15:21:42.664066+0000","last_clean":"2024-05-14T15:21:42.664066+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:21:42.664066+0000","last_undegraded":"2024-05-14T15:21:42.664066+0000","last_fullsized":"2024-05-14T15:21:42.664066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.716911+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:21:42.716911+0000","last_peered":"2024-05-14T15:21:42.716911+0000","last_clean":"2024-05-14T15:21:42.716911+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:21:42.716911+0000","last_undegraded":"2024-05-14T15:21:42.716911+0000","last_fullsized":"2024-05-14T15:21:42.716911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-14T15:21:42.614696+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:21:42.614696+0000","last_peered":"2024-05-14T15:21:42.614696+0000","last_clean":"2024-05-14T15:21:42.614696+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:21:42.614696+0000","last_undegraded":"2024-05-14T15:21:42.614696+0000","last_fullsized":"2024-05-14T15:21:42.614696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283950,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62393,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":174,"num_read_kb":156,"num_write":165,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":191444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":499220,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874277,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1143742,"data_compressed":62526,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037803,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":503552,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":499376,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1095285,"data_compressed":62526,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":494632,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":552934,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":496020,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":554675,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":494826,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":499282,"data_compressed":26424,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1090873,"data_compressed":62526,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":59970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:22:27.282 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T15:22:27.282 INFO:tasks.cephadm:Setting up client nodes... 2024-05-14T15:22:27.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-14T15:22:27.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:26 smithi144 ceph-mon[35650]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T15:22:27.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:26 smithi193 ceph-mon[36001]: pgmap v340: 129 pgs: 129 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T15:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:27.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:27 smithi005 ceph-mon[31559]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:22:28.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:27 smithi144 ceph-mon[35650]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:22:28.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:27 smithi193 ceph-mon[36001]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:22:28.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:28 smithi005 ceph-mon[31559]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/3625661261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104190 msgr2=0x7f3ea01045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/3625661261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104190 0x7f3ea01045a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009990 tx=0x7f3e90009ca0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/3625661261 shutdown_connections 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/3625661261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea0105c60 0x7f3ea010c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/3625661261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104190 0x7f3ea01045a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/3625661261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ea01052c0 0x7f3ea01056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.232+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/3625661261 >> 172.21.15.5:0/3625661261 conn(0x7f3ea00ff7d0 msgr2=0x7f3ea0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.233+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/3625661261 shutdown_connections 2024-05-14T15:22:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.233+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/3625661261 wait complete. 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 Processor -- start 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 -- start start 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ea0104190 0x7f3ea019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea01052c0 0x7f3ea019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 0x7f3ea01a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010fc60 con 0x7f3ea0105c60 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010fae0 con 0x7f3ea01052c0 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3ea4b9e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea010fde0 con 0x7f3ea0104190 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3e9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 0x7f3ea01a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3e9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 0x7f3ea01a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38050/0 (socket says 172.21.15.5:38050) 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3e9ffff700 1 -- 172.21.15.5:0/4292214671 learned_addr learned my addr 172.21.15.5:0/4292214671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3e9effd700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea01052c0 0x7f3ea019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.234+0000 7f3e9f7fe700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ea0104190 0x7f3ea019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9ffff700 1 -- 172.21.15.5:0/4292214671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ea0104190 msgr2=0x7f3ea019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9ffff700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ea0104190 0x7f3ea019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9ffff700 1 -- 172.21.15.5:0/4292214671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea01052c0 msgr2=0x7f3ea019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9ffff700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea01052c0 0x7f3ea019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9ffff700 1 -- 172.21.15.5:0/4292214671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01a2300 con 0x7f3ea0105c60 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9ffff700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 0x7f3ea01a1c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800b970 tx=0x7f3e9800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800d860 con 0x7f3ea0105c60 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9800e430 con 0x7f3ea0105c60 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01a25f0 con 0x7f3ea0105c60 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98017660 con 0x7f3ea0105c60 2024-05-14T15:22:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.235+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01a2b60 con 0x7f3ea0105c60 2024-05-14T15:22:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.237+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea00512c0 con 0x7f3ea0105c60 2024-05-14T15:22:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.238+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3e9800da00 con 0x7f3ea0105c60 2024-05-14T15:22:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.239+0000 7f3e9cff9700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e880b2910 0x7f3e880b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.239+0000 7f3e9f7fe700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e880b2910 0x7f3e880b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.239+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3e980e2460 con 0x7f3ea0105c60 2024-05-14T15:22:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.240+0000 7f3e9f7fe700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e880b2910 0x7f3e880b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e900038a0 tx=0x7f3e9001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.243+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e98053df0 con 0x7f3ea0105c60 2024-05-14T15:22:29.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:28 smithi144 ceph-mon[35650]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-14T15:22:29.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:28 smithi193 ceph-mon[36001]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-14T15:22:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.547+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3ea002e5f0 con 0x7f3ea0105c60 2024-05-14T15:22:29.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.550+0000 7f3e9cff9700 1 -- 172.21.15.5:0/4292214671 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3e98013030 con 0x7f3ea0105c60 2024-05-14T15:22:29.551 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-14T15:22:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC1gUNmyrnEIBAALnUmfrjyZS1aJpELnnhcww== 2024-05-14T15:22:29.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e880b2910 msgr2=0x7f3e880b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e880b2910 0x7f3e880b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e900038a0 tx=0x7f3e9001a040 comp rx=0 tx=0).stop 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 msgr2=0x7f3ea01a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 0x7f3ea01a1c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800b970 tx=0x7f3e9800bc80 comp rx=0 tx=0).stop 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 shutdown_connections 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e880b2910 0x7f3e880b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0105c60 0x7f3ea01a1c80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ea0104190 0x7f3ea019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 --2- 172.21.15.5:0/4292214671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea01052c0 0x7f3ea019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 >> 172.21.15.5:0/4292214671 conn(0x7f3ea00ff7d0 msgr2=0x7f3ea01084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:29.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 shutdown_connections 2024-05-14T15:22:29.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:29.552+0000 7f3ea4b9e700 1 -- 172.21.15.5:0/4292214671 wait complete. 2024-05-14T15:22:30.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:29 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4292214671' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T15:22:30.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:29 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4292214671' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T15:22:30.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:29 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4292214671' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T15:22:30.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:29 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4292214671' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T15:22:30.302 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:22:30.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-14T15:22:30.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-14T15:22:30.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:29 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4292214671' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T15:22:30.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:29 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4292214671' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T15:22:30.342 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-14T15:22:30.342 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-14T15:22:30.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-14T15:22:30.634 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:31.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:30 smithi005 ceph-mon[31559]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T15:22:31.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:30 smithi144 ceph-mon[35650]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T15:22:31.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:30 smithi193 ceph-mon[36001]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 -- 172.21.15.5:0/1986865681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76201085c0 msgr2=0x7f76201089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 --2- 172.21.15.5:0/1986865681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76201085c0 0x7f76201089d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7610009990 tx=0x7f7610009ca0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 -- 172.21.15.5:0/1986865681 shutdown_connections 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 --2- 172.21.15.5:0/1986865681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7620102ed0 0x7f762010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 --2- 172.21.15.5:0/1986865681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76201085c0 0x7f76201089d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 --2- 172.21.15.5:0/1986865681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 0x7f7620102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 -- 172.21.15.5:0/1986865681 >> 172.21.15.5:0/1986865681 conn(0x7f76200fe0d0 msgr2=0x7f76201004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 -- 172.21.15.5:0/1986865681 shutdown_connections 2024-05-14T15:22:32.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.471+0000 7f76248a1700 1 -- 172.21.15.5:0/1986865681 wait complete. 2024-05-14T15:22:32.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 Processor -- start 2024-05-14T15:22:32.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 -- start start 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 0x7f762019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 0x7f762019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76201085c0 0x7f762019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7620113430 con 0x7f7620102ed0 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76201132b0 con 0x7f7620102580 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f76248a1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f76201135b0 con 0x7f76201085c0 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f761dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 0x7f762019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f761dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 0x7f762019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38076/0 (socket says 172.21.15.5:38076) 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f761dd9b700 1 -- 172.21.15.5:0/1634281985 learned_addr learned my addr 172.21.15.5:0/1634281985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.473+0000 7f761ed9d700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76201085c0 0x7f762019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761e59c700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 0x7f762019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:32.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761dd9b700 1 -- 172.21.15.5:0/1634281985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76201085c0 msgr2=0x7f762019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761dd9b700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76201085c0 0x7f762019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761dd9b700 1 -- 172.21.15.5:0/1634281985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 msgr2=0x7f762019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761dd9b700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 0x7f762019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761dd9b700 1 -- 172.21.15.5:0/1634281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762019fd60 con 0x7f7620102ed0 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761e59c700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 0x7f762019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f761dd9b700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 0x7f762019b430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f761400c8c0 tx=0x7f761400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614004220 con 0x7f7620102ed0 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76201a0050 con 0x7f7620102ed0 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76140043c0 con 0x7f7620102ed0 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614007710 con 0x7f7620102ed0 2024-05-14T15:22:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.474+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7620104240 con 0x7f7620102ed0 2024-05-14T15:22:32.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.475+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76200512c0 con 0x7f7620102ed0 2024-05-14T15:22:32.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.477+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7614004750 con 0x7f7620102ed0 2024-05-14T15:22:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.478+0000 7f760f7fe700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f76080b2990 0x7f76080b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.478+0000 7f761e59c700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f76080b2990 0x7f76080b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.478+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f76140e2600 con 0x7f7620102ed0 2024-05-14T15:22:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.478+0000 7f761e59c700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f76080b2990 0x7f76080b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7610003800 tx=0x7f761001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:32.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.481+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7614014030 con 0x7f7620102ed0 2024-05-14T15:22:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.764+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7620104500 con 0x7f7620102ed0 2024-05-14T15:22:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.770+0000 7f760f7fe700 1 -- 172.21.15.5:0/1634281985 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288634 (secure 0 0 0) 0x7f76140a9f00 con 0x7f7620102ed0 2024-05-14T15:22:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f76080b2990 msgr2=0x7f76080b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f76080b2990 0x7f76080b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7610003800 tx=0x7f761001a040 comp rx=0 tx=0).stop 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 msgr2=0x7f762019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 0x7f762019b430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f761400c8c0 tx=0x7f761400cbd0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 shutdown_connections 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f76080b2990 0x7f76080b4ea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76201085c0 0x7f762019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7620102580 0x7f762019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 --2- 172.21.15.5:0/1634281985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7620102ed0 0x7f762019b430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 >> 172.21.15.5:0/1634281985 conn(0x7f76200fe0d0 msgr2=0x7f7620106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 shutdown_connections 2024-05-14T15:22:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:32.774+0000 7f76248a1700 1 -- 172.21.15.5:0/1634281985 wait complete. 2024-05-14T15:22:33.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:32 smithi005 ceph-mon[31559]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-14T15:22:33.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:32 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1634281985' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T15:22:33.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:32 smithi144 ceph-mon[35650]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-14T15:22:33.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:32 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1634281985' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T15:22:33.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:32 smithi193 ceph-mon[36001]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-14T15:22:33.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:32 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1634281985' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T15:22:33.506 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":887023568},{"type":"v1","addr":"172.21.15.5:6801","nonce":887023568}]},"active_addr":"172.21.15.5:6801/887023568","active_change":"2024-05-14T15:13:41.367751+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2413445605}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":365942426}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":4280832550}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3326284548}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2605905112}]}]} 2024-05-14T15:22:33.511 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-14T15:22:33.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-14T15:22:33.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:22:33.765 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:35.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:34 smithi005 ceph-mon[31559]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:22:35.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:34 smithi144 ceph-mon[35650]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:22:35.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:34 smithi193 ceph-mon[36001]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:22:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 -- 172.21.15.5:0/3653990687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0281045d0 msgr2=0x7fa0281049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 --2- 172.21.15.5:0/3653990687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0281045d0 0x7fa0281049e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa0180099c0 tx=0x7fa018009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 -- 172.21.15.5:0/3653990687 shutdown_connections 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 --2- 172.21.15.5:0/3653990687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028109850 0x7fa0281111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 --2- 172.21.15.5:0/3653990687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0281045d0 0x7fa0281049e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 --2- 172.21.15.5:0/3653990687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa028108f00 0x7fa028109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 -- 172.21.15.5:0/3653990687 >> 172.21.15.5:0/3653990687 conn(0x7fa0280ffee0 msgr2=0x7fa028102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 -- 172.21.15.5:0/3653990687 shutdown_connections 2024-05-14T15:22:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.534+0000 7fa02c879700 1 -- 172.21.15.5:0/3653990687 wait complete. 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 Processor -- start 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 -- start start 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0281045d0 0x7fa02819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028108f00 0x7fa02819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 0x7fa02819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa028114480 con 0x7fa028109850 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa028114300 con 0x7fa0281045d0 2024-05-14T15:22:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02c879700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa028114600 con 0x7fa028108f00 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa026d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 0x7fa02819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa026d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 0x7fa02819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38082/0 (socket says 172.21.15.5:38082) 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa026d9d700 1 -- 172.21.15.5:0/513149383 learned_addr learned my addr 172.21.15.5:0/513149383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.535+0000 7fa02659c700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0281045d0 0x7fa02819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa025d9b700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028108f00 0x7fa02819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa026d9d700 1 -- 172.21.15.5:0/513149383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028108f00 msgr2=0x7fa02819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa026d9d700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028108f00 0x7fa02819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa026d9d700 1 -- 172.21.15.5:0/513149383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0281045d0 msgr2=0x7fa02819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa026d9d700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0281045d0 0x7fa02819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa026d9d700 1 -- 172.21.15.5:0/513149383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02819fc10 con 0x7fa028109850 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa025d9b700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028108f00 0x7fa02819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa026d9d700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 0x7fa02819f620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa020007d70 tx=0x7fa02000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020016020 con 0x7fa028109850 2024-05-14T15:22:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02819ff00 con 0x7fa028109850 2024-05-14T15:22:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02000ad40 con 0x7fa028109850 2024-05-14T15:22:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200115e0 con 0x7fa028109850 2024-05-14T15:22:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.536+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0281a03f0 con 0x7fa028109850 2024-05-14T15:22:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.537+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0280512c0 con 0x7fa028109850 2024-05-14T15:22:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.539+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa020019020 con 0x7fa028109850 2024-05-14T15:22:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.539+0000 7fa0177fe700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa0100b2910 0x7fa0100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.540+0000 7fa02659c700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa0100b2910 0x7fa0100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.540+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fa0200e2570 con 0x7fa028109850 2024-05-14T15:22:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.540+0000 7fa02659c700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa0100b2910 0x7fa0100b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa018009990 tx=0x7fa018000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:35.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.545+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa020020020 con 0x7fa028109850 2024-05-14T15:22:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.765+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa02802e5f0 con 0x7fa028109850 2024-05-14T15:22:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.767+0000 7fa0177fe700 1 -- 172.21.15.5:0/513149383 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24528 (secure 0 0 0) 0x7fa0200a9ea0 con 0x7fa028109850 2024-05-14T15:22:35.769 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:35.769 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:42.610573+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa0100b2910 msgr2=0x7fa0100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa0100b2910 0x7fa0100b4e20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa018009990 tx=0x7fa018000bc0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 msgr2=0x7fa02819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 0x7fa02819f620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa020007d70 tx=0x7fa02000a340 comp rx=0 tx=0).stop 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 shutdown_connections 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa0100b2910 0x7fa0100b4e20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa028109850 0x7fa02819f620 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0281045d0 0x7fa02819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 --2- 172.21.15.5:0/513149383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa028108f00 0x7fa02819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 >> 172.21.15.5:0/513149383 conn(0x7fa0280ffee0 msgr2=0x7fa02810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 shutdown_connections 2024-05-14T15:22:35.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:35.771+0000 7fa02c879700 1 -- 172.21.15.5:0/513149383 wait complete. 2024-05-14T15:22:36.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/513149383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:36.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:35 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/513149383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:36.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:35 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/513149383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:36.491 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-14T15:22:36.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:22:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:37.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:36 smithi005 ceph-mon[31559]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:22:37.296 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:36 smithi144 ceph-mon[35650]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:22:37.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:36 smithi193 ceph-mon[36001]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 -- 172.21.15.5:0/2884017422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a8109300 msgr2=0x7fb9a8109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/2884017422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a8109300 0x7fb9a8109710 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009990 tx=0x7fb99c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 -- 172.21.15.5:0/2884017422 shutdown_connections 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/2884017422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a81030e0 0x7fb9a8111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/2884017422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a8109300 0x7fb9a8109710 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/2884017422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8102790 0x7fb9a8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 -- 172.21.15.5:0/2884017422 >> 172.21.15.5:0/2884017422 conn(0x7fb9a80fe520 msgr2=0x7fb9a8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 -- 172.21.15.5:0/2884017422 shutdown_connections 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.423+0000 7fb9b07ee700 1 -- 172.21.15.5:0/2884017422 wait complete. 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 Processor -- start 2024-05-14T15:22:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 -- start start 2024-05-14T15:22:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a8102790 0x7fb9a8103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 0x7fb9a8103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8109300 0x7fb9a81043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8114590 con 0x7fb9a81030e0 2024-05-14T15:22:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8114410 con 0x7fb9a8102790 2024-05-14T15:22:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9b07ee700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a8114710 con 0x7fb9a8109300 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 0x7fb9a8103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 0x7fb9a8103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54528/0 (socket says 172.21.15.5:54528) 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 -- 172.21.15.5:0/3887688194 learned_addr learned my addr 172.21.15.5:0/3887688194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9aed8b700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8109300 0x7fb9a81043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9ae58a700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a8102790 0x7fb9a8103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 -- 172.21.15.5:0/3887688194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8109300 msgr2=0x7fb9a81043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8109300 0x7fb9a81043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 -- 172.21.15.5:0/3887688194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a8102790 msgr2=0x7fb9a8103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a8102790 0x7fb9a8103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9add89700 1 -- 172.21.15.5:0/3887688194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a81a9240 con 0x7fb9a81030e0 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9aed8b700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8109300 0x7fb9a81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.424+0000 7fb9ae58a700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a8102790 0x7fb9a8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.425+0000 7fb9add89700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 0x7fb9a8103eb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400c8f0 tx=0x7fb9a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.425+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4007b00 con 0x7fb9a81030e0 2024-05-14T15:22:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.425+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a4007ca0 con 0x7fb9a81030e0 2024-05-14T15:22:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.425+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4018660 con 0x7fb9a81030e0 2024-05-14T15:22:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.425+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a81a9530 con 0x7fb9a81030e0 2024-05-14T15:22:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.425+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a81a9a70 con 0x7fb9a81030e0 2024-05-14T15:22:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.426+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80512c0 con 0x7fb9a81030e0 2024-05-14T15:22:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.428+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb9a400f450 con 0x7fb9a81030e0 2024-05-14T15:22:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.428+0000 7fb99b7fe700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb9940b2910 0x7fb9940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.428+0000 7fb9ae58a700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb9940b2910 0x7fb9940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.428+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fb9a40e27c0 con 0x7fb9a81030e0 2024-05-14T15:22:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.429+0000 7fb9ae58a700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb9940b2910 0x7fb9940b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb99c006010 tx=0x7fb99c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:38.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.432+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb9a4014030 con 0x7fb9a81030e0 2024-05-14T15:22:38.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.654+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9a810c8a0 con 0x7fb9a81030e0 2024-05-14T15:22:38.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.656+0000 7fb99b7fe700 1 -- 172.21.15.5:0/3887688194 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24528 (secure 0 0 0) 0x7fb9a40aa0d0 con 0x7fb9a81030e0 2024-05-14T15:22:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:22:38.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:21:42.610573+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb9940b2910 msgr2=0x7fb9940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb9940b2910 0x7fb9940b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb99c006010 tx=0x7fb99c01a040 comp rx=0 tx=0).stop 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 msgr2=0x7fb9a8103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 0x7fb9a8103eb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400c8f0 tx=0x7fb9a400ccb0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 shutdown_connections 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb9940b2910 0x7fb9940b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a8109300 0x7fb9a81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9a8102790 0x7fb9a8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 --2- 172.21.15.5:0/3887688194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9a81030e0 0x7fb9a8103eb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 >> 172.21.15.5:0/3887688194 conn(0x7fb9a80fe520 msgr2=0x7fb9a81070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.658+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 shutdown_connections 2024-05-14T15:22:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:38.659+0000 7fb9b07ee700 1 -- 172.21.15.5:0/3887688194 wait complete. 2024-05-14T15:22:38.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:38 smithi005 ceph-mon[31559]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:22:38.980 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:38 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3887688194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:39.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:38 smithi144 ceph-mon[35650]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:22:39.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:38 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3887688194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:39.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:38 smithi193 ceph-mon[36001]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:22:39.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:38 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3887688194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:22:39.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-14T15:22:39.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-14T15:22:39.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-14T15:22:39.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-14T15:22:39.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-14T15:22:39.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-14T15:22:39.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-14T15:22:39.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-14T15:22:39.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-14T15:22:39.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-14T15:22:39.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-14T15:22:39.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-14T15:22:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.429 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.430 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.431 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.860 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.861 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.861 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.862 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.865 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:40.866 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:22:41.295 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:40 smithi005 ceph-mon[31559]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:22:41.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:40 smithi144 ceph-mon[35650]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:22:41.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:40 smithi193 ceph-mon[36001]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:22:42.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-05-14T15:22:42.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]: dispatch 2024-05-14T15:22:42.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-14T15:22:42.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2024-05-14T15:22:42.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:41 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-05-14T15:22:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-05-14T15:22:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]: dispatch 2024-05-14T15:22:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-14T15:22:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2024-05-14T15:22:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:41 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-05-14T15:22:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-05-14T15:22:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]: dispatch 2024-05-14T15:22:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-14T15:22:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]: dispatch 2024-05-14T15:22:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:41 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]': finished 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-05-14T15:22:43.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:42 smithi144 ceph-mon[35650]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]': finished 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-05-14T15:22:43.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:42 smithi193 ceph-mon[36001]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]': finished 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-05-14T15:22:43.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:42 smithi005 ceph-mon[31559]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T15:22:43.983 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:43 smithi005 ceph-mon[31559]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T15:22:44.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:43 smithi144 ceph-mon[35650]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T15:22:44.302 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:43 smithi193 ceph-mon[36001]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T15:22:45.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:44 smithi005 ceph-mon[31559]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-14T15:22:45.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:44 smithi005 ceph-mon[31559]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-14T15:22:45.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:44 smithi144 ceph-mon[35650]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-14T15:22:45.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:44 smithi144 ceph-mon[35650]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-14T15:22:45.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:44 smithi193 ceph-mon[36001]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-14T15:22:45.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:44 smithi193 ceph-mon[36001]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-14T15:22:47.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:46 smithi005 ceph-mon[31559]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:47.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:46 smithi144 ceph-mon[35650]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:46 smithi193 ceph-mon[36001]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 -- 172.21.15.5:0/1262259170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04109850 msgr2=0x7fce041111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 --2- 172.21.15.5:0/1262259170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04109850 0x7fce041111b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b0d0 tx=0x7fcdfc00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 -- 172.21.15.5:0/1262259170 shutdown_connections 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 --2- 172.21.15.5:0/1262259170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04109850 0x7fce041111b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 --2- 172.21.15.5:0/1262259170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce041045d0 0x7fce041049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 --2- 172.21.15.5:0/1262259170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce04108f00 0x7fce04109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 -- 172.21.15.5:0/1262259170 >> 172.21.15.5:0/1262259170 conn(0x7fce040ffea0 msgr2=0x7fce041022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 -- 172.21.15.5:0/1262259170 shutdown_connections 2024-05-14T15:22:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.518+0000 7fce09ea2700 1 -- 172.21.15.5:0/1262259170 wait complete. 2024-05-14T15:22:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 Processor -- start 2024-05-14T15:22:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 -- start start 2024-05-14T15:22:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce041045d0 0x7fce0419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04108f00 0x7fce0419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce04109850 0x7fce0419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce041144d0 con 0x7fce04108f00 2024-05-14T15:22:47.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114350 con 0x7fce04109850 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce09ea2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114650 con 0x7fce041045d0 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04108f00 0x7fce0419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04108f00 0x7fce0419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57978/0 (socket says 172.21.15.5:57978) 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce02ffd700 1 -- 172.21.15.5:0/709259263 learned_addr learned my addr 172.21.15.5:0/709259263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce037fe700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce041045d0 0x7fce0419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.519+0000 7fce03fff700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce04109850 0x7fce0419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce02ffd700 1 -- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce041045d0 msgr2=0x7fce0419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce02ffd700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce041045d0 0x7fce0419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce02ffd700 1 -- 172.21.15.5:0/709259263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce04109850 msgr2=0x7fce0419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce02ffd700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce04109850 0x7fce0419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce02ffd700 1 -- 172.21.15.5:0/709259263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0419fc70 con 0x7fce04108f00 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce02ffd700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04108f00 0x7fce0419b340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800d950 tx=0x7fcdf800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8016070 con 0x7fce04108f00 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce09ea2700 1 -- 172.21.15.5:0/709259263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0419ff60 con 0x7fce04108f00 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce09ea2700 1 -- 172.21.15.5:0/709259263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce04076b00 con 0x7fce04108f00 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.520+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf8004830 con 0x7fce04108f00 2024-05-14T15:22:47.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.521+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8005520 con 0x7fce04108f00 2024-05-14T15:22:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.523+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcde4001030 con 0x7fce04108f00 2024-05-14T15:22:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.525+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcdf8004220 con 0x7fce04108f00 2024-05-14T15:22:47.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.525+0000 7fce00ff9700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcdec0b2990 0x7fcdec0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:47.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.525+0000 7fce037fe700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcdec0b2990 0x7fcdec0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:47.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.525+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fcdf80e23c0 con 0x7fce04108f00 2024-05-14T15:22:47.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.526+0000 7fce00ff9700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] conn(0x7fcdec0bdd40 0x7fcdec0c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.526+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 --> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcdec0c0900 con 0x7fcdec0bdd40 2024-05-14T15:22:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.526+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcdf80e9050 con 0x7fce04108f00 2024-05-14T15:22:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.526+0000 7fce03fff700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] conn(0x7fcdec0bdd40 0x7fcdec0c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.526+0000 7fce037fe700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcdec0b2990 0x7fcdec0b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005950 tx=0x7fcdf400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:47.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.527+0000 7fce03fff700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] conn(0x7fcdec0bdd40 0x7fcdec0c0230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.528+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== osd.11 v2:172.21.15.193:6824/1840598125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcdec0c0900 con 0x7fcdec0bdd40 2024-05-14T15:22:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.547+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 --> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcde4002e60 con 0x7fcdec0bdd40 2024-05-14T15:22:47.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fce00ff9700 1 -- 172.21.15.5:0/709259263 <== osd.11 v2:172.21.15.193:6824/1840598125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcde4002e60 con 0x7fcdec0bdd40 2024-05-14T15:22:47.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] conn(0x7fcdec0bdd40 msgr2=0x7fcdec0c0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:47.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] conn(0x7fcdec0bdd40 0x7fcdec0c0230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:47.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcdec0b2990 msgr2=0x7fcdec0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcdec0b2990 0x7fcdec0b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005950 tx=0x7fcdf400b410 comp rx=0 tx=0).stop 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04108f00 msgr2=0x7fce0419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 --2- 172.21.15.5:0/709259263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce04108f00 0x7fce0419b340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800d950 tx=0x7fcdf800dc60 comp rx=0 tx=0).stop 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fce03fff700 1 -- 172.21.15.5:0/709259263 reap_dead start 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 shutdown_connections 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.548+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 >> 172.21.15.5:0/709259263 conn(0x7fce040ffea0 msgr2=0x7fce04108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.549+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 shutdown_connections 2024-05-14T15:22:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:47.549+0000 7fcdea7fc700 1 -- 172.21.15.5:0/709259263 wait complete. 2024-05-14T15:22:49.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:48 smithi005 ceph-mon[31559]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:49.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:48 smithi144 ceph-mon[35650]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:49.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:48 smithi193 ceph-mon[36001]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:49.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 -- 172.21.15.5:0/1610498455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 msgr2=0x7f3ef0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/1610498455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef0109350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40099c0 tx=0x7f3ee4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 -- 172.21.15.5:0/1610498455 shutdown_connections 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/1610498455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef0109890 0x7f3ef01111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/1610498455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef0102b20 0x7f3ef0102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/1610498455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef0109350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 -- 172.21.15.5:0/1610498455 >> 172.21.15.5:0/1610498455 conn(0x7f3ef00fe520 msgr2=0x7f3ef0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 -- 172.21.15.5:0/1610498455 shutdown_connections 2024-05-14T15:22:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.806+0000 7f3ef50d1700 1 -- 172.21.15.5:0/1610498455 wait complete. 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 Processor -- start 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 -- start start 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef0102b20 0x7f3ef019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef0109890 0x7f3ef019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0114490 con 0x7f3ef0108f40 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0114310 con 0x7f3ef0102b20 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.807+0000 7f3ef50d1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0114610 con 0x7f3ef0109890 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57988/0 (socket says 172.21.15.5:57988) 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 -- 172.21.15.5:0/3512013276 learned_addr learned my addr 172.21.15.5:0/3512013276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eef59e700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef0109890 0x7f3ef019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eeed9d700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef0102b20 0x7f3ef019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 -- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef0109890 msgr2=0x7f3ef019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ef0109890 0x7f3ef019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 -- 172.21.15.5:0/3512013276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef0102b20 msgr2=0x7f3ef019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef0102b20 0x7f3ef019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 -- 172.21.15.5:0/3512013276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef019fcd0 con 0x7f3ef0108f40 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3eee59c700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef019b3a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006010 tx=0x7f3ee40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4026070 con 0x7f3ef0108f40 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.808+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef019ff60 con 0x7f3ef0108f40 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.809+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee400f460 con 0x7f3ef0108f40 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.809+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4009e60 con 0x7f3ef0108f40 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.809+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a78e0 con 0x7f3ef0108f40 2024-05-14T15:22:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.810+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ef01a0490 con 0x7f3ef0108f40 2024-05-14T15:22:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.811+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3ee4029070 con 0x7f3ef0108f40 2024-05-14T15:22:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.812+0000 7f3ed7fff700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ed80b2990 0x7f3ed80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.812+0000 7f3eeed9d700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ed80b2990 0x7f3ed80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.812+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f3ee40e3210 con 0x7f3ef0108f40 2024-05-14T15:22:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.812+0000 7f3ed7fff700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] conn(0x7f3ed80bdc10 0x7f3ed80c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.812+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 --> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ee4012ec0 con 0x7f3ed80bdc10 2024-05-14T15:22:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.812+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3ee40e3640 con 0x7f3ef0108f40 2024-05-14T15:22:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.813+0000 7f3eeed9d700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ed80b2990 0x7f3ed80b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006bbf0 tx=0x7f3ee0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.813+0000 7f3eef59e700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] conn(0x7f3ed80bdc10 0x7f3ed80c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.813+0000 7f3eef59e700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] conn(0x7f3ed80bdc10 0x7f3ed80c0100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.814+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== osd.2 v2:172.21.15.193:6800/3480718798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ee4012ec0 con 0x7f3ed80bdc10 2024-05-14T15:22:49.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.831+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 --> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ef0064160 con 0x7f3ed80bdc10 2024-05-14T15:22:49.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ed7fff700 1 -- 172.21.15.5:0/3512013276 <== osd.2 v2:172.21.15.193:6800/3480718798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ef0064160 con 0x7f3ed80bdc10 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] conn(0x7f3ed80bdc10 msgr2=0x7f3ed80c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] conn(0x7f3ed80bdc10 0x7f3ed80c0100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ed80b2990 msgr2=0x7f3ed80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ed80b2990 0x7f3ed80b4ea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ef006bbf0 tx=0x7f3ee0009260 comp rx=0 tx=0).stop 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 msgr2=0x7f3ef019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 --2- 172.21.15.5:0/3512013276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef0108f40 0x7f3ef019b3a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006010 tx=0x7f3ee40055f0 comp rx=0 tx=0).stop 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3eef59e700 1 -- 172.21.15.5:0/3512013276 reap_dead start 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 shutdown_connections 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 >> 172.21.15.5:0/3512013276 conn(0x7f3ef00fe520 msgr2=0x7f3ef0108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 shutdown_connections 2024-05-14T15:22:49.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:49.832+0000 7f3ef50d1700 1 -- 172.21.15.5:0/3512013276 wait complete. 2024-05-14T15:22:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:49 smithi144 ceph-mon[35650]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:49 smithi144 ceph-mon[35650]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-14T15:22:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:49 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:22:50.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:49 smithi193 ceph-mon[36001]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:50.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:49 smithi193 ceph-mon[36001]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-14T15:22:50.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:49 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:22:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:49 smithi005 ceph-mon[31559]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:49 smithi005 ceph-mon[31559]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-14T15:22:50.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:49 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:22:52.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:52 smithi193 ceph-mon[36001]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:52.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:52 smithi005 ceph-mon[31559]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:53.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:52 smithi144 ceph-mon[35650]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:54.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:54 smithi005 ceph-mon[31559]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:54.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:54 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 -- 172.21.15.5:0/3708039927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d901034e0 msgr2=0x7f3d901038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/3708039927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d901034e0 0x7f3d901038f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d800099c0 tx=0x7f3d80009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 -- 172.21.15.5:0/3708039927 shutdown_connections 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/3708039927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d90108760 0x7f3d901100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/3708039927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d901034e0 0x7f3d901038f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/3708039927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d90107e10 0x7f3d90108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 -- 172.21.15.5:0/3708039927 >> 172.21.15.5:0/3708039927 conn(0x7f3d900fef30 msgr2=0x7f3d90101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 -- 172.21.15.5:0/3708039927 shutdown_connections 2024-05-14T15:22:54.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 -- 172.21.15.5:0/3708039927 wait complete. 2024-05-14T15:22:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 Processor -- start 2024-05-14T15:22:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.628+0000 7f3d96f0c700 1 -- start start 2024-05-14T15:22:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d96f0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901034e0 0x7f3d900ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d96f0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d90107e10 0x7f3d900ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d96f0c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d90108760 0x7f3d900ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d96f0c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90113390 con 0x7f3d90108760 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d96f0c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90113210 con 0x7f3d90107e10 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d96f0c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90113510 con 0x7f3d901034e0 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d954a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d90108760 0x7f3d900ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d954a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d90108760 0x7f3d900ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58004/0 (socket says 172.21.15.5:58004) 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d954a9700 1 -- 172.21.15.5:0/1568653493 learned_addr learned my addr 172.21.15.5:0/1568653493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901034e0 0x7f3d900ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d8ffff700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d90107e10 0x7f3d900ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 -- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d90107e10 msgr2=0x7f3d900ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d90107e10 0x7f3d900ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 -- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d90108760 msgr2=0x7f3d900ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d90108760 0x7f3d900ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 -- 172.21.15.5:0/1568653493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901a9170 con 0x7f3d901034e0 2024-05-14T15:22:54.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d954a9700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d90108760 0x7f3d900ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.629+0000 7f3d94ca8700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901034e0 0x7f3d900ff410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d800098e0 tx=0x7f3d80004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.630+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80026070 con 0x7f3d901034e0 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.630+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901a9400 con 0x7f3d901034e0 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.630+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d80003c70 con 0x7f3d901034e0 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.630+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000f9c0 con 0x7f3d901034e0 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.630+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901a98b0 con 0x7f3d901034e0 2024-05-14T15:22:54.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.631+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d90063ed0 con 0x7f3d901034e0 2024-05-14T15:22:54.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.633+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3d80029070 con 0x7f3d901034e0 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.633+0000 7f3d8dffb700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3d780b2910 0x7f3d780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.633+0000 7f3d8ffff700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3d780b2910 0x7f3d780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.633+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f3d800e3880 con 0x7f3d901034e0 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.634+0000 7f3d8dffb700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] conn(0x7f3d780bdcc0 0x7f3d780c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.634+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 --> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d80001d30 con 0x7f3d780bdcc0 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.634+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3d800e3c30 con 0x7f3d901034e0 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.634+0000 7f3d8ffff700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3d780b2910 0x7f3d780b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d90109b10 tx=0x7f3d84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:54.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.634+0000 7f3d954a9700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] conn(0x7f3d780bdcc0 0x7f3d780c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:54.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.634+0000 7f3d954a9700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] conn(0x7f3d780bdcc0 0x7f3d780c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:54.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.635+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== osd.10 v2:172.21.15.144:6824/366654361 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d80001d30 con 0x7f3d780bdcc0 2024-05-14T15:22:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.655+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 --> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d900642d0 con 0x7f3d780bdcc0 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d8dffb700 1 -- 172.21.15.5:0/1568653493 <== osd.10 v2:172.21.15.144:6824/366654361 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d900642d0 con 0x7f3d780bdcc0 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] conn(0x7f3d780bdcc0 msgr2=0x7f3d780c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] conn(0x7f3d780bdcc0 0x7f3d780c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3d780b2910 msgr2=0x7f3d780b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3d780b2910 0x7f3d780b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3d90109b10 tx=0x7f3d84009260 comp rx=0 tx=0).stop 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901034e0 msgr2=0x7f3d900ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 --2- 172.21.15.5:0/1568653493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901034e0 0x7f3d900ff410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d800098e0 tx=0x7f3d80004270 comp rx=0 tx=0).stop 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d954a9700 1 -- 172.21.15.5:0/1568653493 reap_dead start 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 shutdown_connections 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 >> 172.21.15.5:0/1568653493 conn(0x7f3d900fef30 msgr2=0x7f3d9006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:54.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 shutdown_connections 2024-05-14T15:22:54.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:54.656+0000 7f3d96f0c700 1 -- 172.21.15.5:0/1568653493 wait complete. 2024-05-14T15:22:54.796 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:54 smithi144 ceph-mon[35650]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:54.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:54 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:22:54.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:54 smithi193 ceph-mon[36001]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:54.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:54 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:22:55.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 -- 172.21.15.5:0/217021424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 msgr2=0x7fcb841100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:55.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/217021424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb841100c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00b0d0 tx=0x7fcb7c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 -- 172.21.15.5:0/217021424 shutdown_connections 2024-05-14T15:22:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/217021424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb841100c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/217021424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 0x7fcb841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/217021424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb84107e10 0x7fcb84108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 -- 172.21.15.5:0/217021424 >> 172.21.15.5:0/217021424 conn(0x7fcb840fef30 msgr2=0x7fcb84101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 -- 172.21.15.5:0/217021424 shutdown_connections 2024-05-14T15:22:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.182+0000 7fcb8bec1700 1 -- 172.21.15.5:0/217021424 wait complete. 2024-05-14T15:22:55.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 Processor -- start 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 -- start start 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 0x7fcb840ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb84107e10 0x7fcb840ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb840ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113390 con 0x7fcb84108760 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113210 con 0x7fcb84107e10 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8bec1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113510 con 0x7fcb841034e0 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8a45e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb840ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb89c5d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 0x7fcb840ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8a45e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb840ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58020/0 (socket says 172.21.15.5:58020) 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8a45e700 1 -- 172.21.15.5:0/1011707438 learned_addr learned my addr 172.21.15.5:0/1011707438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb89c5d700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 0x7fcb840ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:43082/0 (socket says 172.21.15.5:43082) 2024-05-14T15:22:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.183+0000 7fcb8945c700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb84107e10 0x7fcb840ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb89c5d700 1 -- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb84107e10 msgr2=0x7fcb840ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb89c5d700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb84107e10 0x7fcb840ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb89c5d700 1 -- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 msgr2=0x7fcb840ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb89c5d700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb840ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb89c5d700 1 -- 172.21.15.5:0/1011707438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb841a9170 con 0x7fcb841034e0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb8a45e700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb84108760 0x7fcb840ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb89c5d700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 0x7fcb840ff410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e9f0 tx=0x7fcb8000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000c9c0 con 0x7fcb841034e0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb841a9460 con 0x7fcb841034e0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8000cb60 con 0x7fcb841034e0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80018680 con 0x7fcb841034e0 2024-05-14T15:22:55.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.184+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a99f0 con 0x7fcb841034e0 2024-05-14T15:22:55.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.185+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb84063ed0 con 0x7fcb841034e0 2024-05-14T15:22:55.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.187+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcb80010430 con 0x7fcb841034e0 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.187+0000 7fcb7affd700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb700b2910 0x7fcb700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.187+0000 7fcb8945c700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb700b2910 0x7fcb700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.188+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fcb800e2ea0 con 0x7fcb841034e0 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.188+0000 7fcb7affd700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] conn(0x7fcb700bdcc0 0x7fcb700c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.188+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 --> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb700c0880 con 0x7fcb700bdcc0 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.188+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcb800e3290 con 0x7fcb841034e0 2024-05-14T15:22:55.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.188+0000 7fcb8945c700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb700b2910 0x7fcb700b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb841098c0 tx=0x7fcb74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:55.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.188+0000 7fcb8a45e700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] conn(0x7fcb700bdcc0 0x7fcb700c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:55.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.189+0000 7fcb8a45e700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] conn(0x7fcb700bdcc0 0x7fcb700c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:55.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.189+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== osd.1 v2:172.21.15.144:6800/3346749766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb700c0880 con 0x7fcb700bdcc0 2024-05-14T15:22:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.206+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 --> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb840642d0 con 0x7fcb700bdcc0 2024-05-14T15:22:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb7affd700 1 -- 172.21.15.5:0/1011707438 <== osd.1 v2:172.21.15.144:6800/3346749766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcb840642d0 con 0x7fcb700bdcc0 2024-05-14T15:22:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] conn(0x7fcb700bdcc0 msgr2=0x7fcb700c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] conn(0x7fcb700bdcc0 0x7fcb700c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb700b2910 msgr2=0x7fcb700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb700b2910 0x7fcb700b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb841098c0 tx=0x7fcb74009260 comp rx=0 tx=0).stop 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 msgr2=0x7fcb840ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 --2- 172.21.15.5:0/1011707438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb841034e0 0x7fcb840ff410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e9f0 tx=0x7fcb8000ed00 comp rx=0 tx=0).stop 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8a45e700 1 -- 172.21.15.5:0/1011707438 reap_dead start 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 shutdown_connections 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 >> 172.21.15.5:0/1011707438 conn(0x7fcb840fef30 msgr2=0x7fcb8406c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 shutdown_connections 2024-05-14T15:22:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:55.207+0000 7fcb8bec1700 1 -- 172.21.15.5:0/1011707438 wait complete. 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 -- 172.21.15.5:0/1238411983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff694105e50 msgr2=0x7ff69410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 --2- 172.21.15.5:0/1238411983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff694105e50 0x7ff69410e3a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688009ca0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 -- 172.21.15.5:0/1238411983 shutdown_connections 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 --2- 172.21.15.5:0/1238411983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff69410e930 0x7ff694110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 --2- 172.21.15.5:0/1238411983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105500 0x7ff694105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 --2- 172.21.15.5:0/1238411983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff694105e50 0x7ff69410e3a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.435+0000 7ff698f24700 1 -- 172.21.15.5:0/1238411983 >> 172.21.15.5:0/1238411983 conn(0x7ff6940fedf0 msgr2=0x7ff694101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:56.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.436+0000 7ff698f24700 1 -- 172.21.15.5:0/1238411983 shutdown_connections 2024-05-14T15:22:56.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.436+0000 7ff698f24700 1 -- 172.21.15.5:0/1238411983 wait complete. 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.436+0000 7ff698f24700 1 Processor -- start 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.436+0000 7ff698f24700 1 -- start start 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff698f24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105500 0x7ff6941031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff698f24700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff694105e50 0x7ff694103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff698f24700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff69410e930 0x7ff694103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff698f24700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff694114450 con 0x7ff69410e930 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff698f24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6941142d0 con 0x7ff694105500 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff698f24700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6941145d0 con 0x7ff694105e50 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff69410e930 0x7ff694103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff691d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff694105e50 0x7ff694103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff69410e930 0x7ff694103c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37782/0 (socket says 172.21.15.5:37782) 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 -- 172.21.15.5:0/2342529240 learned_addr learned my addr 172.21.15.5:0/2342529240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff69259c700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105500 0x7ff6941031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 -- 172.21.15.5:0/2342529240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff694105e50 msgr2=0x7ff694103730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff694105e50 0x7ff694103730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 -- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105500 msgr2=0x7ff6941031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105500 0x7ff6941031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff692d9d700 1 -- 172.21.15.5:0/2342529240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6941a91a0 con 0x7ff69410e930 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff691d9b700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff694105e50 0x7ff694103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.437+0000 7ff69259c700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff694105500 0x7ff6941031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.438+0000 7ff692d9d700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff69410e930 0x7ff694103c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00d980 tx=0x7ff68c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.438+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c017070 con 0x7ff69410e930 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.438+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff68c005020 con 0x7ff69410e930 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.438+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6941a9490 con 0x7ff69410e930 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.438+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0099b0 con 0x7ff69410e930 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.438+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6941a9950 con 0x7ff69410e930 2024-05-14T15:22:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.439+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff694063ed0 con 0x7ff69410e930 2024-05-14T15:22:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.441+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff68c009b50 con 0x7ff69410e930 2024-05-14T15:22:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.441+0000 7ff6837fe700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff67c0b2910 0x7ff67c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.441+0000 7ff69259c700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff67c0b2910 0x7ff67c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.442+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7ff68c0e2c10 con 0x7ff69410e930 2024-05-14T15:22:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.442+0000 7ff6837fe700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] conn(0x7ff67c0bdcc0 0x7ff67c0c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.442+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 --> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff67c0c0880 con 0x7ff67c0bdcc0 2024-05-14T15:22:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.442+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff68c0e3040 con 0x7ff69410e930 2024-05-14T15:22:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.442+0000 7ff69259c700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff67c0b2910 0x7ff67c0b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff69406d150 tx=0x7ff684009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.442+0000 7ff691d9b700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] conn(0x7ff67c0bdcc0 0x7ff67c0c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.443+0000 7ff691d9b700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] conn(0x7ff67c0bdcc0 0x7ff67c0c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.443+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== osd.4 v2:172.21.15.144:6808/3402609207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff67c0c0880 con 0x7ff67c0bdcc0 2024-05-14T15:22:56.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.460+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 --> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff6940642d0 con 0x7ff67c0bdcc0 2024-05-14T15:22:56.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff6837fe700 1 -- 172.21.15.5:0/2342529240 <== osd.4 v2:172.21.15.144:6808/3402609207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff6940642d0 con 0x7ff67c0bdcc0 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] conn(0x7ff67c0bdcc0 msgr2=0x7ff67c0c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] conn(0x7ff67c0bdcc0 0x7ff67c0c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff67c0b2910 msgr2=0x7ff67c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff67c0b2910 0x7ff67c0b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff69406d150 tx=0x7ff684009040 comp rx=0 tx=0).stop 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff69410e930 msgr2=0x7ff694103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 --2- 172.21.15.5:0/2342529240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff69410e930 0x7ff694103c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00d980 tx=0x7ff68c00dc90 comp rx=0 tx=0).stop 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff692d9d700 1 -- 172.21.15.5:0/2342529240 reap_dead start 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 shutdown_connections 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 >> 172.21.15.5:0/2342529240 conn(0x7ff6940fedf0 msgr2=0x7ff6940ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:56.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 shutdown_connections 2024-05-14T15:22:56.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:56.461+0000 7ff698f24700 1 -- 172.21.15.5:0/2342529240 wait complete. 2024-05-14T15:22:56.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:56 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:56 smithi005 ceph-mon[31559]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:56.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:56 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:56 smithi144 ceph-mon[35650]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:56.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:56 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:22:56.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:56 smithi193 ceph-mon[36001]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 -- 172.21.15.5:0/423980826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 msgr2=0x7fd16c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 --2- 172.21.15.5:0/423980826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c108200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1600099c0 tx=0x7fd160009cd0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 -- 172.21.15.5:0/423980826 shutdown_connections 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 --2- 172.21.15.5:0/423980826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd16c108740 0x7fd16c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 --2- 172.21.15.5:0/423980826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd16c1034c0 0x7fd16c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 --2- 172.21.15.5:0/423980826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c108200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 -- 172.21.15.5:0/423980826 >> 172.21.15.5:0/423980826 conn(0x7fd16c0fedc0 msgr2=0x7fd16c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 -- 172.21.15.5:0/423980826 shutdown_connections 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 -- 172.21.15.5:0/423980826 wait complete. 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.074+0000 7fd171e99700 1 Processor -- start 2024-05-14T15:22:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 -- start start 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd16c1034c0 0x7fd16c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd16c108740 0x7fd16c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c113350 con 0x7fd16c107df0 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1131d0 con 0x7fd16c1034c0 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd171e99700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c1134d0 con 0x7fd16c108740 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd16affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd16affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c0ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37796/0 (socket says 172.21.15.5:37796) 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd16affd700 1 -- 172.21.15.5:0/264217113 learned_addr learned my addr 172.21.15.5:0/264217113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd16b7fe700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd16c1034c0 0x7fd16c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.075+0000 7fd16bfff700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd16c108740 0x7fd16c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16affd700 1 -- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd16c108740 msgr2=0x7fd16c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16affd700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd16c108740 0x7fd16c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16affd700 1 -- 172.21.15.5:0/264217113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd16c1034c0 msgr2=0x7fd16c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16affd700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd16c1034c0 0x7fd16c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16affd700 1 -- 172.21.15.5:0/264217113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c1a9150 con 0x7fd16c107df0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16b7fe700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd16c1034c0 0x7fd16c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd16affd700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c0ff7e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd160005f50 tx=0x7fd160004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160026070 con 0x7fd16c107df0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd160003c70 con 0x7fd16c107df0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c1a93e0 con 0x7fd16c107df0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16000f9c0 con 0x7fd16c107df0 2024-05-14T15:22:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.076+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c1a9830 con 0x7fd16c107df0 2024-05-14T15:22:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.077+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd16c063e90 con 0x7fd16c107df0 2024-05-14T15:22:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.079+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd160029070 con 0x7fd16c107df0 2024-05-14T15:22:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.080+0000 7fd168ff9700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1540b2910 0x7fd1540b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.080+0000 7fd16b7fe700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1540b2910 0x7fd1540b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.080+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fd1600e3930 con 0x7fd16c107df0 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.080+0000 7fd168ff9700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] conn(0x7fd1540bdcc0 0x7fd1540c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.080+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 --> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd160012ec0 con 0x7fd1540bdcc0 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.080+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd1600e3d60 con 0x7fd16c107df0 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.081+0000 7fd16b7fe700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1540b2910 0x7fd1540b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c011490 tx=0x7fd15c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.081+0000 7fd16bfff700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] conn(0x7fd1540bdcc0 0x7fd1540c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.081+0000 7fd16bfff700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] conn(0x7fd1540bdcc0 0x7fd1540c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.082+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== osd.5 v2:172.21.15.193:6808/2216132149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd160012ec0 con 0x7fd1540bdcc0 2024-05-14T15:22:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.099+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 --> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd16c064290 con 0x7fd1540bdcc0 2024-05-14T15:22:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.099+0000 7fd168ff9700 1 -- 172.21.15.5:0/264217113 <== osd.5 v2:172.21.15.193:6808/2216132149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd16c064290 con 0x7fd1540bdcc0 2024-05-14T15:22:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] conn(0x7fd1540bdcc0 msgr2=0x7fd1540c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] conn(0x7fd1540bdcc0 0x7fd1540c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1540b2910 msgr2=0x7fd1540b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd1540b2910 0x7fd1540b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c011490 tx=0x7fd15c009260 comp rx=0 tx=0).stop 2024-05-14T15:22:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 msgr2=0x7fd16c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 --2- 172.21.15.5:0/264217113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd16c107df0 0x7fd16c0ff7e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd160005f50 tx=0x7fd160004270 comp rx=0 tx=0).stop 2024-05-14T15:22:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd16bfff700 1 -- 172.21.15.5:0/264217113 reap_dead start 2024-05-14T15:22:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 shutdown_connections 2024-05-14T15:22:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 >> 172.21.15.5:0/264217113 conn(0x7fd16c0fedc0 msgr2=0x7fd16c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 shutdown_connections 2024-05-14T15:22:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:57.100+0000 7fd171e99700 1 -- 172.21.15.5:0/264217113 wait complete. 2024-05-14T15:22:57.593 INFO:teuthology.orchestra.run.smithi005.stdout:304942678055 2024-05-14T15:22:57.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-14T15:22:58.493 INFO:teuthology.orchestra.run.smithi005.stdout:77309411411 2024-05-14T15:22:58.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-14T15:22:58.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:22:58 smithi005 ceph-mon[31559]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:58.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:22:58 smithi193 ceph-mon[36001]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:59.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:22:58 smithi144 ceph-mon[35650]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-14T15:22:59.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.070+0000 7fa2b16b5700 1 -- 172.21.15.5:0/3394210964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac108740 msgr2=0x7fa2ac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.070+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/3394210964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac108740 0x7fa2ac1100a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400b0d0 tx=0x7fa2a400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 -- 172.21.15.5:0/3394210964 shutdown_connections 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/3394210964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac108740 0x7fa2ac1100a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/3394210964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ac1034c0 0x7fa2ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/3394210964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2ac107df0 0x7fa2ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 -- 172.21.15.5:0/3394210964 >> 172.21.15.5:0/3394210964 conn(0x7fa2ac0fedc0 msgr2=0x7fa2ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 -- 172.21.15.5:0/3394210964 shutdown_connections 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 -- 172.21.15.5:0/3394210964 wait complete. 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.071+0000 7fa2b16b5700 1 Processor -- start 2024-05-14T15:22:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 -- start start 2024-05-14T15:22:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac1034c0 0x7fa2ac0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ac107df0 0x7fa2ac0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2ac108740 0x7fa2ac0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac113370 con 0x7fa2ac1034c0 2024-05-14T15:22:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1131f0 con 0x7fa2ac108740 2024-05-14T15:22:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2b16b5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1134f0 con 0x7fa2ac107df0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac1034c0 0x7fa2ac0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac1034c0 0x7fa2ac0ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37804/0 (socket says 172.21.15.5:37804) 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2aaffd700 1 -- 172.21.15.5:0/1636161511 learned_addr learned my addr 172.21.15.5:0/1636161511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2ab7fe700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2ac108740 0x7fa2ac0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.072+0000 7fa2aa7fc700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ac107df0 0x7fa2ac0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2aaffd700 1 -- 172.21.15.5:0/1636161511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ac107df0 msgr2=0x7fa2ac0ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2aaffd700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2ac107df0 0x7fa2ac0ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2aaffd700 1 -- 172.21.15.5:0/1636161511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2ac108740 msgr2=0x7fa2ac0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2aaffd700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2ac108740 0x7fa2ac0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2aaffd700 1 -- 172.21.15.5:0/1636161511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac1a9210 con 0x7fa2ac1034c0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2ab7fe700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2ac108740 0x7fa2ac0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2aaffd700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac1034c0 0x7fa2ac0ff2a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00d950 tx=0x7fa29c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c016070 con 0x7fa2ac1034c0 2024-05-14T15:22:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac1a9500 con 0x7fa2ac1034c0 2024-05-14T15:22:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29c00fd40 con 0x7fa2ac1034c0 2024-05-14T15:22:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00e3b0 con 0x7fa2ac1034c0 2024-05-14T15:22:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.073+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac1a9a90 con 0x7fa2ac1034c0 2024-05-14T15:22:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.074+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa2ac063ed0 con 0x7fa2ac1034c0 2024-05-14T15:22:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.076+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa29c019070 con 0x7fa2ac1034c0 2024-05-14T15:22:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2a3fff700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa2980b2910 0x7fa2980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2aa7fc700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa2980b2910 0x7fa2980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fa29c0e2790 con 0x7fa2ac1034c0 2024-05-14T15:22:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2a3fff700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] conn(0x7fa2980bdcc0 0x7fa2980c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:22:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 --> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2980c0880 con 0x7fa2980bdcc0 2024-05-14T15:22:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa29c0e2bc0 con 0x7fa2ac1034c0 2024-05-14T15:22:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.077+0000 7fa2ab7fe700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] conn(0x7fa2980bdcc0 0x7fa2980c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:22:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.078+0000 7fa2aa7fc700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa2980b2910 0x7fa2980b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac1098e0 tx=0x7fa29400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.078+0000 7fa2ab7fe700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] conn(0x7fa2980bdcc0 0x7fa2980c01b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:22:59.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.079+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== osd.3 v2:172.21.15.5:6810/431512675 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2980c0880 con 0x7fa2980bdcc0 2024-05-14T15:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.098+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 --> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa2ac064300 con 0x7fa2980bdcc0 2024-05-14T15:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.098+0000 7fa2a3fff700 1 -- 172.21.15.5:0/1636161511 <== osd.3 v2:172.21.15.5:6810/431512675 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa2ac064300 con 0x7fa2980bdcc0 2024-05-14T15:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] conn(0x7fa2980bdcc0 msgr2=0x7fa2980c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] conn(0x7fa2980bdcc0 0x7fa2980c01b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa2980b2910 msgr2=0x7fa2980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa2980b2910 0x7fa2980b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac1098e0 tx=0x7fa29400b410 comp rx=0 tx=0).stop 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac1034c0 msgr2=0x7fa2ac0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 --2- 172.21.15.5:0/1636161511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac1034c0 0x7fa2ac0ff2a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00d950 tx=0x7fa29c00dc60 comp rx=0 tx=0).stop 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2ab7fe700 1 -- 172.21.15.5:0/1636161511 reap_dead start 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 shutdown_connections 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 >> 172.21.15.5:0/1636161511 conn(0x7fa2ac0fedc0 msgr2=0x7fa2ac06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 shutdown_connections 2024-05-14T15:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:22:59.099+0000 7fa2b16b5700 1 -- 172.21.15.5:0/1636161511 wait complete. 2024-05-14T15:22:59.548 INFO:teuthology.orchestra.run.smithi005.stdout:279172874284 2024-05-14T15:22:59.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-14T15:23:00.290 INFO:teuthology.orchestra.run.smithi005.stdout:158913790022 2024-05-14T15:23:00.290 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-14T15:23:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 -- 172.21.15.5:0/3926657301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024109880 msgr2=0x7f30241111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 --2- 172.21.15.5:0/3926657301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024109880 0x7f30241111e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f301c00b0d0 tx=0x7f301c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 -- 172.21.15.5:0/3926657301 shutdown_connections 2024-05-14T15:23:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 --2- 172.21.15.5:0/3926657301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024109880 0x7f30241111e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 --2- 172.21.15.5:0/3926657301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3024104650 0x7f3024104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 --2- 172.21.15.5:0/3926657301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024108f30 0x7f3024109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 -- 172.21.15.5:0/3926657301 >> 172.21.15.5:0/3926657301 conn(0x7f3024079f00 msgr2=0x7f302407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 -- 172.21.15.5:0/3926657301 shutdown_connections 2024-05-14T15:23:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.736+0000 7f302ac66700 1 -- 172.21.15.5:0/3926657301 wait complete. 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 Processor -- start 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 -- start start 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3024104650 0x7f302419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024108f30 0x7f302419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024109880 0x7f302419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30241144b0 con 0x7f3024108f30 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3024114330 con 0x7f3024109880 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f302ac66700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3024114630 con 0x7f3024104650 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f3023fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024108f30 0x7f302419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f3023fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024108f30 0x7f302419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37816/0 (socket says 172.21.15.5:37816) 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f3023fff700 1 -- 172.21.15.5:0/2156762288 learned_addr learned my addr 172.21.15.5:0/2156762288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f3029203700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024109880 0x7f302419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.737+0000 7f3028a02700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3024104650 0x7f302419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3023fff700 1 -- 172.21.15.5:0/2156762288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3024104650 msgr2=0x7f302419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3023fff700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3024104650 0x7f302419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3023fff700 1 -- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024109880 msgr2=0x7f302419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3023fff700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024109880 0x7f302419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3023fff700 1 -- 172.21.15.5:0/2156762288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302419fcb0 con 0x7f3024108f30 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3023fff700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024108f30 0x7f302419b380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f301000e970 tx=0x7f301000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010004a30 con 0x7f3024108f30 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302419ffa0 con 0x7f3024108f30 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301000cc90 con 0x7f3024108f30 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.738+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010005610 con 0x7f3024108f30 2024-05-14T15:23:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.739+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241a78c0 con 0x7f3024108f30 2024-05-14T15:23:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.739+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3024063f50 con 0x7f3024108f30 2024-05-14T15:23:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.742+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f301000c7e0 con 0x7f3024108f30 2024-05-14T15:23:00.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3021ffb700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f30140b2910 0x7f30140b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:00.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3028a02700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f30140b2910 0x7f30140b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:00.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f30100e30c0 con 0x7f3024108f30 2024-05-14T15:23:00.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3021ffb700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] conn(0x7f30140bdcc0 0x7f30140c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:00.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 --> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3010004500 con 0x7f30140bdcc0 2024-05-14T15:23:00.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f30100e34f0 con 0x7f3024108f30 2024-05-14T15:23:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3029203700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] conn(0x7f30140bdcc0 0x7f30140c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.743+0000 7f3028a02700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f30140b2910 0x7f30140b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3018009c60 tx=0x7f3018009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.744+0000 7f3029203700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] conn(0x7f30140bdcc0 0x7f30140c01b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.745+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== osd.7 v2:172.21.15.144:6816/3028125205 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3010004500 con 0x7f30140bdcc0 2024-05-14T15:23:00.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.762+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 --> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30240643d0 con 0x7f30140bdcc0 2024-05-14T15:23:00.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.762+0000 7f3021ffb700 1 -- 172.21.15.5:0/2156762288 <== osd.7 v2:172.21.15.144:6816/3028125205 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30240643d0 con 0x7f30140bdcc0 2024-05-14T15:23:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] conn(0x7f30140bdcc0 msgr2=0x7f30140c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] conn(0x7f30140bdcc0 0x7f30140c01b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f30140b2910 msgr2=0x7f30140b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:00.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f30140b2910 0x7f30140b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3018009c60 tx=0x7f3018009260 comp rx=0 tx=0).stop 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024108f30 msgr2=0x7f302419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 --2- 172.21.15.5:0/2156762288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3024108f30 0x7f302419b380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f301000e970 tx=0x7f301000ec80 comp rx=0 tx=0).stop 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f3029203700 1 -- 172.21.15.5:0/2156762288 reap_dead start 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 shutdown_connections 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 >> 172.21.15.5:0/2156762288 conn(0x7f3024079f00 msgr2=0x7f3024101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 shutdown_connections 2024-05-14T15:23:00.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:00.763+0000 7f302ac66700 1 -- 172.21.15.5:0/2156762288 wait complete. 2024-05-14T15:23:00.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:00 smithi193 ceph-mon[36001]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T15:23:00.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:00 smithi005 ceph-mon[31559]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T15:23:01.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:00 smithi144 ceph-mon[35650]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T15:23:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:55834574938 2024-05-14T15:23:01.255 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-14T15:23:01.880 INFO:teuthology.orchestra.run.smithi005.stdout:111669149776 2024-05-14T15:23:01.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.064+0000 7fa92cd45700 1 -- 172.21.15.5:0/1040649749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928107df0 msgr2=0x7fa928108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.064+0000 7fa92cd45700 1 --2- 172.21.15.5:0/1040649749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928107df0 0x7fa928108200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0099c0 tx=0x7fa91c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 -- 172.21.15.5:0/1040649749 shutdown_connections 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 --2- 172.21.15.5:0/1040649749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928108740 0x7fa9281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 --2- 172.21.15.5:0/1040649749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9281034c0 0x7fa9281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 --2- 172.21.15.5:0/1040649749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928107df0 0x7fa928108200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 -- 172.21.15.5:0/1040649749 >> 172.21.15.5:0/1040649749 conn(0x7fa9280fedc0 msgr2=0x7fa9281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 -- 172.21.15.5:0/1040649749 shutdown_connections 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.065+0000 7fa92cd45700 1 -- 172.21.15.5:0/1040649749 wait complete. 2024-05-14T15:23:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 Processor -- start 2024-05-14T15:23:02.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 -- start start 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9281034c0 0x7fa92819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928107df0 0x7fa92819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928108740 0x7fa9281a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa928114480 con 0x7fa928108740 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa928114300 con 0x7fa928107df0 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa92cd45700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa928114600 con 0x7fa9281034c0 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa926d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928108740 0x7fa9281a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa926d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928108740 0x7fa9281a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37824/0 (socket says 172.21.15.5:37824) 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.066+0000 7fa926d9d700 1 -- 172.21.15.5:0/658957396 learned_addr learned my addr 172.21.15.5:0/658957396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:02.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa925d9b700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928107df0 0x7fa92819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa92659c700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9281034c0 0x7fa92819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa926d9d700 1 -- 172.21.15.5:0/658957396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9281034c0 msgr2=0x7fa92819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa926d9d700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9281034c0 0x7fa92819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa926d9d700 1 -- 172.21.15.5:0/658957396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928107df0 msgr2=0x7fa92819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa926d9d700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa928107df0 0x7fa92819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa926d9d700 1 -- 172.21.15.5:0/658957396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281a1ec0 con 0x7fa928108740 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa92659c700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9281034c0 0x7fa92819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa926d9d700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928108740 0x7fa9281a1820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa920007ee0 tx=0x7fa92000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92000dbd0 con 0x7fa928108740 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa92000dd70 con 0x7fa928108740 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9281a21b0 con 0x7fa928108740 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920005990 con 0x7fa928108740 2024-05-14T15:23:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.067+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92810a050 con 0x7fa928108740 2024-05-14T15:23:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.068+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa928106fb0 con 0x7fa928108740 2024-05-14T15:23:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.070+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa92001a020 con 0x7fa928108740 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa9177fe700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa9100b2990 0x7fa9100b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa92659c700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa9100b2990 0x7fa9100b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fa9200e2890 con 0x7fa928108740 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa9177fe700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] conn(0x7fa9100bdd30 0x7fa9100c0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 --> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9100c08f0 con 0x7fa9100bdd30 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa9200e2c40 con 0x7fa928108740 2024-05-14T15:23:02.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa925d9b700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] conn(0x7fa9100bdd30 0x7fa9100c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:02.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa92659c700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa9100b2990 0x7fa9100b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa918005950 tx=0x7fa9180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:02.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.071+0000 7fa925d9b700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] conn(0x7fa9100bdd30 0x7fa9100c0220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:02.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.072+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== osd.6 v2:172.21.15.5:6818/1238248767 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9100c08f0 con 0x7fa9100bdd30 2024-05-14T15:23:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.089+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 --> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa928104970 con 0x7fa9100bdd30 2024-05-14T15:23:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.089+0000 7fa9177fe700 1 -- 172.21.15.5:0/658957396 <== osd.6 v2:172.21.15.5:6818/1238248767 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa928104970 con 0x7fa9100bdd30 2024-05-14T15:23:02.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] conn(0x7fa9100bdd30 msgr2=0x7fa9100c0220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:02.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] conn(0x7fa9100bdd30 0x7fa9100c0220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa9100b2990 msgr2=0x7fa9100b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa9100b2990 0x7fa9100b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa918005950 tx=0x7fa9180058e0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928108740 msgr2=0x7fa9281a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 --2- 172.21.15.5:0/658957396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa928108740 0x7fa9281a1820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa920007ee0 tx=0x7fa92000d3b0 comp rx=0 tx=0).stop 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa926d9d700 1 -- 172.21.15.5:0/658957396 reap_dead start 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 shutdown_connections 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 >> 172.21.15.5:0/658957396 conn(0x7fa9280fedc0 msgr2=0x7fa9281096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 shutdown_connections 2024-05-14T15:23:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:02.090+0000 7fa92cd45700 1 -- 172.21.15.5:0/658957396 wait complete. 2024-05-14T15:23:02.769 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:02 smithi005 ceph-mon[31559]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:02.801 INFO:teuthology.orchestra.run.smithi005.stdout:184683593794 2024-05-14T15:23:02.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-14T15:23:03.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:02 smithi144 ceph-mon[35650]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:03.217 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:02 smithi193 ceph-mon[36001]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.183+0000 7f57f5159700 1 -- 172.21.15.5:0/1047422800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f0107df0 msgr2=0x7f57f0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.183+0000 7f57f5159700 1 --2- 172.21.15.5:0/1047422800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f0107df0 0x7f57f0108200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57d80099c0 tx=0x7f57d8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.183+0000 7f57f5159700 1 -- 172.21.15.5:0/1047422800 shutdown_connections 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.183+0000 7f57f5159700 1 --2- 172.21.15.5:0/1047422800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57f0108740 0x7f57f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.183+0000 7f57f5159700 1 --2- 172.21.15.5:0/1047422800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f01034c0 0x7f57f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.183+0000 7f57f5159700 1 --2- 172.21.15.5:0/1047422800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f0107df0 0x7f57f0108200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.184+0000 7f57f5159700 1 -- 172.21.15.5:0/1047422800 >> 172.21.15.5:0/1047422800 conn(0x7f57f00fedc0 msgr2=0x7f57f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.184+0000 7f57f5159700 1 -- 172.21.15.5:0/1047422800 shutdown_connections 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.184+0000 7f57f5159700 1 -- 172.21.15.5:0/1047422800 wait complete. 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.184+0000 7f57f5159700 1 Processor -- start 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 -- start start 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f01034c0 0x7f57f019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f0107df0 0x7f57f019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57f0108740 0x7f57f019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0113350 con 0x7f57f01034c0 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01131d0 con 0x7f57f0107df0 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57f5159700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01134d0 con 0x7f57f0108740 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f01034c0 0x7f57f019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:03.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f01034c0 0x7f57f019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37830/0 (socket says 172.21.15.5:37830) 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 -- 172.21.15.5:0/3104305010 learned_addr learned my addr 172.21.15.5:0/3104305010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57ee59c700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f0107df0 0x7f57f019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57ef59e700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57f0108740 0x7f57f019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 -- 172.21.15.5:0/3104305010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57f0108740 msgr2=0x7f57f019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57f0108740 0x7f57f019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 -- 172.21.15.5:0/3104305010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f0107df0 msgr2=0x7f57f019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f0107df0 0x7f57f019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.185+0000 7f57eed9d700 1 -- 172.21.15.5:0/3104305010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f01a6fa0 con 0x7f57f01034c0 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.186+0000 7f57eed9d700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f01034c0 0x7f57f019bc50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f57e000ef10 tx=0x7f57e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.186+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e000c9c0 con 0x7f57f01034c0 2024-05-14T15:23:03.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.186+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f01a7230 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.186+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e0019070 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.186+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0014480 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.186+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f01a7790 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.187+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57f0063e90 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.189+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f57e0012050 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.189+0000 7f57e7fff700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f57dc0b2990 0x7f57dc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.189+0000 7f57ee59c700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f57dc0b2990 0x7f57dc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.189+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f57e00e3080 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.190+0000 7f57e7fff700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] conn(0x7f57dc0bdd40 0x7f57dc0c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.190+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 --> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57dc0c0900 con 0x7f57dc0bdd40 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.190+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f57e00e34b0 con 0x7f57f01034c0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.190+0000 7f57ef59e700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] conn(0x7f57dc0bdd40 0x7f57dc0c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.190+0000 7f57ee59c700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f57dc0b2990 0x7f57dc0b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000c00 tx=0x7f57d801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.190+0000 7f57ef59e700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] conn(0x7f57dc0bdd40 0x7f57dc0c0230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.191+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== osd.0 v2:172.21.15.5:6802/1440842370 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57dc0c0900 con 0x7f57dc0bdd40 2024-05-14T15:23:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.210+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 --> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f57f00642c0 con 0x7f57dc0bdd40 2024-05-14T15:23:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57e7fff700 1 -- 172.21.15.5:0/3104305010 <== osd.0 v2:172.21.15.5:6802/1440842370 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f57f00642c0 con 0x7f57dc0bdd40 2024-05-14T15:23:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] conn(0x7f57dc0bdd40 msgr2=0x7f57dc0c0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] conn(0x7f57dc0bdd40 0x7f57dc0c0230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f57dc0b2990 msgr2=0x7f57dc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f57dc0b2990 0x7f57dc0b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000c00 tx=0x7f57d801a040 comp rx=0 tx=0).stop 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f01034c0 msgr2=0x7f57f019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 --2- 172.21.15.5:0/3104305010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f01034c0 0x7f57f019bc50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f57e000ef10 tx=0x7f57e000c3a0 comp rx=0 tx=0).stop 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57ef59e700 1 -- 172.21.15.5:0/3104305010 reap_dead start 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 shutdown_connections 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 >> 172.21.15.5:0/3104305010 conn(0x7f57f00fedc0 msgr2=0x7f57f01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 shutdown_connections 2024-05-14T15:23:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:03.211+0000 7f57f5159700 1 -- 172.21.15.5:0/3104305010 wait complete. 2024-05-14T15:23:04.114 INFO:teuthology.orchestra.run.smithi005.stdout:34359738464 2024-05-14T15:23:04.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-14T15:23:04.449 INFO:teuthology.orchestra.run.smithi005.stdout:137438953548 2024-05-14T15:23:04.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-14T15:23:04.743 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:04 smithi005 ceph-mon[31559]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 -- 172.21.15.5:0/3139730114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 msgr2=0x7f8c0c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3139730114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c1111d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400b0d0 tx=0x7f8c0400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:23:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 -- 172.21.15.5:0/3139730114 shutdown_connections 2024-05-14T15:23:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3139730114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c1111d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3139730114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c0c1045f0 0x7f8c0c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3139730114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c108f20 0x7f8c0c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 -- 172.21.15.5:0/3139730114 >> 172.21.15.5:0/3139730114 conn(0x7f8c0c100040 msgr2=0x7f8c0c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 -- 172.21.15.5:0/3139730114 shutdown_connections 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.978+0000 7f8c12c60700 1 -- 172.21.15.5:0/3139730114 wait complete. 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 Processor -- start 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 -- start start 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c1045f0 0x7f8c0c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c0c108f20 0x7f8c0c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1144a0 con 0x7f8c0c109870 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c114320 con 0x7f8c0c108f20 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c12c60700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c114620 con 0x7f8c0c1045f0 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.979+0000 7f8c111fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37876/0 (socket says 172.21.15.5:37876) 2024-05-14T15:23:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 -- 172.21.15.5:0/3374921203 learned_addr learned my addr 172.21.15.5:0/3374921203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c109fc700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c1045f0 0x7f8c0c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c0bfff700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c0c108f20 0x7f8c0c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 -- 172.21.15.5:0/3374921203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c1045f0 msgr2=0x7f8c0c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c1045f0 0x7f8c0c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 -- 172.21.15.5:0/3374921203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c0c108f20 msgr2=0x7f8c0c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c0c108f20 0x7f8c0c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 -- 172.21.15.5:0/3374921203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c19fd10 con 0x7f8c0c109870 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c109fc700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c0c1045f0 0x7f8c0c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c0bfff700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c0c108f20 0x7f8c0c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:23:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c111fd700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c19f670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c040095c0 tx=0x7f8c04007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:04.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0401f070 con 0x7f8c0c109870 2024-05-14T15:23:04.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c04003c00 con 0x7f8c0c109870 2024-05-14T15:23:04.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0401b6a0 con 0x7f8c0c109870 2024-05-14T15:23:04.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.980+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c19ffa0 con 0x7f8c0c109870 2024-05-14T15:23:04.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.981+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c0759f0 con 0x7f8c0c109870 2024-05-14T15:23:04.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.981+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c0c1a04d0 con 0x7f8c0c109870 2024-05-14T15:23:04.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.983+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f8c04022070 con 0x7f8c0c109870 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c09ffb700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bf40b2990 0x7f8bf40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c109fc700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bf40b2990 0x7f8bf40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f8c0400f020 con 0x7f8c0c109870 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c09ffb700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] conn(0x7f8bf40bdd40 0x7f8bf40c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 --> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bf40c0900 con 0x7f8bf40bdd40 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8c040e4090 con 0x7f8c0c109870 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.984+0000 7f8c0bfff700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] conn(0x7f8bf40bdd40 0x7f8bf40c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.985+0000 7f8c109fc700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bf40b2990 0x7f8bf40b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc005fd0 tx=0x7f8bfc005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.985+0000 7f8c0bfff700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] conn(0x7f8bf40bdd40 0x7f8bf40c0230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:04.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.986+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== osd.9 v2:172.21.15.5:6826/2591214668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bf40c0900 con 0x7f8bf40bdd40 2024-05-14T15:23:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.002+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 --> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c0c0640e0 con 0x7f8bf40bdd40 2024-05-14T15:23:05.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c09ffb700 1 -- 172.21.15.5:0/3374921203 <== osd.9 v2:172.21.15.5:6826/2591214668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c0c0640e0 con 0x7f8bf40bdd40 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] conn(0x7f8bf40bdd40 msgr2=0x7f8bf40c0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] conn(0x7f8bf40bdd40 0x7f8bf40c0230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bf40b2990 msgr2=0x7f8bf40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bf40b2990 0x7f8bf40b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc005fd0 tx=0x7f8bfc005dc0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 msgr2=0x7f8c0c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 --2- 172.21.15.5:0/3374921203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c0c109870 0x7f8c0c19f670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c040095c0 tx=0x7f8c04007ad0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c111fd700 1 -- 172.21.15.5:0/3374921203 reap_dead start 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 shutdown_connections 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.003+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 >> 172.21.15.5:0/3374921203 conn(0x7f8c0c100040 msgr2=0x7f8c0c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.004+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 shutdown_connections 2024-05-14T15:23:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:05.004+0000 7f8c12c60700 1 -- 172.21.15.5:0/3374921203 wait complete. 2024-05-14T15:23:05.046 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:04 smithi144 ceph-mon[35650]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:05.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:04 smithi193 ceph-mon[36001]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 -- 172.21.15.5:0/424990463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1039a0 msgr2=0x7f0dac103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/424990463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1039a0 0x7f0dac103db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009990 tx=0x7f0da0009ca0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 -- 172.21.15.5:0/424990463 shutdown_connections 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/424990463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dac1042f0 0x7f0dac110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/424990463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac108200 0x7f0dac108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/424990463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1039a0 0x7f0dac103db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 -- 172.21.15.5:0/424990463 >> 172.21.15.5:0/424990463 conn(0x7f0dac0fe4e0 msgr2=0x7f0dac100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 -- 172.21.15.5:0/424990463 shutdown_connections 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.742+0000 7f0db0d0d700 1 -- 172.21.15.5:0/424990463 wait complete. 2024-05-14T15:23:05.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.743+0000 7f0db0d0d700 1 Processor -- start 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.743+0000 7f0db0d0d700 1 -- start start 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0db0d0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac1039a0 0x7f0dac19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0db0d0d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1042f0 0x7f0dac19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0db0d0d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dac108200 0x7f0dac19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0db0d0d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1133a0 con 0x7f0dac1042f0 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0db0d0d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac113220 con 0x7f0dac1039a0 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0db0d0d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac113520 con 0x7f0dac108200 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1042f0 0x7f0dac19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1042f0 0x7f0dac19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37844/0 (socket says 172.21.15.5:37844) 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 -- 172.21.15.5:0/4229095451 learned_addr learned my addr 172.21.15.5:0/4229095451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0daad9d700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dac108200 0x7f0dac19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0daa59c700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac1039a0 0x7f0dac19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 -- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dac108200 msgr2=0x7f0dac19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dac108200 0x7f0dac19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 -- 172.21.15.5:0/4229095451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac1039a0 msgr2=0x7f0dac19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dac1039a0 0x7f0dac19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 -- 172.21.15.5:0/4229095451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19fc90 con 0x7f0dac1042f0 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.744+0000 7f0da9d9b700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1042f0 0x7f0dac19b360 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0da0000c00 tx=0x7f0da00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.745+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0027070 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.745+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19ff20 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.745+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da0003f20 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.745+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0022cf0 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.745+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a78a0 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.746+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0dac1a0450 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.747+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0da002a070 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.748+0000 7f0d9b7fe700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d940b2910 0x7f0d940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.748+0000 7f0daa59c700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d940b2910 0x7f0d940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.748+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f0da00e3d70 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.748+0000 7f0d9b7fe700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] conn(0x7f0d940bdcc0 0x7f0d940c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.748+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 --> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0da000b0b0 con 0x7f0d940bdcc0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.748+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0da00e41a0 con 0x7f0dac1042f0 2024-05-14T15:23:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.749+0000 7f0daa59c700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d940b2910 0x7f0d940b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0dac06c310 tx=0x7f0d9c00bef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.749+0000 7f0daad9d700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] conn(0x7f0d940bdcc0 0x7f0d940c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.749+0000 7f0daad9d700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] conn(0x7f0d940bdcc0 0x7f0d940c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.750+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== osd.8 v2:172.21.15.193:6816/1885686165 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0da000b0b0 con 0x7f0d940bdcc0 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.767+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 --> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0dac064160 con 0x7f0d940bdcc0 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0d9b7fe700 1 -- 172.21.15.5:0/4229095451 <== osd.8 v2:172.21.15.193:6816/1885686165 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0dac064160 con 0x7f0d940bdcc0 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] conn(0x7f0d940bdcc0 msgr2=0x7f0d940c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] conn(0x7f0d940bdcc0 0x7f0d940c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d940b2910 msgr2=0x7f0d940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f0d940b2910 0x7f0d940b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0dac06c310 tx=0x7f0d9c00bef0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1042f0 msgr2=0x7f0dac19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 --2- 172.21.15.5:0/4229095451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dac1042f0 0x7f0dac19b360 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0da0000c00 tx=0x7f0da00047b0 comp rx=0 tx=0).stop 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0daad9d700 1 -- 172.21.15.5:0/4229095451 reap_dead start 2024-05-14T15:23:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 shutdown_connections 2024-05-14T15:23:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 >> 172.21.15.5:0/4229095451 conn(0x7f0dac0fe4e0 msgr2=0x7f0dac102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 shutdown_connections 2024-05-14T15:23:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:04.768+0000 7f0db0d0d700 1 -- 172.21.15.5:0/4229095451 wait complete. 2024-05-14T15:23:05.760 INFO:teuthology.orchestra.run.smithi005.stdout:231928234041 2024-05-14T15:23:05.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-14T15:23:06.407 INFO:teuthology.orchestra.run.smithi005.stdout:257698037812 2024-05-14T15:23:06.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-14T15:23:06.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:06 smithi005 ceph-mon[31559]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:07.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:06 smithi144 ceph-mon[35650]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:07.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:06 smithi193 ceph-mon[36001]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:07.117 INFO:teuthology.orchestra.run.smithi005.stdout:206158430270 2024-05-14T15:23:07.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-14T15:23:07.212 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.225 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:07.343 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:08.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:08 smithi005 ceph-mon[31559]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T15:23:09.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:08 smithi144 ceph-mon[35650]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T15:23:09.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:08 smithi193 ceph-mon[36001]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-14T15:23:10.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:10 smithi005 ceph-mon[31559]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:11.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:10 smithi144 ceph-mon[35650]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:11.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:10 smithi193 ceph-mon[36001]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:12.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:12 smithi005 ceph-mon[31559]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:13.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:12 smithi144 ceph-mon[35650]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:13.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:12 smithi193 ceph-mon[36001]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:14.768 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:14 smithi005 ceph-mon[31559]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:15.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:14 smithi144 ceph-mon[35650]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:15.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:14 smithi193 ceph-mon[36001]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:16.693 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:16 smithi005 ceph-mon[31559]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:17.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:16 smithi144 ceph-mon[35650]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:17.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:16 smithi193 ceph-mon[36001]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 -- 172.21.15.5:0/576018077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 msgr2=0x7fa7dc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/576018077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc1100a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400b0d0 tx=0x7fa7d400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 -- 172.21.15.5:0/576018077 shutdown_connections 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/576018077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc1100a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/576018077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1034c0 0x7fa7dc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/576018077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 0x7fa7dc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 -- 172.21.15.5:0/576018077 >> 172.21.15.5:0/576018077 conn(0x7fa7dc0fedf0 msgr2=0x7fa7dc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 -- 172.21.15.5:0/576018077 shutdown_connections 2024-05-14T15:23:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.381+0000 7fa7e44d1700 1 -- 172.21.15.5:0/576018077 wait complete. 2024-05-14T15:23:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.382+0000 7fa7e44d1700 1 Processor -- start 2024-05-14T15:23:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.382+0000 7fa7e44d1700 1 -- start start 2024-05-14T15:23:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.382+0000 7fa7e44d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1034c0 0x7fa7dc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.382+0000 7fa7e44d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 0x7fa7dc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.382+0000 7fa7e44d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e44d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1144f0 con 0x7fa7dc108740 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e44d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc114370 con 0x7fa7dc1034c0 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e44d1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc114670 con 0x7fa7dc107df0 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46876/0 (socket says 172.21.15.5:46876) 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 -- 172.21.15.5:0/1973374242 learned_addr learned my addr 172.21.15.5:0/1973374242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e226d700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1034c0 0x7fa7dc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e1a6c700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 0x7fa7dc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 -- 172.21.15.5:0/1973374242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 msgr2=0x7fa7dc19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 0x7fa7dc19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 -- 172.21.15.5:0/1973374242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1034c0 msgr2=0x7fa7dc19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1034c0 0x7fa7dc19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 -- 172.21.15.5:0/1973374242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc19fc60 con 0x7fa7dc108740 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e1a6c700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 0x7fa7dc19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7e2a6e700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc19f670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009fd0 tx=0x7fa7d4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.383+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d401f070 con 0x7fa7dc108740 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.384+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d4003c00 con 0x7fa7dc108740 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.384+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc19fef0 con 0x7fa7dc108740 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.384+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d401b7b0 con 0x7fa7dc108740 2024-05-14T15:23:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.384+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc1a0420 con 0x7fa7dc108740 2024-05-14T15:23:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.385+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7dc0512c0 con 0x7fa7dc108740 2024-05-14T15:23:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.386+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa7d4022070 con 0x7fa7dc108740 2024-05-14T15:23:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.387+0000 7fa7d37fe700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa7c80b2990 0x7fa7c80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.387+0000 7fa7e226d700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa7c80b2990 0x7fa7c80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.387+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fa7d400f020 con 0x7fa7dc108740 2024-05-14T15:23:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.388+0000 7fa7e226d700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa7c80b2990 0x7fa7c80b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800ac50 tx=0x7fa7d800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.392+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7d4029020 con 0x7fa7dc108740 2024-05-14T15:23:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.613+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa7dc02e5f0 con 0x7fa7dc108740 2024-05-14T15:23:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.614+0000 7fa7d37fe700 1 -- 172.21.15.5:0/1973374242 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa7d40b3030 con 0x7fa7dc108740 2024-05-14T15:23:17.615 INFO:teuthology.orchestra.run.smithi005.stdout:111669149780 2024-05-14T15:23:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa7c80b2990 msgr2=0x7fa7c80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa7c80b2990 0x7fa7c80b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800ac50 tx=0x7fa7d800a380 comp rx=0 tx=0).stop 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 msgr2=0x7fa7dc19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc19f670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009fd0 tx=0x7fa7d4007a20 comp rx=0 tx=0).stop 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 shutdown_connections 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa7c80b2990 0x7fa7c80b4ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108740 0x7fa7dc19f670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc1034c0 0x7fa7dc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 --2- 172.21.15.5:0/1973374242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc107df0 0x7fa7dc19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 >> 172.21.15.5:0/1973374242 conn(0x7fa7dc0fedf0 msgr2=0x7fa7dc107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 shutdown_connections 2024-05-14T15:23:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:17.616+0000 7fa7e44d1700 1 -- 172.21.15.5:0/1973374242 wait complete. 2024-05-14T15:23:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 -- 172.21.15.5:0/1006987196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801034c0 msgr2=0x7fb9801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 --2- 172.21.15.5:0/1006987196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801034c0 0x7fb9801038d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0099c0 tx=0x7fb97c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 -- 172.21.15.5:0/1006987196 shutdown_connections 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 --2- 172.21.15.5:0/1006987196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb980108740 0x7fb9801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 --2- 172.21.15.5:0/1006987196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801034c0 0x7fb9801038d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 --2- 172.21.15.5:0/1006987196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb980107df0 0x7fb980108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 -- 172.21.15.5:0/1006987196 >> 172.21.15.5:0/1006987196 conn(0x7fb9800fedc0 msgr2=0x7fb9801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 -- 172.21.15.5:0/1006987196 shutdown_connections 2024-05-14T15:23:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.226+0000 7fb987474700 1 -- 172.21.15.5:0/1006987196 wait complete. 2024-05-14T15:23:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 Processor -- start 2024-05-14T15:23:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 -- start start 2024-05-14T15:23:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801034c0 0x7fb98019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 0x7fb98019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb980108740 0x7fb98019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb980113370 con 0x7fb980107df0 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801131f0 con 0x7fb9801034c0 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb987474700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801134f0 con 0x7fb980108740 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.227+0000 7fb984a0f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 0x7fb98019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 0x7fb98019c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46904/0 (socket says 172.21.15.5:46904) 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 -- 172.21.15.5:0/265858170 learned_addr learned my addr 172.21.15.5:0/265858170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb985210700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801034c0 0x7fb98019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb985a11700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb980108740 0x7fb98019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 -- 172.21.15.5:0/265858170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb980108740 msgr2=0x7fb98019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb980108740 0x7fb98019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 -- 172.21.15.5:0/265858170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801034c0 msgr2=0x7fb98019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801034c0 0x7fb98019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 -- 172.21.15.5:0/265858170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb980199ac0 con 0x7fb980107df0 2024-05-14T15:23:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb985210700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801034c0 0x7fb98019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.228+0000 7fb984a0f700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 0x7fb98019c210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c8f0 tx=0x7fb97000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.229+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700078d0 con 0x7fb980107df0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.229+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb97000f450 con 0x7fb980107df0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.229+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb980199db0 con 0x7fb980107df0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.229+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970018660 con 0x7fb980107df0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.229+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98019a340 con 0x7fb980107df0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.230+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb964005330 con 0x7fb980107df0 2024-05-14T15:23:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.231+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb970007a70 con 0x7fb980107df0 2024-05-14T15:23:18.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.232+0000 7fb9767fc700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb96c0b2990 0x7fb96c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.232+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fb9700e2370 con 0x7fb980107df0 2024-05-14T15:23:18.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.232+0000 7fb985210700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb96c0b2990 0x7fb96c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.233+0000 7fb985210700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb96c0b2990 0x7fb96c0b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009990 tx=0x7fb97c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:18.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.238+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb970014030 con 0x7fb980107df0 2024-05-14T15:23:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.457+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb9640058f0 con 0x7fb980107df0 2024-05-14T15:23:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.457+0000 7fb9767fc700 1 -- 172.21.15.5:0/265858170 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb9700a9c20 con 0x7fb980107df0 2024-05-14T15:23:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:231928234044 2024-05-14T15:23:18.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb96c0b2990 msgr2=0x7fb96c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb96c0b2990 0x7fb96c0b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009990 tx=0x7fb97c01a040 comp rx=0 tx=0).stop 2024-05-14T15:23:18.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 msgr2=0x7fb98019c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 0x7fb98019c210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb97000c8f0 tx=0x7fb97000cc00 comp rx=0 tx=0).stop 2024-05-14T15:23:18.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 shutdown_connections 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fb96c0b2990 0x7fb96c0b4ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb980108740 0x7fb98019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9801034c0 0x7fb98019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 --2- 172.21.15.5:0/265858170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980107df0 0x7fb98019c210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 >> 172.21.15.5:0/265858170 conn(0x7fb9800fedc0 msgr2=0x7fb9801096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 shutdown_connections 2024-05-14T15:23:18.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.460+0000 7fb987474700 1 -- 172.21.15.5:0/265858170 wait complete. 2024-05-14T15:23:18.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:18 smithi005 ceph-mon[31559]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-14T15:23:18.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:18 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1973374242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T15:23:18.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:18 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/265858170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T15:23:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 -- 172.21.15.5:0/2911898452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28107df0 msgr2=0x7f4b28108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/2911898452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28107df0 0x7f4b28108200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0099c0 tx=0x7f4b1c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 -- 172.21.15.5:0/2911898452 shutdown_connections 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/2911898452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28108740 0x7f4b281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/2911898452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b281034c0 0x7f4b281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/2911898452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28107df0 0x7f4b28108200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 -- 172.21.15.5:0/2911898452 >> 172.21.15.5:0/2911898452 conn(0x7f4b280fedc0 msgr2=0x7f4b281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 -- 172.21.15.5:0/2911898452 shutdown_connections 2024-05-14T15:23:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.749+0000 7f4b2d71b700 1 -- 172.21.15.5:0/2911898452 wait complete. 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 Processor -- start 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 -- start start 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b281034c0 0x7f4b2819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28107df0 0x7f4b2819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 0x7f4b2819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28113350 con 0x7f4b28108740 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281131d0 con 0x7f4b281034c0 2024-05-14T15:23:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.750+0000 7f4b2d71b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281134d0 con 0x7f4b28107df0 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 0x7f4b2819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 0x7f4b2819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46922/0 (socket says 172.21.15.5:46922) 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 -- 172.21.15.5:0/3626238030 learned_addr learned my addr 172.21.15.5:0/3626238030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b26ffd700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b281034c0 0x7f4b2819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b267fc700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28107df0 0x7f4b2819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 -- 172.21.15.5:0/3626238030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28107df0 msgr2=0x7f4b2819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28107df0 0x7f4b2819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 -- 172.21.15.5:0/3626238030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b281034c0 msgr2=0x7f4b2819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b281034c0 0x7f4b2819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 -- 172.21.15.5:0/3626238030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b281a6fa0 con 0x7f4b28108740 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b267fc700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28107df0 0x7f4b2819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b277fe700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 0x7f4b2819c6f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007ee0 tx=0x7f4b2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000dbd0 con 0x7f4b28108740 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281a7290 con 0x7f4b28108740 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b2000dd70 con 0x7f4b28108740 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.751+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20005990 con 0x7f4b28108740 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.752+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281a7750 con 0x7f4b28108740 2024-05-14T15:23:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.752+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280512c0 con 0x7f4b28108740 2024-05-14T15:23:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.754+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f4b2001a020 con 0x7f4b28108740 2024-05-14T15:23:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.755+0000 7f4b0ffff700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4b100b2910 0x7f4b100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.755+0000 7f4b26ffd700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4b100b2910 0x7f4b100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.755+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f4b200e25b0 con 0x7f4b28108740 2024-05-14T15:23:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.755+0000 7f4b26ffd700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4b100b2910 0x7f4b100b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b18007be0 tx=0x7f4b180073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.758+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b20010070 con 0x7f4b28108740 2024-05-14T15:23:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.979+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4b2802e7b0 con 0x7f4b28108740 2024-05-14T15:23:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.980+0000 7f4b0ffff700 1 -- 172.21.15.5:0/3626238030 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b200a9d60 con 0x7f4b28108740 2024-05-14T15:23:18.982 INFO:teuthology.orchestra.run.smithi005.stdout:34359738467 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4b100b2910 msgr2=0x7f4b100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4b100b2910 0x7f4b100b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b18007be0 tx=0x7f4b180073d0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 msgr2=0x7f4b2819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 0x7f4b2819c6f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007ee0 tx=0x7f4b2000d3b0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 shutdown_connections 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4b100b2910 0x7f4b100b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28108740 0x7f4b2819c6f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b281034c0 0x7f4b2819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 --2- 172.21.15.5:0/3626238030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b28107df0 0x7f4b2819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.982+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 >> 172.21.15.5:0/3626238030 conn(0x7f4b280fedc0 msgr2=0x7f4b281096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.983+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 shutdown_connections 2024-05-14T15:23:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:18.983+0000 7f4b2d71b700 1 -- 172.21.15.5:0/3626238030 wait complete. 2024-05-14T15:23:19.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:18 smithi144 ceph-mon[35650]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-14T15:23:19.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:18 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1973374242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T15:23:19.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:18 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/265858170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T15:23:19.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:18 smithi193 ceph-mon[36001]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-14T15:23:19.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:18 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1973374242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T15:23:19.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:18 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/265858170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T15:23:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:19 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3626238030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T15:23:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:19 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3626238030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T15:23:20.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:19 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3626238030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T15:23:20.754 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:20 smithi005 ceph-mon[31559]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-14T15:23:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:20 smithi144 ceph-mon[35650]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-14T15:23:21.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:20 smithi193 ceph-mon[36001]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-14T15:23:21.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738464 got 34359738467 for osd.0 2024-05-14T15:23:21.257 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 -- 172.21.15.5:0/101967104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 msgr2=0x7fd078108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 --2- 172.21.15.5:0/101967104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd078108200 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd0680099c0 tx=0x7fd068009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 -- 172.21.15.5:0/101967104 shutdown_connections 2024-05-14T15:23:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 --2- 172.21.15.5:0/101967104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd078108740 0x7fd0781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 --2- 172.21.15.5:0/101967104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0781034c0 0x7fd0781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 --2- 172.21.15.5:0/101967104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd078108200 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 -- 172.21.15.5:0/101967104 >> 172.21.15.5:0/101967104 conn(0x7fd0780fedf0 msgr2=0x7fd078101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 -- 172.21.15.5:0/101967104 shutdown_connections 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.524+0000 7fd07f733700 1 -- 172.21.15.5:0/101967104 wait complete. 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 Processor -- start 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 -- start start 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0781034c0 0x7fd07819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd07819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd078108740 0x7fd07819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd078113350 con 0x7fd078107df0 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781131d0 con 0x7fd078108740 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07f733700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781134d0 con 0x7fd0781034c0 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.525+0000 7fd07ccce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd07819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd07819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46934/0 (socket says 172.21.15.5:46934) 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 -- 172.21.15.5:0/4112777451 learned_addr learned my addr 172.21.15.5:0/4112777451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07d4cf700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0781034c0 0x7fd07819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07dcd0700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd078108740 0x7fd07819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 -- 172.21.15.5:0/4112777451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0781034c0 msgr2=0x7fd07819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0781034c0 0x7fd07819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stdout:158913790026 2024-05-14T15:23:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 -- 172.21.15.5:0/4112777451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd078108740 msgr2=0x7fd07819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd078108740 0x7fd07819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 -- 172.21.15.5:0/4112777451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07819fca0 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07dcd0700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd078108740 0x7fd07819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07ccce700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd07819b370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd068006010 tx=0x7fd0680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068027070 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd068003f20 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07819ff30 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068022cd0 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.526+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0781a0410 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.528+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0780512c0 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.529+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd06802a070 con 0x7fd078107df0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.530+0000 7fd06e7fc700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd0640b2910 0x7fd0640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.530+0000 7fd07d4cf700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd0640b2910 0x7fd0640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.530+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fd0680e3a00 con 0x7fd078107df0 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.531+0000 7fd07d4cf700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd0640b2910 0x7fd0640b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd074005950 tx=0x7fd07400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.534+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd068031080 con 0x7fd078107df0 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.753+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd07802e5f0 con 0x7fd078107df0 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.753+0000 7fd06e7fc700 1 -- 172.21.15.5:0/4112777451 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd068013070 con 0x7fd078107df0 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd0640b2910 msgr2=0x7fd0640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd0640b2910 0x7fd0640b4e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd074005950 tx=0x7fd07400a300 comp rx=0 tx=0).stop 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 msgr2=0x7fd07819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd07819b370 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd068006010 tx=0x7fd0680047b0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 shutdown_connections 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fd0640b2910 0x7fd0640b4e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd078108740 0x7fd07819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0781034c0 0x7fd07819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 --2- 172.21.15.5:0/4112777451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd078107df0 0x7fd07819b370 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 >> 172.21.15.5:0/4112777451 conn(0x7fd0780fedf0 msgr2=0x7fd078075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.756+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 shutdown_connections 2024-05-14T15:23:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:20.757+0000 7fd07f733700 1 -- 172.21.15.5:0/4112777451 wait complete. 2024-05-14T15:23:21.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:21 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4112777451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T15:23:22.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:21 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4112777451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T15:23:22.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:21 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4112777451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T15:23:22.390 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790022 got 158913790026 for osd.5 2024-05-14T15:23:22.390 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:22.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:22 smithi005 ceph-mon[31559]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-14T15:23:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:22 smithi144 ceph-mon[35650]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-14T15:23:23.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:22 smithi193 ceph-mon[36001]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-05-14T15:23:23.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 -- 172.21.15.5:0/3660512535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe90107df0 msgr2=0x7fbe90108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 --2- 172.21.15.5:0/3660512535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe90107df0 0x7fbe90108200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe840099c0 tx=0x7fbe84009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 -- 172.21.15.5:0/3660512535 shutdown_connections 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 --2- 172.21.15.5:0/3660512535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe90108740 0x7fbe901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 --2- 172.21.15.5:0/3660512535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe901034c0 0x7fbe901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 --2- 172.21.15.5:0/3660512535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe90107df0 0x7fbe90108200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 -- 172.21.15.5:0/3660512535 >> 172.21.15.5:0/3660512535 conn(0x7fbe900fedf0 msgr2=0x7fbe90101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 -- 172.21.15.5:0/3660512535 shutdown_connections 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.599+0000 7fbe98632700 1 -- 172.21.15.5:0/3660512535 wait complete. 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 Processor -- start 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 -- start start 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 0x7fbe9019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe90107df0 0x7fbe9019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe90108740 0x7fbe9019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90113350 con 0x7fbe901034c0 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901131d0 con 0x7fbe90108740 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.600+0000 7fbe98632700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901134d0 con 0x7fbe90107df0 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 0x7fbe9019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 0x7fbe9019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46952/0 (socket says 172.21.15.5:46952) 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 -- 172.21.15.5:0/254252598 learned_addr learned my addr 172.21.15.5:0/254252598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe96bcf700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe90108740 0x7fbe9019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe95bcd700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe90107df0 0x7fbe9019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 -- 172.21.15.5:0/254252598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe90107df0 msgr2=0x7fbe9019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe90107df0 0x7fbe9019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 -- 172.21.15.5:0/254252598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe90108740 msgr2=0x7fbe9019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe90108740 0x7fbe9019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 -- 172.21.15.5:0/254252598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9019fc90 con 0x7fbe901034c0 2024-05-14T15:23:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe95bcd700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe90107df0 0x7fbe9019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe963ce700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 0x7fbe9019ae70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbe800060b0 tx=0x7fbe80007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8001a070 con 0x7fbe901034c0 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9019ff80 con 0x7fbe901034c0 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe8000de20 con 0x7fbe901034c0 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.601+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80014720 con 0x7fbe901034c0 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.602+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901a04c0 con 0x7fbe901034c0 2024-05-14T15:23:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.602+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe900512c0 con 0x7fbe901034c0 2024-05-14T15:23:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.604+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fbe8001d070 con 0x7fbe901034c0 2024-05-14T15:23:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.605+0000 7fbe8f7fe700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe780b2910 0x7fbe780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.605+0000 7fbe95bcd700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe780b2910 0x7fbe780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.605+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fbe800e2590 con 0x7fbe901034c0 2024-05-14T15:23:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.606+0000 7fbe95bcd700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe780b2910 0x7fbe780b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006010 tx=0x7fbe84005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:23.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.608+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe80010070 con 0x7fbe901034c0 2024-05-14T15:23:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.825+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbe9002e5f0 con 0x7fbe901034c0 2024-05-14T15:23:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.826+0000 7fbe8f7fe700 1 -- 172.21.15.5:0/254252598 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbe800a9d40 con 0x7fbe901034c0 2024-05-14T15:23:23.829 INFO:teuthology.orchestra.run.smithi005.stdout:77309411418 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe780b2910 msgr2=0x7fbe780b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe780b2910 0x7fbe780b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006010 tx=0x7fbe84005e80 comp rx=0 tx=0).stop 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 msgr2=0x7fbe9019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 0x7fbe9019ae70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbe800060b0 tx=0x7fbe80007f90 comp rx=0 tx=0).stop 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 shutdown_connections 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbe780b2910 0x7fbe780b4e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe90108740 0x7fbe9019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe901034c0 0x7fbe9019ae70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 --2- 172.21.15.5:0/254252598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe90107df0 0x7fbe9019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:23.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 >> 172.21.15.5:0/254252598 conn(0x7fbe900fedf0 msgr2=0x7fbe90075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:23.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 shutdown_connections 2024-05-14T15:23:23.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:23.828+0000 7fbe98632700 1 -- 172.21.15.5:0/254252598 wait complete. 2024-05-14T15:23:24.632 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234041 got 231928234044 for osd.8 2024-05-14T15:23:24.632 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:24.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:24 smithi005 ceph-mon[31559]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-14T15:23:24.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:24 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/254252598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T15:23:25.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:24 smithi144 ceph-mon[35650]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-14T15:23:25.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:24 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/254252598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T15:23:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:24 smithi193 ceph-mon[36001]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-14T15:23:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:24 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/254252598' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T15:23:25.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149776 got 111669149780 for osd.3 2024-05-14T15:23:25.087 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 -- 172.21.15.5:0/205106172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4404108740 msgr2=0x7f44041100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 --2- 172.21.15.5:0/205106172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4404108740 0x7f44041100a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00b0d0 tx=0x7f43fc00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 -- 172.21.15.5:0/205106172 shutdown_connections 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 --2- 172.21.15.5:0/205106172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4404108740 0x7f44041100a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 --2- 172.21.15.5:0/205106172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44041034c0 0x7f44041038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 --2- 172.21.15.5:0/205106172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4404107df0 0x7f4404108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 -- 172.21.15.5:0/205106172 >> 172.21.15.5:0/205106172 conn(0x7f44040fedf0 msgr2=0x7f4404101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 -- 172.21.15.5:0/205106172 shutdown_connections 2024-05-14T15:23:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.279+0000 7f4409d86700 1 -- 172.21.15.5:0/205106172 wait complete. 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 Processor -- start 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 -- start start 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 0x7f440419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4404107df0 0x7f440419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4404108740 0x7f4404199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4404113370 con 0x7f44041034c0 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44041131f0 con 0x7f4404108740 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f4409d86700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44041134f0 con 0x7f4404107df0 2024-05-14T15:23:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.280+0000 7f44037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 0x7f440419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 0x7f440419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46970/0 (socket says 172.21.15.5:46970) 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 -- 172.21.15.5:0/3665005393 learned_addr learned my addr 172.21.15.5:0/3665005393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f43faffd700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4404107df0 0x7f440419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f4403fff700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4404108740 0x7f4404199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 -- 172.21.15.5:0/3665005393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4404107df0 msgr2=0x7f440419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4404107df0 0x7f440419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 -- 172.21.15.5:0/3665005393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4404108740 msgr2=0x7f4404199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4404108740 0x7f4404199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 -- 172.21.15.5:0/3665005393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440419a1d0 con 0x7f44041034c0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44037fe700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 0x7f440419f8b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d950 tx=0x7f43f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4016070 con 0x7f44041034c0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f400fd40 con 0x7f44041034c0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f400e3b0 con 0x7f44041034c0 2024-05-14T15:23:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440419a4c0 con 0x7f44041034c0 2024-05-14T15:23:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.281+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44041a6760 con 0x7f44041034c0 2024-05-14T15:23:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.283+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44040512c0 con 0x7f44041034c0 2024-05-14T15:23:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.284+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f43f4019070 con 0x7f44041034c0 2024-05-14T15:23:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.285+0000 7f44017fa700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f43f00b2990 0x7f43f00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.285+0000 7f43faffd700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f43f00b2990 0x7f43f00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.285+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f43f40e2220 con 0x7f44041034c0 2024-05-14T15:23:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.286+0000 7f43faffd700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f43f00b2990 0x7f43f00b4ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44041028a0 tx=0x7f43ec00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.290+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f43f40a99d0 con 0x7f44041034c0 2024-05-14T15:23:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.512+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f440402e5f0 con 0x7f44041034c0 2024-05-14T15:23:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.513+0000 7f44017fa700 1 -- 172.21.15.5:0/3665005393 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43f40a92d0 con 0x7f44041034c0 2024-05-14T15:23:25.514 INFO:teuthology.orchestra.run.smithi005.stdout:55834574944 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f43f00b2990 msgr2=0x7f43f00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f43f00b2990 0x7f43f00b4ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44041028a0 tx=0x7f43ec00a300 comp rx=0 tx=0).stop 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 msgr2=0x7f440419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 0x7f440419f8b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d950 tx=0x7f43f400dc60 comp rx=0 tx=0).stop 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 shutdown_connections 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f43f00b2990 0x7f43f00b4ea0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4404108740 0x7f4404199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44041034c0 0x7f440419f8b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 --2- 172.21.15.5:0/3665005393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4404107df0 0x7f440419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 >> 172.21.15.5:0/3665005393 conn(0x7f44040fedf0 msgr2=0x7f4404107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 shutdown_connections 2024-05-14T15:23:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.515+0000 7f4409d86700 1 -- 172.21.15.5:0/3665005393 wait complete. 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 -- 172.21.15.5:0/91659038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c1034c0 msgr2=0x7ff62c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 --2- 172.21.15.5:0/91659038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c1034c0 0x7ff62c1038d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff6280099c0 tx=0x7ff628009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 -- 172.21.15.5:0/91659038 shutdown_connections 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 --2- 172.21.15.5:0/91659038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff62c108740 0x7ff62c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 --2- 172.21.15.5:0/91659038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c1034c0 0x7ff62c1038d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 --2- 172.21.15.5:0/91659038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c107df0 0x7ff62c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 -- 172.21.15.5:0/91659038 >> 172.21.15.5:0/91659038 conn(0x7ff62c0fedc0 msgr2=0x7ff62c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:25.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 -- 172.21.15.5:0/91659038 shutdown_connections 2024-05-14T15:23:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.677+0000 7ff63449e700 1 -- 172.21.15.5:0/91659038 wait complete. 2024-05-14T15:23:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 Processor -- start 2024-05-14T15:23:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 -- start start 2024-05-14T15:23:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c1034c0 0x7ff62c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff62c107df0 0x7ff62c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 0x7ff62c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1133e0 con 0x7ff62c108740 2024-05-14T15:23:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c113260 con 0x7ff62c107df0 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63449e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c113560 con 0x7ff62c1034c0 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff632a3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 0x7ff62c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff632a3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 0x7ff62c19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38502/0 (socket says 172.21.15.5:38502) 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff632a3b700 1 -- 172.21.15.5:0/251701422 learned_addr learned my addr 172.21.15.5:0/251701422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff63223a700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c1034c0 0x7ff62c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.678+0000 7ff631a39700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff62c107df0 0x7ff62c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff632a3b700 1 -- 172.21.15.5:0/251701422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c1034c0 msgr2=0x7ff62c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff632a3b700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c1034c0 0x7ff62c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff632a3b700 1 -- 172.21.15.5:0/251701422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff62c107df0 msgr2=0x7ff62c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff632a3b700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff62c107df0 0x7ff62c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff632a3b700 1 -- 172.21.15.5:0/251701422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c19fcb0 con 0x7ff62c108740 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff632a3b700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 0x7ff62c19f610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff624007ee0 tx=0x7ff62400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624004bb0 con 0x7ff62c108740 2024-05-14T15:23:25.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c19ffa0 con 0x7ff62c108740 2024-05-14T15:23:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff624004d50 con 0x7ff62c108740 2024-05-14T15:23:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624005920 con 0x7ff62c108740 2024-05-14T15:23:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.679+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1a78c0 con 0x7ff62c108740 2024-05-14T15:23:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.680+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c0512c0 con 0x7ff62c108740 2024-05-14T15:23:25.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.683+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff62401a020 con 0x7ff62c108740 2024-05-14T15:23:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.683+0000 7ff6237fe700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff6180b2910 0x7ff6180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.684+0000 7ff63223a700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff6180b2910 0x7ff6180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.684+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7ff6240e2480 con 0x7ff62c108740 2024-05-14T15:23:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.684+0000 7ff63223a700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff6180b2910 0x7ff6180b4e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff628009990 tx=0x7ff628000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.687+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff624010070 con 0x7ff62c108740 2024-05-14T15:23:25.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3665005393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T15:23:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.899+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff62c02e5f0 con 0x7ff62c108740 2024-05-14T15:23:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.899+0000 7ff6237fe700 1 -- 172.21.15.5:0/251701422 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff6240a9c70 con 0x7ff62c108740 2024-05-14T15:23:25.900 INFO:teuthology.orchestra.run.smithi005.stdout:279172874290 2024-05-14T15:23:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff6180b2910 msgr2=0x7ff6180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff6180b2910 0x7ff6180b4e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff628009990 tx=0x7ff628000bc0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 msgr2=0x7ff62c19f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 0x7ff62c19f610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff624007ee0 tx=0x7ff62400d3b0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 shutdown_connections 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7ff6180b2910 0x7ff6180b4e20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff62c108740 0x7ff62c19f610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff62c1034c0 0x7ff62c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 --2- 172.21.15.5:0/251701422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff62c107df0 0x7ff62c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 >> 172.21.15.5:0/251701422 conn(0x7ff62c0fedc0 msgr2=0x7ff62c1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 shutdown_connections 2024-05-14T15:23:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:25.902+0000 7ff63449e700 1 -- 172.21.15.5:0/251701422 wait complete. 2024-05-14T15:23:26.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3665005393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T15:23:26.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3665005393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T15:23:26.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:26 smithi005 ceph-mon[31559]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:26.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:26 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/251701422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T15:23:26.948 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:26 smithi144 ceph-mon[35650]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:26.948 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:26 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/251701422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T15:23:26.954 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:26 smithi193 ceph-mon[36001]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-14T15:23:26.954 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:26 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/251701422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 -- 172.21.15.5:0/4107465817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 msgr2=0x7fc5b81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 --2- 172.21.15.5:0/4107465817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b81038f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80099c0 tx=0x7fc5a8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 -- 172.21.15.5:0/4107465817 shutdown_connections 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 --2- 172.21.15.5:0/4107465817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8108760 0x7fc5b81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 --2- 172.21.15.5:0/4107465817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b81038f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 --2- 172.21.15.5:0/4107465817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5b8107e10 0x7fc5b8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 -- 172.21.15.5:0/4107465817 >> 172.21.15.5:0/4107465817 conn(0x7fc5b80fef30 msgr2=0x7fc5b8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 -- 172.21.15.5:0/4107465817 shutdown_connections 2024-05-14T15:23:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.153+0000 7fc5be489700 1 -- 172.21.15.5:0/4107465817 wait complete. 2024-05-14T15:23:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 Processor -- start 2024-05-14T15:23:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 -- start start 2024-05-14T15:23:27.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8107e10 0x7fc5b819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5b8108760 0x7fc5b819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8113390 con 0x7fc5b81034e0 2024-05-14T15:23:27.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8113210 con 0x7fc5b8108760 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5be489700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8113510 con 0x7fc5b8107e10 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38524/0 (socket says 172.21.15.5:38524) 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.154+0000 7fc5b7fff700 1 -- 172.21.15.5:0/3705033264 learned_addr learned my addr 172.21.15.5:0/3705033264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5bca26700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5b8108760 0x7fc5b819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b77fe700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8107e10 0x7fc5b819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b7fff700 1 -- 172.21.15.5:0/3705033264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8107e10 msgr2=0x7fc5b819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b7fff700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8107e10 0x7fc5b819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b7fff700 1 -- 172.21.15.5:0/3705033264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5b8108760 msgr2=0x7fc5b819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b7fff700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5b8108760 0x7fc5b819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b7fff700 1 -- 172.21.15.5:0/3705033264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b819fc90 con 0x7fc5b81034e0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b7fff700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b819ae70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800b580 tx=0x7fc5a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8027070 con 0x7fc5b81034e0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a8003f20 con 0x7fc5b81034e0 2024-05-14T15:23:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b819ff20 con 0x7fc5b81034e0 2024-05-14T15:23:27.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8022d10 con 0x7fc5b81034e0 2024-05-14T15:23:27.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.155+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b81a0400 con 0x7fc5b81034e0 2024-05-14T15:23:27.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.156+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b80512c0 con 0x7fc5b81034e0 2024-05-14T15:23:27.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.158+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc5a8022470 con 0x7fc5b81034e0 2024-05-14T15:23:27.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.159+0000 7fc5b57fa700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc5a00b2910 0x7fc5a00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.159+0000 7fc5b77fe700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc5a00b2910 0x7fc5a00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.159+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fc5a80e3790 con 0x7fc5b81034e0 2024-05-14T15:23:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.160+0000 7fc5b77fe700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc5a00b2910 0x7fc5a00b4e20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8102950 tx=0x7fc5ac00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:27.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.163+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5a8013070 con 0x7fc5b81034e0 2024-05-14T15:23:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.357+0000 7f44839c4700 1 -- 172.21.15.5:0/3124264076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c107df0 msgr2=0x7f447c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.357+0000 7f44839c4700 1 --2- 172.21.15.5:0/3124264076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c107df0 0x7f447c108200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f446c0099c0 tx=0x7f446c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 -- 172.21.15.5:0/3124264076 shutdown_connections 2024-05-14T15:23:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 --2- 172.21.15.5:0/3124264076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c108740 0x7f447c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 --2- 172.21.15.5:0/3124264076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1034c0 0x7f447c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 --2- 172.21.15.5:0/3124264076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c107df0 0x7f447c108200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 -- 172.21.15.5:0/3124264076 >> 172.21.15.5:0/3124264076 conn(0x7f447c0fedf0 msgr2=0x7f447c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 -- 172.21.15.5:0/3124264076 shutdown_connections 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 -- 172.21.15.5:0/3124264076 wait complete. 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 Processor -- start 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.358+0000 7f44839c4700 1 -- start start 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f44839c4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1034c0 0x7f447c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f44839c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c107df0 0x7f447c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f44839c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 0x7f447c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f44839c4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f447c113370 con 0x7f447c108740 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f44839c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1131f0 con 0x7f447c107df0 2024-05-14T15:23:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f44839c4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1134f0 con 0x7f447c1034c0 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 0x7f447c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 0x7f447c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38536/0 (socket says 172.21.15.5:38536) 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 -- 172.21.15.5:0/1028882020 learned_addr learned my addr 172.21.15.5:0/1028882020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481760700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1034c0 0x7f447c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4480f5f700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c107df0 0x7f447c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 -- 172.21.15.5:0/1028882020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1034c0 msgr2=0x7f447c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1034c0 0x7f447c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 -- 172.21.15.5:0/1028882020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c107df0 msgr2=0x7f447c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c107df0 0x7f447c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 -- 172.21.15.5:0/1028882020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c19fcc0 con 0x7f447c108740 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.359+0000 7f4481f61700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 0x7f447c19f620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4474007ee0 tx=0x7f447400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.360+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474004bb0 con 0x7f447c108740 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.360+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4474004d50 con 0x7f447c108740 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.360+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c19ffb0 con 0x7f447c108740 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.360+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474005920 con 0x7f447c108740 2024-05-14T15:23:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.360+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a78d0 con 0x7f447c108740 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.361+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c0512c0 con 0x7f447c108740 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.364+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f447401a020 con 0x7f447c108740 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.364+0000 7f44727fc700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f44680b2990 0x7f44680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.364+0000 7f4481760700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f44680b2990 0x7f44680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.364+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f44740e2360 con 0x7f447c108740 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.365+0000 7f4481760700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f44680b2990 0x7f44680b4ea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4478004630 tx=0x7f447800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.367+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4474010070 con 0x7f447c108740 2024-05-14T15:23:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:27.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.392+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc5b81a7950 con 0x7fc5b81034e0 2024-05-14T15:23:27.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.393+0000 7fc5b57fa700 1 -- 172.21.15.5:0/3705033264 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc5a80ab010 con 0x7fc5b81034e0 2024-05-14T15:23:27.394 INFO:teuthology.orchestra.run.smithi005.stdout:137438953553 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.396+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc5a00b2910 msgr2=0x7fc5a00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.396+0000 7fc5be489700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc5a00b2910 0x7fc5a00b4e20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8102950 tx=0x7fc5ac00a380 comp rx=0 tx=0).stop 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.396+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 msgr2=0x7fc5b819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.396+0000 7fc5be489700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b819ae70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800b580 tx=0x7fc5a80047b0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 shutdown_connections 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc5a00b2910 0x7fc5a00b4e20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5b8108760 0x7fc5b819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b81034e0 0x7fc5b819ae70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 --2- 172.21.15.5:0/3705033264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8107e10 0x7fc5b819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 >> 172.21.15.5:0/3705033264 conn(0x7fc5b80fef30 msgr2=0x7fc5b81096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 shutdown_connections 2024-05-14T15:23:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.397+0000 7fc5be489700 1 -- 172.21.15.5:0/3705033264 wait complete. 2024-05-14T15:23:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.618+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f447c02e5f0 con 0x7f447c108740 2024-05-14T15:23:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.618+0000 7f44727fc700 1 -- 172.21.15.5:0/1028882020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f44740a9b50 con 0x7f447c108740 2024-05-14T15:23:27.619 INFO:teuthology.orchestra.run.smithi005.stdout:304942678062 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.621+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f44680b2990 msgr2=0x7f44680b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.621+0000 7f44839c4700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f44680b2990 0x7f44680b4ea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4478004630 tx=0x7f447800b3c0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.621+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 msgr2=0x7f447c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.621+0000 7f44839c4700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 0x7f447c19f620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4474007ee0 tx=0x7f447400d3b0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.621+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 shutdown_connections 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f44680b2990 0x7f44680b4ea0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447c108740 0x7f447c19f620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1034c0 0x7f447c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 --2- 172.21.15.5:0/1028882020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c107df0 0x7f447c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 >> 172.21.15.5:0/1028882020 conn(0x7f447c0fedf0 msgr2=0x7f447c075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 shutdown_connections 2024-05-14T15:23:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:27.622+0000 7f44839c4700 1 -- 172.21.15.5:0/1028882020 wait complete. 2024-05-14T15:23:28.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:27 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3705033264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T15:23:28.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874284 got 279172874290 for osd.10 2024-05-14T15:23:28.035 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:28.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:27 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3705033264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T15:23:28.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:27 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3705033264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T15:23:28.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:28 smithi005 ceph-mon[31559]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T15:23:28.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:28 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1028882020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T15:23:28.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953548 got 137438953553 for osd.4 2024-05-14T15:23:28.917 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:29.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:28 smithi144 ceph-mon[35650]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T15:23:29.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:28 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1028882020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T15:23:29.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:28 smithi193 ceph-mon[36001]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-14T15:23:29.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:28 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1028882020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T15:23:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.352+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2628914656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb241034c0 msgr2=0x7fcb241038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.352+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2628914656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb241034c0 0x7fcb241038d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcb200099c0 tx=0x7fcb20009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2628914656 shutdown_connections 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2628914656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb24108740 0x7fcb241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2628914656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb241034c0 0x7fcb241038d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2628914656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb24107df0 0x7fcb24108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2628914656 >> 172.21.15.5:0/2628914656 conn(0x7fcb240fedf0 msgr2=0x7fcb24101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2628914656 shutdown_connections 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2628914656 wait complete. 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 Processor -- start 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.353+0000 7fcb2c7ae700 1 -- start start 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2c7ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241034c0 0x7fcb24199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2c7ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 0x7fcb2419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2c7ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb24108740 0x7fcb2419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2c7ae700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24113370 con 0x7fcb24107df0 2024-05-14T15:23:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2c7ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241131f0 con 0x7fcb241034c0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2c7ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241134f0 con 0x7fcb24108740 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 0x7fcb2419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 0x7fcb2419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38548/0 (socket says 172.21.15.5:38548) 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 -- 172.21.15.5:0/2917284384 learned_addr learned my addr 172.21.15.5:0/2917284384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2ad4b700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb24108740 0x7fcb2419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb2a54a700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241034c0 0x7fcb24199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 -- 172.21.15.5:0/2917284384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb24108740 msgr2=0x7fcb2419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb24108740 0x7fcb2419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 -- 172.21.15.5:0/2917284384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241034c0 msgr2=0x7fcb24199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241034c0 0x7fcb24199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 -- 172.21.15.5:0/2917284384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2419eb40 con 0x7fcb24107df0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.354+0000 7fcb29d49700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 0x7fcb2419a2c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c9e0 tx=0x7fcb1400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.355+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14007aa0 con 0x7fcb24107df0 2024-05-14T15:23:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.355+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb14007c40 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.355+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2419ee30 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.355+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400fe60 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.355+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2419f370 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.356+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb240512c0 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.357+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcb1400f450 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.358+0000 7fcb1b7fe700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb100b2910 0x7fcb100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.358+0000 7fcb2a54a700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb100b2910 0x7fcb100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.358+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fcb140e2940 con 0x7fcb24107df0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.359+0000 7fcb2a54a700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb100b2910 0x7fcb100b4e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009990 tx=0x7fcb20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.362+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb14014030 con 0x7fcb24107df0 2024-05-14T15:23:29.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411411 got 77309411418 for osd.2 2024-05-14T15:23:29.480 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.591+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcb2419b080 con 0x7fcb24107df0 2024-05-14T15:23:29.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.591+0000 7fcb1b7fe700 1 -- 172.21.15.5:0/2917284384 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb140aa1c0 con 0x7fcb24107df0 2024-05-14T15:23:29.592 INFO:teuthology.orchestra.run.smithi005.stdout:184683593799 2024-05-14T15:23:29.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb100b2910 msgr2=0x7fcb100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb100b2910 0x7fcb100b4e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009990 tx=0x7fcb20000bc0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 msgr2=0x7fcb2419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 0x7fcb2419a2c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c9e0 tx=0x7fcb1400ccf0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 shutdown_connections 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb100b2910 0x7fcb100b4e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb24108740 0x7fcb2419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241034c0 0x7fcb24199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 --2- 172.21.15.5:0/2917284384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb24107df0 0x7fcb2419a2c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 >> 172.21.15.5:0/2917284384 conn(0x7fcb240fedf0 msgr2=0x7fcb24101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 shutdown_connections 2024-05-14T15:23:29.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:29.594+0000 7fcb2c7ae700 1 -- 172.21.15.5:0/2917284384 wait complete. 2024-05-14T15:23:29.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:29 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2917284384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T15:23:29.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574938 got 55834574944 for osd.1 2024-05-14T15:23:29.869 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:30.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:29 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2917284384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T15:23:30.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:29 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2917284384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T15:23:30.498 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678055 got 304942678062 for osd.11 2024-05-14T15:23:30.498 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:30.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:30 smithi005 ceph-mon[31559]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:31.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:30 smithi144 ceph-mon[35650]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:31.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:30 smithi193 ceph-mon[36001]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:31.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:31 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2379555841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T15:23:32.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:31 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2379555841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T15:23:32.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:31 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2379555841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 -- 172.21.15.5:0/2553805151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c1034c0 msgr2=0x7fcf7c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 --2- 172.21.15.5:0/2553805151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c1038d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b0d0 tx=0x7fcf6c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 -- 172.21.15.5:0/2553805151 shutdown_connections 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 --2- 172.21.15.5:0/2553805151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf7c108740 0x7fcf7c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 --2- 172.21.15.5:0/2553805151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c1038d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 --2- 172.21.15.5:0/2553805151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c107df0 0x7fcf7c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 -- 172.21.15.5:0/2553805151 >> 172.21.15.5:0/2553805151 conn(0x7fcf7c0fedc0 msgr2=0x7fcf7c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:32.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 -- 172.21.15.5:0/2553805151 shutdown_connections 2024-05-14T15:23:32.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 -- 172.21.15.5:0/2553805151 wait complete. 2024-05-14T15:23:32.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.245+0000 7fcf81e58700 1 Processor -- start 2024-05-14T15:23:32.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 -- start start 2024-05-14T15:23:32.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf7c107df0 0x7fcf7c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 0x7fcf7c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1133c0 con 0x7fcf7c108740 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c113240 con 0x7fcf7c107df0 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf81e58700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c113540 con 0x7fcf7c1034c0 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf7bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 0x7fcf7c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf7bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 0x7fcf7c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38588/0 (socket says 172.21.15.5:38588) 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf7bfff700 1 -- 172.21.15.5:0/1359425745 learned_addr learned my addr 172.21.15.5:0/1359425745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf7affd700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf7c107df0 0x7fcf7c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.246+0000 7fcf7b7fe700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7bfff700 1 -- 172.21.15.5:0/1359425745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c1034c0 msgr2=0x7fcf7c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7bfff700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7bfff700 1 -- 172.21.15.5:0/1359425745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf7c107df0 msgr2=0x7fcf7c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7bfff700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf7c107df0 0x7fcf7c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7bfff700 1 -- 172.21.15.5:0/1359425745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c1a9130 con 0x7fcf7c108740 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7b7fe700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf7bfff700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 0x7fcf7c19a530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400c930 tx=0x7fcf7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74007ac0 con 0x7fcf7c108740 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf74007c60 con 0x7fcf7c108740 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c1a9420 con 0x7fcf7c108740 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74018660 con 0x7fcf7c108740 2024-05-14T15:23:32.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.247+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a9910 con 0x7fcf7c108740 2024-05-14T15:23:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.248+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c104970 con 0x7fcf7c108740 2024-05-14T15:23:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.250+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcf7400f450 con 0x7fcf7c108740 2024-05-14T15:23:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.250+0000 7fcf78ff9700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcf680b2910 0x7fcf680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.251+0000 7fcf7b7fe700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcf680b2910 0x7fcf680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.251+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fcf740e2600 con 0x7fcf7c108740 2024-05-14T15:23:32.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.251+0000 7fcf7b7fe700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcf680b2910 0x7fcf680b4e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00d010 tx=0x7fcf6c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.255+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf74014030 con 0x7fcf7c108740 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 -- 172.21.15.5:0/3175649374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c109870 msgr2=0x7fcb6c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 --2- 172.21.15.5:0/3175649374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c109870 0x7fcb6c1111d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcb640099c0 tx=0x7fcb64009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 -- 172.21.15.5:0/3175649374 shutdown_connections 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 --2- 172.21.15.5:0/3175649374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c109870 0x7fcb6c1111d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 --2- 172.21.15.5:0/3175649374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb6c1045f0 0x7fcb6c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 --2- 172.21.15.5:0/3175649374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c108f20 0x7fcb6c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.826+0000 7fcb733f9700 1 -- 172.21.15.5:0/3175649374 >> 172.21.15.5:0/3175649374 conn(0x7fcb6c100040 msgr2=0x7fcb6c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.827+0000 7fcb733f9700 1 -- 172.21.15.5:0/3175649374 shutdown_connections 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.827+0000 7fcb733f9700 1 -- 172.21.15.5:0/3175649374 wait complete. 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.827+0000 7fcb733f9700 1 Processor -- start 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.827+0000 7fcb733f9700 1 -- start start 2024-05-14T15:23:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb733f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c1045f0 0x7fcb6c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb733f9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 0x7fcb6c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb733f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb6c109870 0x7fcb6c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb733f9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c1144a0 con 0x7fcb6c108f20 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb733f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c114320 con 0x7fcb6c1045f0 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb733f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c114620 con 0x7fcb6c109870 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 0x7fcb6c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 0x7fcb6c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38572/0 (socket says 172.21.15.5:38572) 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 -- 172.21.15.5:0/2379555841 learned_addr learned my addr 172.21.15.5:0/2379555841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb71996700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb6c109870 0x7fcb6c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb71195700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c1045f0 0x7fcb6c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 -- 172.21.15.5:0/2379555841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb6c109870 msgr2=0x7fcb6c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb6c109870 0x7fcb6c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 -- 172.21.15.5:0/2379555841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c1045f0 msgr2=0x7fcb6c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c1045f0 0x7fcb6c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 -- 172.21.15.5:0/2379555841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c19fce0 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.828+0000 7fcb70994700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 0x7fcb6c19b3b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c8f0 tx=0x7fcb5c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.829+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0078d0 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.829+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c19ffd0 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.829+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5c00f450 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.829+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c018660 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.829+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c1a78f0 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.830+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c0512c0 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.831+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fcb5c00f900 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.833+0000 7fcb627fc700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb580b2910 0x7fcb580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.833+0000 7fcb71195700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb580b2910 0x7fcb580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.833+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fcb5c0e2590 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.833+0000 7fcb71195700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb580b2910 0x7fcb580b4e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009550 tx=0x7fcb680091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:30.837+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb5c014030 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.055+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fcb6c19c170 con 0x7fcb6c108f20 2024-05-14T15:23:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.055+0000 7fcb627fc700 1 -- 172.21.15.5:0/2379555841 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb5c0a9e10 con 0x7fcb6c108f20 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stdout:257698037817 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb580b2910 msgr2=0x7fcb580b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb580b2910 0x7fcb580b4e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009550 tx=0x7fcb680091b0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 msgr2=0x7fcb6c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 0x7fcb6c19b3b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c8f0 tx=0x7fcb5c00cc00 comp rx=0 tx=0).stop 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 shutdown_connections 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcb580b2910 0x7fcb580b4e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb6c109870 0x7fcb6c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c1045f0 0x7fcb6c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 --2- 172.21.15.5:0/2379555841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb6c108f20 0x7fcb6c19b3b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 >> 172.21.15.5:0/2379555841 conn(0x7fcb6c100040 msgr2=0x7fcb6c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 shutdown_connections 2024-05-14T15:23:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:31.057+0000 7fcb733f9700 1 -- 172.21.15.5:0/2379555841 wait complete. 2024-05-14T15:23:32.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.470+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcf7c06bc00 con 0x7fcf7c108740 2024-05-14T15:23:32.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.471+0000 7fcf78ff9700 1 -- 172.21.15.5:0/1359425745 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcf740b0e20 con 0x7fcf7c108740 2024-05-14T15:23:32.471 INFO:teuthology.orchestra.run.smithi005.stdout:206158430276 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcf680b2910 msgr2=0x7fcf680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcf680b2910 0x7fcf680b4e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00d010 tx=0x7fcf6c00b040 comp rx=0 tx=0).stop 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 msgr2=0x7fcf7c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 0x7fcf7c19a530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400c930 tx=0x7fcf7400cc40 comp rx=0 tx=0).stop 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 shutdown_connections 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fcf680b2910 0x7fcf680b4e20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf7c108740 0x7fcf7c19a530 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c1034c0 0x7fcf7c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 --2- 172.21.15.5:0/1359425745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf7c107df0 0x7fcf7c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 >> 172.21.15.5:0/1359425745 conn(0x7fcf7c0fedc0 msgr2=0x7fcf7c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 shutdown_connections 2024-05-14T15:23:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:32.473+0000 7fcf81e58700 1 -- 172.21.15.5:0/1359425745 wait complete. 2024-05-14T15:23:32.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:32 smithi005 ceph-mon[31559]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:32.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:32 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1359425745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T15:23:33.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:32 smithi144 ceph-mon[35650]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:33.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:32 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1359425745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T15:23:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:32 smithi193 ceph-mon[36001]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:32 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1359425745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T15:23:33.124 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037812 got 257698037817 for osd.9 2024-05-14T15:23:33.124 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:33.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430270 got 206158430276 for osd.7 2024-05-14T15:23:33.763 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:34.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593794 got 184683593799 for osd.6 2024-05-14T15:23:34.394 DEBUG:teuthology.parallel:result is None 2024-05-14T15:23:34.394 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T15:23:34.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T15:23:34.691 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:34.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:34 smithi005 ceph-mon[31559]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:34.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:34.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:34 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:34 smithi144 ceph-mon[35650]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:34 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:35.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:34 smithi193 ceph-mon[36001]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:35.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:35.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:34 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:35.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:23:35.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:23:35.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:36.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:23:36.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:23:36.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:36.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:23:36.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:23:36.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:23:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 -- 172.21.15.5:0/3414414400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c103310 msgr2=0x7fea3c10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 --2- 172.21.15.5:0/3414414400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c103310 0x7fea3c10f890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea34009990 tx=0x7fea34009ca0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 -- 172.21.15.5:0/3414414400 shutdown_connections 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 --2- 172.21.15.5:0/3414414400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c103310 0x7fea3c10f890 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 --2- 172.21.15.5:0/3414414400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea3c1089f0 0x7fea3c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 --2- 172.21.15.5:0/3414414400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea3c1029c0 0x7fea3c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 -- 172.21.15.5:0/3414414400 >> 172.21.15.5:0/3414414400 conn(0x7fea3c0fe520 msgr2=0x7fea3c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 -- 172.21.15.5:0/3414414400 shutdown_connections 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.499+0000 7fea42e94700 1 -- 172.21.15.5:0/3414414400 wait complete. 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.500+0000 7fea42e94700 1 Processor -- start 2024-05-14T15:23:36.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.500+0000 7fea42e94700 1 -- start start 2024-05-14T15:23:36.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea42e94700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 0x7fea3c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea42e94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea3c103310 0x7fea3c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea42e94700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea3c1089f0 0x7fea3c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea42e94700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c114520 con 0x7fea3c1029c0 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea42e94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1143a0 con 0x7fea3c103310 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea42e94700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1146a0 con 0x7fea3c1089f0 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 0x7fea3c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 0x7fea3c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48016/0 (socket says 172.21.15.5:48016) 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 -- 172.21.15.5:0/3237766512 learned_addr learned my addr 172.21.15.5:0/3237766512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea3bfff700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea3c103310 0x7fea3c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea41431700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea3c1089f0 0x7fea3c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 -- 172.21.15.5:0/3237766512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea3c1089f0 msgr2=0x7fea3c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea3c1089f0 0x7fea3c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 -- 172.21.15.5:0/3237766512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea3c103310 msgr2=0x7fea3c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea3c103310 0x7fea3c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 -- 172.21.15.5:0/3237766512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c19fc80 con 0x7fea3c1029c0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.501+0000 7fea40c30700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 0x7fea3c19adb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00d980 tx=0x7fea2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.502+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c017070 con 0x7fea3c1029c0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.502+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c19ff70 con 0x7fea3c1029c0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.502+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c00fd40 con 0x7fea3c1029c0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.502+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00e3b0 con 0x7fea3c1029c0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.502+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1a7890 con 0x7fea3c1029c0 2024-05-14T15:23:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.503+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c0512c0 con 0x7fea3c1029c0 2024-05-14T15:23:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.504+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fea2c010610 con 0x7fea3c1029c0 2024-05-14T15:23:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.505+0000 7fea39ffb700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fea280b2910 0x7fea280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.505+0000 7fea3bfff700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fea280b2910 0x7fea280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.505+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fea2c0e2170 con 0x7fea3c1029c0 2024-05-14T15:23:36.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.506+0000 7fea3bfff700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fea280b2910 0x7fea280b4e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fea24009550 tx=0x7fea240091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:36.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.510+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea2c0a9960 con 0x7fea3c1029c0 2024-05-14T15:23:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.719+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea3c19c110 con 0x7fea280b2910 2024-05-14T15:23:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.729+0000 7fea39ffb700 1 -- 172.21.15.5:0/3237766512 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340622 (secure 0 0 0) 0x7fea3c19c110 con 0x7fea280b2910 2024-05-14T15:23:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:36.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fea280b2910 msgr2=0x7fea280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fea280b2910 0x7fea280b4e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fea24009550 tx=0x7fea240091b0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 msgr2=0x7fea3c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 0x7fea3c19adb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00d980 tx=0x7fea2c00dc90 comp rx=0 tx=0).stop 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 shutdown_connections 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fea280b2910 0x7fea280b4e20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea3c1089f0 0x7fea3c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea3c1029c0 0x7fea3c19adb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 --2- 172.21.15.5:0/3237766512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea3c103310 0x7fea3c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 >> 172.21.15.5:0/3237766512 conn(0x7fea3c0fe520 msgr2=0x7fea3c106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 shutdown_connections 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:36.734+0000 7fea42e94700 1 -- 172.21.15.5:0/3237766512 wait complete. 2024-05-14T15:23:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:23:36.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:36 smithi005 ceph-mon[31559]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:37.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:36 smithi144 ceph-mon[35650]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:37.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:36 smithi193 ceph-mon[36001]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:37.484 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-05-14T15:23:35.507849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657260,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":672,"num_read_kb":437,"num_write":293,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377924,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563900,"statfs":{"total":1151940427776,"available":1151553433600,"internally_reserved":0,"allocated":12357632,"data_stored":8166554,"data_compressed":496698,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003635"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893561+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:22:42.893561+0000","last_peered":"2024-05-14T15:22:42.893561+0000","last_clean":"2024-05-14T15:22:42.893561+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:22:42.893561+0000","last_undegraded":"2024-05-14T15:22:42.893561+0000","last_fullsized":"2024-05-14T15:22:42.893561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229275+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:22:43.229275+0000","last_peered":"2024-05-14T15:22:43.229275+0000","last_clean":"2024-05-14T15:22:43.229275+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:22:43.229275+0000","last_undegraded":"2024-05-14T15:22:43.229275+0000","last_fullsized":"2024-05-14T15:22:43.229275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228275+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:22:43.228275+0000","last_peered":"2024-05-14T15:22:43.228275+0000","last_clean":"2024-05-14T15:22:43.228275+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:22:43.228275+0000","last_undegraded":"2024-05-14T15:22:43.228275+0000","last_fullsized":"2024-05-14T15:22:43.228275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229623+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:22:43.229623+0000","last_peered":"2024-05-14T15:22:43.229623+0000","last_clean":"2024-05-14T15:22:43.229623+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:22:43.229623+0000","last_undegraded":"2024-05-14T15:22:43.229623+0000","last_fullsized":"2024-05-14T15:22:43.229623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893476+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:22:42.893476+0000","last_peered":"2024-05-14T15:22:42.893476+0000","last_clean":"2024-05-14T15:22:42.893476+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:22:42.893476+0000","last_undegraded":"2024-05-14T15:22:42.893476+0000","last_fullsized":"2024-05-14T15:22:42.893476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892798+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:22:42.892798+0000","last_peered":"2024-05-14T15:22:42.892798+0000","last_clean":"2024-05-14T15:22:42.892798+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:22:42.892798+0000","last_undegraded":"2024-05-14T15:22:42.892798+0000","last_fullsized":"2024-05-14T15:22:42.892798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890039+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:22:42.890039+0000","last_peered":"2024-05-14T15:22:42.890039+0000","last_clean":"2024-05-14T15:22:42.890039+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:22:42.890039+0000","last_undegraded":"2024-05-14T15:22:42.890039+0000","last_fullsized":"2024-05-14T15:22:42.890039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892127+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:22:42.892127+0000","last_peered":"2024-05-14T15:22:42.892127+0000","last_clean":"2024-05-14T15:22:42.892127+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:22:42.892127+0000","last_undegraded":"2024-05-14T15:22:42.892127+0000","last_fullsized":"2024-05-14T15:22:42.892127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896215+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:22:42.896215+0000","last_peered":"2024-05-14T15:22:42.896215+0000","last_clean":"2024-05-14T15:22:42.896215+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:22:42.896215+0000","last_undegraded":"2024-05-14T15:22:42.896215+0000","last_fullsized":"2024-05-14T15:22:42.896215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892800+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:22:42.892800+0000","last_peered":"2024-05-14T15:22:42.892800+0000","last_clean":"2024-05-14T15:22:42.892800+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:22:42.892800+0000","last_undegraded":"2024-05-14T15:22:42.892800+0000","last_fullsized":"2024-05-14T15:22:42.892800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891307+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:22:42.891307+0000","last_peered":"2024-05-14T15:22:42.891307+0000","last_clean":"2024-05-14T15:22:42.891307+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:22:42.891307+0000","last_undegraded":"2024-05-14T15:22:42.891307+0000","last_fullsized":"2024-05-14T15:22:42.891307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229522+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:22:43.229522+0000","last_peered":"2024-05-14T15:22:43.229522+0000","last_clean":"2024-05-14T15:22:43.229522+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:22:43.229522+0000","last_undegraded":"2024-05-14T15:22:43.229522+0000","last_fullsized":"2024-05-14T15:22:43.229522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889246+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:22:42.889246+0000","last_peered":"2024-05-14T15:22:42.889246+0000","last_clean":"2024-05-14T15:22:42.889246+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:22:42.889246+0000","last_undegraded":"2024-05-14T15:22:42.889246+0000","last_fullsized":"2024-05-14T15:22:42.889246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.227979+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:22:43.227979+0000","last_peered":"2024-05-14T15:22:43.227979+0000","last_clean":"2024-05-14T15:22:43.227979+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:22:43.227979+0000","last_undegraded":"2024-05-14T15:22:43.227979+0000","last_fullsized":"2024-05-14T15:22:43.227979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228055+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:22:43.228055+0000","last_peered":"2024-05-14T15:22:43.228055+0000","last_clean":"2024-05-14T15:22:43.228055+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:22:43.228055+0000","last_undegraded":"2024-05-14T15:22:43.228055+0000","last_fullsized":"2024-05-14T15:22:43.228055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895788+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:22:42.895788+0000","last_peered":"2024-05-14T15:22:42.895788+0000","last_clean":"2024-05-14T15:22:42.895788+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:22:42.895788+0000","last_undegraded":"2024-05-14T15:22:42.895788+0000","last_fullsized":"2024-05-14T15:22:42.895788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893247+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:22:42.893247+0000","last_peered":"2024-05-14T15:22:42.893247+0000","last_clean":"2024-05-14T15:22:42.893247+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:22:42.893247+0000","last_undegraded":"2024-05-14T15:22:42.893247+0000","last_fullsized":"2024-05-14T15:22:42.893247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229325+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:22:43.229325+0000","last_peered":"2024-05-14T15:22:43.229325+0000","last_clean":"2024-05-14T15:22:43.229325+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:22:43.229325+0000","last_undegraded":"2024-05-14T15:22:43.229325+0000","last_fullsized":"2024-05-14T15:22:43.229325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895766+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:22:42.895766+0000","last_peered":"2024-05-14T15:22:42.895766+0000","last_clean":"2024-05-14T15:22:42.895766+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:22:42.895766+0000","last_undegraded":"2024-05-14T15:22:42.895766+0000","last_fullsized":"2024-05-14T15:22:42.895766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895668+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:22:42.895668+0000","last_peered":"2024-05-14T15:22:42.895668+0000","last_clean":"2024-05-14T15:22:42.895668+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:22:42.895668+0000","last_undegraded":"2024-05-14T15:22:42.895668+0000","last_fullsized":"2024-05-14T15:22:42.895668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228749+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:22:43.228749+0000","last_peered":"2024-05-14T15:22:43.228749+0000","last_clean":"2024-05-14T15:22:43.228749+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:22:43.228749+0000","last_undegraded":"2024-05-14T15:22:43.228749+0000","last_fullsized":"2024-05-14T15:22:43.228749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892480+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:22:42.892480+0000","last_peered":"2024-05-14T15:22:42.892480+0000","last_clean":"2024-05-14T15:22:42.892480+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:22:42.892480+0000","last_undegraded":"2024-05-14T15:22:42.892480+0000","last_fullsized":"2024-05-14T15:22:42.892480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228543+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:22:43.228543+0000","last_peered":"2024-05-14T15:22:43.228543+0000","last_clean":"2024-05-14T15:22:43.228543+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:22:43.228543+0000","last_undegraded":"2024-05-14T15:22:43.228543+0000","last_fullsized":"2024-05-14T15:22:43.228543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892694+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:22:42.892694+0000","last_peered":"2024-05-14T15:22:42.892694+0000","last_clean":"2024-05-14T15:22:42.892694+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:22:42.892694+0000","last_undegraded":"2024-05-14T15:22:42.892694+0000","last_fullsized":"2024-05-14T15:22:42.892694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228669+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:22:43.228669+0000","last_peered":"2024-05-14T15:22:43.228669+0000","last_clean":"2024-05-14T15:22:43.228669+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:22:43.228669+0000","last_undegraded":"2024-05-14T15:22:43.228669+0000","last_fullsized":"2024-05-14T15:22:43.228669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228770+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:22:43.228770+0000","last_peered":"2024-05-14T15:22:43.228770+0000","last_clean":"2024-05-14T15:22:43.228770+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:22:43.228770+0000","last_undegraded":"2024-05-14T15:22:43.228770+0000","last_fullsized":"2024-05-14T15:22:43.228770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895009+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:22:42.895009+0000","last_peered":"2024-05-14T15:22:42.895009+0000","last_clean":"2024-05-14T15:22:42.895009+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:22:42.895009+0000","last_undegraded":"2024-05-14T15:22:42.895009+0000","last_fullsized":"2024-05-14T15:22:42.895009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895721+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:22:42.895721+0000","last_peered":"2024-05-14T15:22:42.895721+0000","last_clean":"2024-05-14T15:22:42.895721+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:22:42.895721+0000","last_undegraded":"2024-05-14T15:22:42.895721+0000","last_fullsized":"2024-05-14T15:22:42.895721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228753+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:22:43.228753+0000","last_peered":"2024-05-14T15:22:43.228753+0000","last_clean":"2024-05-14T15:22:43.228753+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:22:43.228753+0000","last_undegraded":"2024-05-14T15:22:43.228753+0000","last_fullsized":"2024-05-14T15:22:43.228753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894409+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:22:42.894409+0000","last_peered":"2024-05-14T15:22:42.894409+0000","last_clean":"2024-05-14T15:22:42.894409+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:22:42.894409+0000","last_undegraded":"2024-05-14T15:22:42.894409+0000","last_fullsized":"2024-05-14T15:22:42.894409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891060+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:22:42.891060+0000","last_peered":"2024-05-14T15:22:42.891060+0000","last_clean":"2024-05-14T15:22:42.891060+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:22:42.891060+0000","last_undegraded":"2024-05-14T15:22:42.891060+0000","last_fullsized":"2024-05-14T15:22:42.891060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895391+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:22:42.895391+0000","last_peered":"2024-05-14T15:22:42.895391+0000","last_clean":"2024-05-14T15:22:42.895391+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:22:42.895391+0000","last_undegraded":"2024-05-14T15:22:42.895391+0000","last_fullsized":"2024-05-14T15:22:42.895391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:28.918717+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:23:28.918717+0000","last_peered":"2024-05-14T15:23:28.918717+0000","last_clean":"2024-05-14T15:23:28.918717+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:23:28.918717+0000","last_undegraded":"2024-05-14T15:23:28.918717+0000","last_fullsized":"2024-05-14T15:23:28.918717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893810+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:22:42.893810+0000","last_peered":"2024-05-14T15:22:42.893810+0000","last_clean":"2024-05-14T15:22:42.893810+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:22:42.893810+0000","last_undegraded":"2024-05-14T15:22:42.893810+0000","last_fullsized":"2024-05-14T15:22:42.893810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895150+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:22:42.895150+0000","last_peered":"2024-05-14T15:22:42.895150+0000","last_clean":"2024-05-14T15:22:42.895150+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:22:42.895150+0000","last_undegraded":"2024-05-14T15:22:42.895150+0000","last_fullsized":"2024-05-14T15:22:42.895150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.233584+0000","last_change":"2024-05-14T15:22:43.233044+0000","last_active":"2024-05-14T15:22:43.233584+0000","last_peered":"2024-05-14T15:22:43.233584+0000","last_clean":"2024-05-14T15:22:43.233584+0000","last_became_active":"2024-05-14T15:22:43.229808+0000","last_became_peered":"2024-05-14T15:22:43.229808+0000","last_unstale":"2024-05-14T15:22:43.233584+0000","last_undegraded":"2024-05-14T15:22:43.233584+0000","last_fullsized":"2024-05-14T15:22:43.233584+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:07.226460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890849+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:22:42.890849+0000","last_peered":"2024-05-14T15:22:42.890849+0000","last_clean":"2024-05-14T15:22:42.890849+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:22:42.890849+0000","last_undegraded":"2024-05-14T15:22:42.890849+0000","last_fullsized":"2024-05-14T15:22:42.890849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892474+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:22:42.892474+0000","last_peered":"2024-05-14T15:22:42.892474+0000","last_clean":"2024-05-14T15:22:42.892474+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:22:42.892474+0000","last_undegraded":"2024-05-14T15:22:42.892474+0000","last_fullsized":"2024-05-14T15:22:42.892474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889261+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:22:42.889261+0000","last_peered":"2024-05-14T15:22:42.889261+0000","last_clean":"2024-05-14T15:22:42.889261+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:22:42.889261+0000","last_undegraded":"2024-05-14T15:22:42.889261+0000","last_fullsized":"2024-05-14T15:22:42.889261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892324+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:22:42.892324+0000","last_peered":"2024-05-14T15:22:42.892324+0000","last_clean":"2024-05-14T15:22:42.892324+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:22:42.892324+0000","last_undegraded":"2024-05-14T15:22:42.892324+0000","last_fullsized":"2024-05-14T15:22:42.892324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229134+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:22:43.229134+0000","last_peered":"2024-05-14T15:22:43.229134+0000","last_clean":"2024-05-14T15:22:43.229134+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:22:43.229134+0000","last_undegraded":"2024-05-14T15:22:43.229134+0000","last_fullsized":"2024-05-14T15:22:43.229134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.900544+0000","last_change":"2024-05-14T15:22:42.900544+0000","last_active":"2024-05-14T15:22:42.900544+0000","last_peered":"2024-05-14T15:22:42.900544+0000","last_clean":"2024-05-14T15:22:42.900544+0000","last_became_active":"2024-05-14T15:22:42.900015+0000","last_became_peered":"2024-05-14T15:22:42.900015+0000","last_unstale":"2024-05-14T15:22:42.900544+0000","last_undegraded":"2024-05-14T15:22:42.900544+0000","last_fullsized":"2024-05-14T15:22:42.900544+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:15.904051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228181+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:22:43.228181+0000","last_peered":"2024-05-14T15:22:43.228181+0000","last_clean":"2024-05-14T15:22:43.228181+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:22:43.228181+0000","last_undegraded":"2024-05-14T15:22:43.228181+0000","last_fullsized":"2024-05-14T15:22:43.228181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892276+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:22:42.892276+0000","last_peered":"2024-05-14T15:22:42.892276+0000","last_clean":"2024-05-14T15:22:42.892276+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:22:42.892276+0000","last_undegraded":"2024-05-14T15:22:42.892276+0000","last_fullsized":"2024-05-14T15:22:42.892276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":141,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:28.917997+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:23:28.917997+0000","last_peered":"2024-05-14T15:23:28.917997+0000","last_clean":"2024-05-14T15:23:28.917997+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:23:28.917997+0000","last_undegraded":"2024-05-14T15:23:28.917997+0000","last_fullsized":"2024-05-14T15:23:28.917997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896216+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:22:42.896216+0000","last_peered":"2024-05-14T15:22:42.896216+0000","last_clean":"2024-05-14T15:22:42.896216+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:22:42.896216+0000","last_undegraded":"2024-05-14T15:22:42.896216+0000","last_fullsized":"2024-05-14T15:22:42.896216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.227944+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:22:43.227944+0000","last_peered":"2024-05-14T15:22:43.227944+0000","last_clean":"2024-05-14T15:22:43.227944+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:22:43.227944+0000","last_undegraded":"2024-05-14T15:22:43.227944+0000","last_fullsized":"2024-05-14T15:22:43.227944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891487+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:22:42.891487+0000","last_peered":"2024-05-14T15:22:42.891487+0000","last_clean":"2024-05-14T15:22:42.891487+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:22:42.891487+0000","last_undegraded":"2024-05-14T15:22:42.891487+0000","last_fullsized":"2024-05-14T15:22:42.891487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228713+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:22:43.228713+0000","last_peered":"2024-05-14T15:22:43.228713+0000","last_clean":"2024-05-14T15:22:43.228713+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:22:43.228713+0000","last_undegraded":"2024-05-14T15:22:43.228713+0000","last_fullsized":"2024-05-14T15:22:43.228713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228987+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:22:43.228987+0000","last_peered":"2024-05-14T15:22:43.228987+0000","last_clean":"2024-05-14T15:22:43.228987+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:22:43.228987+0000","last_undegraded":"2024-05-14T15:22:43.228987+0000","last_fullsized":"2024-05-14T15:22:43.228987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890800+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:22:42.890800+0000","last_peered":"2024-05-14T15:22:42.890800+0000","last_clean":"2024-05-14T15:22:42.890800+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:22:42.890800+0000","last_undegraded":"2024-05-14T15:22:42.890800+0000","last_fullsized":"2024-05-14T15:22:42.890800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895782+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:22:42.895782+0000","last_peered":"2024-05-14T15:22:42.895782+0000","last_clean":"2024-05-14T15:22:42.895782+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:22:42.895782+0000","last_undegraded":"2024-05-14T15:22:42.895782+0000","last_fullsized":"2024-05-14T15:22:42.895782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228275+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:22:43.228275+0000","last_peered":"2024-05-14T15:22:43.228275+0000","last_clean":"2024-05-14T15:22:43.228275+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:22:43.228275+0000","last_undegraded":"2024-05-14T15:22:43.228275+0000","last_fullsized":"2024-05-14T15:22:43.228275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":272,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:29.919444+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:23:29.919444+0000","last_peered":"2024-05-14T15:23:29.919444+0000","last_clean":"2024-05-14T15:23:29.919444+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:23:29.919444+0000","last_undegraded":"2024-05-14T15:23:29.919444+0000","last_fullsized":"2024-05-14T15:23:29.919444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":195,"num_read_kb":159,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894843+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:22:42.894843+0000","last_peered":"2024-05-14T15:22:42.894843+0000","last_clean":"2024-05-14T15:22:42.894843+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:22:42.894843+0000","last_undegraded":"2024-05-14T15:22:42.894843+0000","last_fullsized":"2024-05-14T15:22:42.894843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891424+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:22:42.891424+0000","last_peered":"2024-05-14T15:22:42.891424+0000","last_clean":"2024-05-14T15:22:42.891424+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:22:42.891424+0000","last_undegraded":"2024-05-14T15:22:42.891424+0000","last_fullsized":"2024-05-14T15:22:42.891424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896121+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:22:42.896121+0000","last_peered":"2024-05-14T15:22:42.896121+0000","last_clean":"2024-05-14T15:22:42.896121+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:22:42.896121+0000","last_undegraded":"2024-05-14T15:22:42.896121+0000","last_fullsized":"2024-05-14T15:22:42.896121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889457+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:22:42.889457+0000","last_peered":"2024-05-14T15:22:42.889457+0000","last_clean":"2024-05-14T15:22:42.889457+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:22:42.889457+0000","last_undegraded":"2024-05-14T15:22:42.889457+0000","last_fullsized":"2024-05-14T15:22:42.889457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892668+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:22:42.892668+0000","last_peered":"2024-05-14T15:22:42.892668+0000","last_clean":"2024-05-14T15:22:42.892668+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:22:42.892668+0000","last_undegraded":"2024-05-14T15:22:42.892668+0000","last_fullsized":"2024-05-14T15:22:42.892668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890575+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:22:42.890575+0000","last_peered":"2024-05-14T15:22:42.890575+0000","last_clean":"2024-05-14T15:22:42.890575+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:22:42.890575+0000","last_undegraded":"2024-05-14T15:22:42.890575+0000","last_fullsized":"2024-05-14T15:22:42.890575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890147+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:22:42.890147+0000","last_peered":"2024-05-14T15:22:42.890147+0000","last_clean":"2024-05-14T15:22:42.890147+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:22:42.890147+0000","last_undegraded":"2024-05-14T15:22:42.890147+0000","last_fullsized":"2024-05-14T15:22:42.890147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228449+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:22:43.228449+0000","last_peered":"2024-05-14T15:22:43.228449+0000","last_clean":"2024-05-14T15:22:43.228449+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:22:43.228449+0000","last_undegraded":"2024-05-14T15:22:43.228449+0000","last_fullsized":"2024-05-14T15:22:43.228449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:29.923229+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:23:29.923229+0000","last_peered":"2024-05-14T15:23:29.923229+0000","last_clean":"2024-05-14T15:23:29.923229+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:23:29.923229+0000","last_undegraded":"2024-05-14T15:23:29.923229+0000","last_fullsized":"2024-05-14T15:23:29.923229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890455+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:22:42.890455+0000","last_peered":"2024-05-14T15:22:42.890455+0000","last_clean":"2024-05-14T15:22:42.890455+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:22:42.890455+0000","last_undegraded":"2024-05-14T15:22:42.890455+0000","last_fullsized":"2024-05-14T15:22:42.890455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889844+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:22:42.889844+0000","last_peered":"2024-05-14T15:22:42.889844+0000","last_clean":"2024-05-14T15:22:42.889844+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:22:42.889844+0000","last_undegraded":"2024-05-14T15:22:42.889844+0000","last_fullsized":"2024-05-14T15:22:42.889844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893796+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:22:42.893796+0000","last_peered":"2024-05-14T15:22:42.893796+0000","last_clean":"2024-05-14T15:22:42.893796+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:22:42.893796+0000","last_undegraded":"2024-05-14T15:22:42.893796+0000","last_fullsized":"2024-05-14T15:22:42.893796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890230+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:22:42.890230+0000","last_peered":"2024-05-14T15:22:42.890230+0000","last_clean":"2024-05-14T15:22:42.890230+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:22:42.890230+0000","last_undegraded":"2024-05-14T15:22:42.890230+0000","last_fullsized":"2024-05-14T15:22:42.890230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890053+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:22:42.890053+0000","last_peered":"2024-05-14T15:22:42.890053+0000","last_clean":"2024-05-14T15:22:42.890053+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:22:42.890053+0000","last_undegraded":"2024-05-14T15:22:42.890053+0000","last_fullsized":"2024-05-14T15:22:42.890053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894730+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:22:42.894730+0000","last_peered":"2024-05-14T15:22:42.894730+0000","last_clean":"2024-05-14T15:22:42.894730+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:22:42.894730+0000","last_undegraded":"2024-05-14T15:22:42.894730+0000","last_fullsized":"2024-05-14T15:22:42.894730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892793+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:22:42.892793+0000","last_peered":"2024-05-14T15:22:42.892793+0000","last_clean":"2024-05-14T15:22:42.892793+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:22:42.892793+0000","last_undegraded":"2024-05-14T15:22:42.892793+0000","last_fullsized":"2024-05-14T15:22:42.892793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893702+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:22:42.893702+0000","last_peered":"2024-05-14T15:22:42.893702+0000","last_clean":"2024-05-14T15:22:42.893702+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:22:42.893702+0000","last_undegraded":"2024-05-14T15:22:42.893702+0000","last_fullsized":"2024-05-14T15:22:42.893702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228983+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:22:43.228983+0000","last_peered":"2024-05-14T15:22:43.228983+0000","last_clean":"2024-05-14T15:22:43.228983+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:22:43.228983+0000","last_undegraded":"2024-05-14T15:22:43.228983+0000","last_fullsized":"2024-05-14T15:22:43.228983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892043+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:22:42.892043+0000","last_peered":"2024-05-14T15:22:42.892043+0000","last_clean":"2024-05-14T15:22:42.892043+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:22:42.892043+0000","last_undegraded":"2024-05-14T15:22:42.892043+0000","last_fullsized":"2024-05-14T15:22:42.892043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889458+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:22:42.889458+0000","last_peered":"2024-05-14T15:22:42.889458+0000","last_clean":"2024-05-14T15:22:42.889458+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:22:42.889458+0000","last_undegraded":"2024-05-14T15:22:42.889458+0000","last_fullsized":"2024-05-14T15:22:42.889458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890147+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:22:42.890147+0000","last_peered":"2024-05-14T15:22:42.890147+0000","last_clean":"2024-05-14T15:22:42.890147+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:22:42.890147+0000","last_undegraded":"2024-05-14T15:22:42.890147+0000","last_fullsized":"2024-05-14T15:22:42.890147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892446+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:22:42.892446+0000","last_peered":"2024-05-14T15:22:42.892446+0000","last_clean":"2024-05-14T15:22:42.892446+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:22:42.892446+0000","last_undegraded":"2024-05-14T15:22:42.892446+0000","last_fullsized":"2024-05-14T15:22:42.892446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889900+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:22:42.889900+0000","last_peered":"2024-05-14T15:22:42.889900+0000","last_clean":"2024-05-14T15:22:42.889900+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:22:42.889900+0000","last_undegraded":"2024-05-14T15:22:42.889900+0000","last_fullsized":"2024-05-14T15:22:42.889900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890875+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:22:42.890875+0000","last_peered":"2024-05-14T15:22:42.890875+0000","last_clean":"2024-05-14T15:22:42.890875+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:22:42.890875+0000","last_undegraded":"2024-05-14T15:22:42.890875+0000","last_fullsized":"2024-05-14T15:22:42.890875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:32.926980+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:23:32.926980+0000","last_peered":"2024-05-14T15:23:32.926980+0000","last_clean":"2024-05-14T15:23:32.926980+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:23:32.926980+0000","last_undegraded":"2024-05-14T15:23:32.926980+0000","last_fullsized":"2024-05-14T15:23:32.926980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891147+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:22:42.891147+0000","last_peered":"2024-05-14T15:22:42.891147+0000","last_clean":"2024-05-14T15:22:42.891147+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:22:42.891147+0000","last_undegraded":"2024-05-14T15:22:42.891147+0000","last_fullsized":"2024-05-14T15:22:42.891147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892930+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:22:42.892930+0000","last_peered":"2024-05-14T15:22:42.892930+0000","last_clean":"2024-05-14T15:22:42.892930+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:22:42.892930+0000","last_undegraded":"2024-05-14T15:22:42.892930+0000","last_fullsized":"2024-05-14T15:22:42.892930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228957+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:22:43.228957+0000","last_peered":"2024-05-14T15:22:43.228957+0000","last_clean":"2024-05-14T15:22:43.228957+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:22:43.228957+0000","last_undegraded":"2024-05-14T15:22:43.228957+0000","last_fullsized":"2024-05-14T15:22:43.228957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889825+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:22:42.889825+0000","last_peered":"2024-05-14T15:22:42.889825+0000","last_clean":"2024-05-14T15:22:42.889825+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:22:42.889825+0000","last_undegraded":"2024-05-14T15:22:42.889825+0000","last_fullsized":"2024-05-14T15:22:42.889825+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891290+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:22:42.891290+0000","last_peered":"2024-05-14T15:22:42.891290+0000","last_clean":"2024-05-14T15:22:42.891290+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:22:42.891290+0000","last_undegraded":"2024-05-14T15:22:42.891290+0000","last_fullsized":"2024-05-14T15:22:42.891290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889043+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:22:42.889043+0000","last_peered":"2024-05-14T15:22:42.889043+0000","last_clean":"2024-05-14T15:22:42.889043+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:22:42.889043+0000","last_undegraded":"2024-05-14T15:22:42.889043+0000","last_fullsized":"2024-05-14T15:22:42.889043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.903832+0000","last_change":"2024-05-14T15:22:42.903272+0000","last_active":"2024-05-14T15:22:42.903832+0000","last_peered":"2024-05-14T15:22:42.903832+0000","last_clean":"2024-05-14T15:22:42.903832+0000","last_became_active":"2024-05-14T15:22:42.900088+0000","last_became_peered":"2024-05-14T15:22:42.900088+0000","last_unstale":"2024-05-14T15:22:42.903832+0000","last_undegraded":"2024-05-14T15:22:42.903832+0000","last_fullsized":"2024-05-14T15:22:42.903832+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:23.975904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895414+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:22:42.895414+0000","last_peered":"2024-05-14T15:22:42.895414+0000","last_clean":"2024-05-14T15:22:42.895414+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:22:42.895414+0000","last_undegraded":"2024-05-14T15:22:42.895414+0000","last_fullsized":"2024-05-14T15:22:42.895414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894757+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:22:42.894757+0000","last_peered":"2024-05-14T15:22:42.894757+0000","last_clean":"2024-05-14T15:22:42.894757+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:22:42.894757+0000","last_undegraded":"2024-05-14T15:22:42.894757+0000","last_fullsized":"2024-05-14T15:22:42.894757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889686+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:22:42.889686+0000","last_peered":"2024-05-14T15:22:42.889686+0000","last_clean":"2024-05-14T15:22:42.889686+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:22:42.889686+0000","last_undegraded":"2024-05-14T15:22:42.889686+0000","last_fullsized":"2024-05-14T15:22:42.889686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892174+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:22:42.892174+0000","last_peered":"2024-05-14T15:22:42.892174+0000","last_clean":"2024-05-14T15:22:42.892174+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:22:42.892174+0000","last_undegraded":"2024-05-14T15:22:42.892174+0000","last_fullsized":"2024-05-14T15:22:42.892174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229405+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:22:43.229405+0000","last_peered":"2024-05-14T15:22:43.229405+0000","last_clean":"2024-05-14T15:22:43.229405+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:22:43.229405+0000","last_undegraded":"2024-05-14T15:22:43.229405+0000","last_fullsized":"2024-05-14T15:22:43.229405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892912+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:22:42.892912+0000","last_peered":"2024-05-14T15:22:42.892912+0000","last_clean":"2024-05-14T15:22:42.892912+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:22:42.892912+0000","last_undegraded":"2024-05-14T15:22:42.892912+0000","last_fullsized":"2024-05-14T15:22:42.892912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895293+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:22:42.895293+0000","last_peered":"2024-05-14T15:22:42.895293+0000","last_clean":"2024-05-14T15:22:42.895293+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:22:42.895293+0000","last_undegraded":"2024-05-14T15:22:42.895293+0000","last_fullsized":"2024-05-14T15:22:42.895293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229439+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:22:43.229439+0000","last_peered":"2024-05-14T15:22:43.229439+0000","last_clean":"2024-05-14T15:22:43.229439+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:22:43.229439+0000","last_undegraded":"2024-05-14T15:22:43.229439+0000","last_fullsized":"2024-05-14T15:22:43.229439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229283+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:22:43.229283+0000","last_peered":"2024-05-14T15:22:43.229283+0000","last_clean":"2024-05-14T15:22:43.229283+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:22:43.229283+0000","last_undegraded":"2024-05-14T15:22:43.229283+0000","last_fullsized":"2024-05-14T15:22:43.229283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229394+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:22:43.229394+0000","last_peered":"2024-05-14T15:22:43.229394+0000","last_clean":"2024-05-14T15:22:43.229394+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:22:43.229394+0000","last_undegraded":"2024-05-14T15:22:43.229394+0000","last_fullsized":"2024-05-14T15:22:43.229394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895849+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:22:42.895849+0000","last_peered":"2024-05-14T15:22:42.895849+0000","last_clean":"2024-05-14T15:22:42.895849+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:22:42.895849+0000","last_undegraded":"2024-05-14T15:22:42.895849+0000","last_fullsized":"2024-05-14T15:22:42.895849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894144+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:22:42.894144+0000","last_peered":"2024-05-14T15:22:42.894144+0000","last_clean":"2024-05-14T15:22:42.894144+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:22:42.894144+0000","last_undegraded":"2024-05-14T15:22:42.894144+0000","last_fullsized":"2024-05-14T15:22:42.894144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228624+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:22:43.228624+0000","last_peered":"2024-05-14T15:22:43.228624+0000","last_clean":"2024-05-14T15:22:43.228624+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:22:43.228624+0000","last_undegraded":"2024-05-14T15:22:43.228624+0000","last_fullsized":"2024-05-14T15:22:43.228624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892417+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:22:42.892417+0000","last_peered":"2024-05-14T15:22:42.892417+0000","last_clean":"2024-05-14T15:22:42.892417+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:22:42.892417+0000","last_undegraded":"2024-05-14T15:22:42.892417+0000","last_fullsized":"2024-05-14T15:22:42.892417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893361+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:22:42.893361+0000","last_peered":"2024-05-14T15:22:42.893361+0000","last_clean":"2024-05-14T15:22:42.893361+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:22:42.893361+0000","last_undegraded":"2024-05-14T15:22:42.893361+0000","last_fullsized":"2024-05-14T15:22:42.893361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229122+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:22:43.229122+0000","last_peered":"2024-05-14T15:22:43.229122+0000","last_clean":"2024-05-14T15:22:43.229122+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:22:43.229122+0000","last_undegraded":"2024-05-14T15:22:43.229122+0000","last_fullsized":"2024-05-14T15:22:43.229122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895168+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:22:42.895168+0000","last_peered":"2024-05-14T15:22:42.895168+0000","last_clean":"2024-05-14T15:22:42.895168+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:22:42.895168+0000","last_undegraded":"2024-05-14T15:22:42.895168+0000","last_fullsized":"2024-05-14T15:22:42.895168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892727+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:22:42.892727+0000","last_peered":"2024-05-14T15:22:42.892727+0000","last_clean":"2024-05-14T15:22:42.892727+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:22:42.892727+0000","last_undegraded":"2024-05-14T15:22:42.892727+0000","last_fullsized":"2024-05-14T15:22:42.892727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229462+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:22:43.229462+0000","last_peered":"2024-05-14T15:22:43.229462+0000","last_clean":"2024-05-14T15:22:43.229462+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:22:43.229462+0000","last_undegraded":"2024-05-14T15:22:43.229462+0000","last_fullsized":"2024-05-14T15:22:43.229462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.902298+0000","last_change":"2024-05-14T15:22:42.901814+0000","last_active":"2024-05-14T15:22:42.902298+0000","last_peered":"2024-05-14T15:22:42.902298+0000","last_clean":"2024-05-14T15:22:42.902298+0000","last_became_active":"2024-05-14T15:22:42.896795+0000","last_became_peered":"2024-05-14T15:22:42.896795+0000","last_unstale":"2024-05-14T15:22:42.902298+0000","last_undegraded":"2024-05-14T15:22:42.902298+0000","last_fullsized":"2024-05-14T15:22:42.902298+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:15.981532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890195+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:22:42.890195+0000","last_peered":"2024-05-14T15:22:42.890195+0000","last_clean":"2024-05-14T15:22:42.890195+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:22:42.890195+0000","last_undegraded":"2024-05-14T15:22:42.890195+0000","last_fullsized":"2024-05-14T15:22:42.890195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895408+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:22:42.895408+0000","last_peered":"2024-05-14T15:22:42.895408+0000","last_clean":"2024-05-14T15:22:42.895408+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:22:42.895408+0000","last_undegraded":"2024-05-14T15:22:42.895408+0000","last_fullsized":"2024-05-14T15:22:42.895408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895524+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:22:42.895524+0000","last_peered":"2024-05-14T15:22:42.895524+0000","last_clean":"2024-05-14T15:22:42.895524+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:22:42.895524+0000","last_undegraded":"2024-05-14T15:22:42.895524+0000","last_fullsized":"2024-05-14T15:22:42.895524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892220+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:22:42.892220+0000","last_peered":"2024-05-14T15:22:42.892220+0000","last_clean":"2024-05-14T15:22:42.892220+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:22:42.892220+0000","last_undegraded":"2024-05-14T15:22:42.892220+0000","last_fullsized":"2024-05-14T15:22:42.892220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228424+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:22:43.228424+0000","last_peered":"2024-05-14T15:22:43.228424+0000","last_clean":"2024-05-14T15:22:43.228424+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:22:43.228424+0000","last_undegraded":"2024-05-14T15:22:43.228424+0000","last_fullsized":"2024-05-14T15:22:43.228424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891192+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:22:42.891192+0000","last_peered":"2024-05-14T15:22:42.891192+0000","last_clean":"2024-05-14T15:22:42.891192+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:22:42.891192+0000","last_undegraded":"2024-05-14T15:22:42.891192+0000","last_fullsized":"2024-05-14T15:22:42.891192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891158+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:22:42.891158+0000","last_peered":"2024-05-14T15:22:42.891158+0000","last_clean":"2024-05-14T15:22:42.891158+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:22:42.891158+0000","last_undegraded":"2024-05-14T15:22:42.891158+0000","last_fullsized":"2024-05-14T15:22:42.891158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889807+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:22:42.889807+0000","last_peered":"2024-05-14T15:22:42.889807+0000","last_clean":"2024-05-14T15:22:42.889807+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:22:42.889807+0000","last_undegraded":"2024-05-14T15:22:42.889807+0000","last_fullsized":"2024-05-14T15:22:42.889807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.236629+0000","last_change":"2024-05-14T15:22:43.235879+0000","last_active":"2024-05-14T15:22:43.236629+0000","last_peered":"2024-05-14T15:22:43.236629+0000","last_clean":"2024-05-14T15:22:43.236629+0000","last_became_active":"2024-05-14T15:22:43.230116+0000","last_became_peered":"2024-05-14T15:22:43.230116+0000","last_unstale":"2024-05-14T15:22:43.236629+0000","last_undegraded":"2024-05-14T15:22:43.236629+0000","last_fullsized":"2024-05-14T15:22:43.236629+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:25:50.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891625+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:22:42.891625+0000","last_peered":"2024-05-14T15:22:42.891625+0000","last_clean":"2024-05-14T15:22:42.891625+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:22:42.891625+0000","last_undegraded":"2024-05-14T15:22:42.891625+0000","last_fullsized":"2024-05-14T15:22:42.891625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893515+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:22:42.893515+0000","last_peered":"2024-05-14T15:22:42.893515+0000","last_clean":"2024-05-14T15:22:42.893515+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:22:42.893515+0000","last_undegraded":"2024-05-14T15:22:42.893515+0000","last_fullsized":"2024-05-14T15:22:42.893515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896064+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:22:42.896064+0000","last_peered":"2024-05-14T15:22:42.896064+0000","last_clean":"2024-05-14T15:22:42.896064+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:22:42.896064+0000","last_undegraded":"2024-05-14T15:22:42.896064+0000","last_fullsized":"2024-05-14T15:22:42.896064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895071+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:22:42.895071+0000","last_peered":"2024-05-14T15:22:42.895071+0000","last_clean":"2024-05-14T15:22:42.895071+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:22:42.895071+0000","last_undegraded":"2024-05-14T15:22:42.895071+0000","last_fullsized":"2024-05-14T15:22:42.895071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228542+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:22:43.228542+0000","last_peered":"2024-05-14T15:22:43.228542+0000","last_clean":"2024-05-14T15:22:43.228542+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:22:43.228542+0000","last_undegraded":"2024-05-14T15:22:43.228542+0000","last_fullsized":"2024-05-14T15:22:43.228542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889722+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:22:42.889722+0000","last_peered":"2024-05-14T15:22:42.889722+0000","last_clean":"2024-05-14T15:22:42.889722+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:22:42.889722+0000","last_undegraded":"2024-05-14T15:22:42.889722+0000","last_fullsized":"2024-05-14T15:22:42.889722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893428+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:22:42.893428+0000","last_peered":"2024-05-14T15:22:42.893428+0000","last_clean":"2024-05-14T15:22:42.893428+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:22:42.893428+0000","last_undegraded":"2024-05-14T15:22:42.893428+0000","last_fullsized":"2024-05-14T15:22:42.893428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896524+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:22:42.896524+0000","last_peered":"2024-05-14T15:22:42.896524+0000","last_clean":"2024-05-14T15:22:42.896524+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:22:42.896524+0000","last_undegraded":"2024-05-14T15:22:42.896524+0000","last_fullsized":"2024-05-14T15:22:42.896524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893046+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:22:42.893046+0000","last_peered":"2024-05-14T15:22:42.893046+0000","last_clean":"2024-05-14T15:22:42.893046+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:22:42.893046+0000","last_undegraded":"2024-05-14T15:22:42.893046+0000","last_fullsized":"2024-05-14T15:22:42.893046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889979+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:22:42.889979+0000","last_peered":"2024-05-14T15:22:42.889979+0000","last_clean":"2024-05-14T15:22:42.889979+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:22:42.889979+0000","last_undegraded":"2024-05-14T15:22:42.889979+0000","last_fullsized":"2024-05-14T15:22:42.889979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892587+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:22:42.892587+0000","last_peered":"2024-05-14T15:22:42.892587+0000","last_clean":"2024-05-14T15:22:42.892587+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:22:42.892587+0000","last_undegraded":"2024-05-14T15:22:42.892587+0000","last_fullsized":"2024-05-14T15:22:42.892587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895659+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:22:42.895659+0000","last_peered":"2024-05-14T15:22:42.895659+0000","last_clean":"2024-05-14T15:22:42.895659+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:22:42.895659+0000","last_undegraded":"2024-05-14T15:22:42.895659+0000","last_fullsized":"2024-05-14T15:22:42.895659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889808+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:22:42.889808+0000","last_peered":"2024-05-14T15:22:42.889808+0000","last_clean":"2024-05-14T15:22:42.889808+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:22:42.889808+0000","last_undegraded":"2024-05-14T15:22:42.889808+0000","last_fullsized":"2024-05-14T15:22:42.889808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893157+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:22:42.893157+0000","last_peered":"2024-05-14T15:22:42.893157+0000","last_clean":"2024-05-14T15:22:42.893157+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:22:42.893157+0000","last_undegraded":"2024-05-14T15:22:42.893157+0000","last_fullsized":"2024-05-14T15:22:42.893157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283950,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66658,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":591,"num_read_kb":368,"num_write":168,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678064,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":518237,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874292,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":1168162,"data_compressed":68468,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037818,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":522569,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":518393,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430277,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":1114302,"data_compressed":68468,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593801,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":513649,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":576216,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":515037,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":577957,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":513843,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953555,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":518299,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1109890,"data_compressed":68468,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:23:37.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T15:23:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:37.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:37 smithi005 ceph-mon[31559]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:23:38.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:37 smithi144 ceph-mon[35650]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:23:38.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:37 smithi193 ceph-mon[36001]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:23:38.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:38 smithi005 ceph-mon[31559]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:23:39.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:38 smithi144 ceph-mon[35650]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:23:39.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:38 smithi193 ceph-mon[36001]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.519+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/2910614380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff81034c0 msgr2=0x7f7ff81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.519+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/2910614380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff81034c0 0x7f7ff81038d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80099c0 tx=0x7f7fe8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/2910614380 shutdown_connections 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/2910614380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff8108740 0x7f7ff81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/2910614380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff81034c0 0x7f7ff81038d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/2910614380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff8107df0 0x7f7ff8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/2910614380 >> 172.21.15.5:0/2910614380 conn(0x7f7ff80fedf0 msgr2=0x7f7ff8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/2910614380 shutdown_connections 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.520+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/2910614380 wait complete. 2024-05-14T15:23:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 Processor -- start 2024-05-14T15:23:39.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 -- start start 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff81034c0 0x7f7ff8199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff8107df0 0x7f7ff819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 0x7f7ff819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff8113370 con 0x7f7ff8108740 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81131f0 con 0x7f7ff8107df0 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffe4ea700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81134f0 con 0x7f7ff81034c0 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 0x7f7ff819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 0x7f7ff819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48044/0 (socket says 172.21.15.5:48044) 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 -- 172.21.15.5:0/3976634935 learned_addr learned my addr 172.21.15.5:0/3976634935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7feffff700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff8107df0 0x7f7ff819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ff7fff700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff81034c0 0x7f7ff8199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 -- 172.21.15.5:0/3976634935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff81034c0 msgr2=0x7f7ff8199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff81034c0 0x7f7ff8199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 -- 172.21.15.5:0/3976634935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff8107df0 msgr2=0x7f7ff819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff8107df0 0x7f7ff819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.521+0000 7f7ffca87700 1 -- 172.21.15.5:0/3976634935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff819ec30 con 0x7f7ff8108740 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.522+0000 7f7ffca87700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 0x7f7ff819e590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000ada0 tx=0x7f7ff000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.522+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0017020 con 0x7f7ff8108740 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.522+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff819ef20 con 0x7f7ff8108740 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.522+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff000ce30 con 0x7f7ff8108740 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.522+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0005350 con 0x7f7ff8108740 2024-05-14T15:23:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.522+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff81a6840 con 0x7f7ff8108740 2024-05-14T15:23:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.523+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff80512c0 con 0x7f7ff8108740 2024-05-14T15:23:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.525+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7ff000c980 con 0x7f7ff8108740 2024-05-14T15:23:39.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.526+0000 7f7ff5ffb700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7fe40b2910 0x7f7fe40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:39.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.526+0000 7f7ff7fff700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7fe40b2910 0x7f7fe40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:39.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.526+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f7ff00e23e0 con 0x7f7ff8108740 2024-05-14T15:23:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.527+0000 7f7ff7fff700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7fe40b2910 0x7f7fe40b4e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8003800 tx=0x7f7fe801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.530+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ff0021020 con 0x7f7ff8108740 2024-05-14T15:23:39.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.740+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ff819b0c0 con 0x7f7fe40b2910 2024-05-14T15:23:39.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.750+0000 7f7ff5ffb700 1 -- 172.21.15.5:0/3976634935 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340624 (secure 0 0 0) 0x7f7ff819b0c0 con 0x7f7fe40b2910 2024-05-14T15:23:39.754 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:39.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7fe40b2910 msgr2=0x7f7fe40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7fe40b2910 0x7f7fe40b4e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8003800 tx=0x7f7fe801a040 comp rx=0 tx=0).stop 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 msgr2=0x7f7ff819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 0x7f7ff819e590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000ada0 tx=0x7f7ff000c330 comp rx=0 tx=0).stop 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 shutdown_connections 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f7fe40b2910 0x7f7fe40b4e20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8108740 0x7f7ff819e590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff81034c0 0x7f7ff8199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 --2- 172.21.15.5:0/3976634935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff8107df0 0x7f7ff819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.753+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 >> 172.21.15.5:0/3976634935 conn(0x7f7ff80fedf0 msgr2=0x7f7ff81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.754+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 shutdown_connections 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:39.754+0000 7f7ffe4ea700 1 -- 172.21.15.5:0/3976634935 wait complete. 2024-05-14T15:23:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:23:40.438 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-05-14T15:23:39.509084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657260,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":697,"num_read_kb":447,"num_write":293,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377924,"kb_used_data":12068,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563900,"statfs":{"total":1151940427776,"available":1151553433600,"internally_reserved":0,"allocated":12357632,"data_stored":8166554,"data_compressed":496698,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003788"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893561+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:22:42.893561+0000","last_peered":"2024-05-14T15:22:42.893561+0000","last_clean":"2024-05-14T15:22:42.893561+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:22:42.893561+0000","last_undegraded":"2024-05-14T15:22:42.893561+0000","last_fullsized":"2024-05-14T15:22:42.893561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229275+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:22:43.229275+0000","last_peered":"2024-05-14T15:22:43.229275+0000","last_clean":"2024-05-14T15:22:43.229275+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:22:43.229275+0000","last_undegraded":"2024-05-14T15:22:43.229275+0000","last_fullsized":"2024-05-14T15:22:43.229275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228275+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:22:43.228275+0000","last_peered":"2024-05-14T15:22:43.228275+0000","last_clean":"2024-05-14T15:22:43.228275+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:22:43.228275+0000","last_undegraded":"2024-05-14T15:22:43.228275+0000","last_fullsized":"2024-05-14T15:22:43.228275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229623+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:22:43.229623+0000","last_peered":"2024-05-14T15:22:43.229623+0000","last_clean":"2024-05-14T15:22:43.229623+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:22:43.229623+0000","last_undegraded":"2024-05-14T15:22:43.229623+0000","last_fullsized":"2024-05-14T15:22:43.229623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893476+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:22:42.893476+0000","last_peered":"2024-05-14T15:22:42.893476+0000","last_clean":"2024-05-14T15:22:42.893476+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:22:42.893476+0000","last_undegraded":"2024-05-14T15:22:42.893476+0000","last_fullsized":"2024-05-14T15:22:42.893476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892798+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:22:42.892798+0000","last_peered":"2024-05-14T15:22:42.892798+0000","last_clean":"2024-05-14T15:22:42.892798+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:22:42.892798+0000","last_undegraded":"2024-05-14T15:22:42.892798+0000","last_fullsized":"2024-05-14T15:22:42.892798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890039+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:22:42.890039+0000","last_peered":"2024-05-14T15:22:42.890039+0000","last_clean":"2024-05-14T15:22:42.890039+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:22:42.890039+0000","last_undegraded":"2024-05-14T15:22:42.890039+0000","last_fullsized":"2024-05-14T15:22:42.890039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892127+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:22:42.892127+0000","last_peered":"2024-05-14T15:22:42.892127+0000","last_clean":"2024-05-14T15:22:42.892127+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:22:42.892127+0000","last_undegraded":"2024-05-14T15:22:42.892127+0000","last_fullsized":"2024-05-14T15:22:42.892127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896215+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:22:42.896215+0000","last_peered":"2024-05-14T15:22:42.896215+0000","last_clean":"2024-05-14T15:22:42.896215+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:22:42.896215+0000","last_undegraded":"2024-05-14T15:22:42.896215+0000","last_fullsized":"2024-05-14T15:22:42.896215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892800+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:22:42.892800+0000","last_peered":"2024-05-14T15:22:42.892800+0000","last_clean":"2024-05-14T15:22:42.892800+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:22:42.892800+0000","last_undegraded":"2024-05-14T15:22:42.892800+0000","last_fullsized":"2024-05-14T15:22:42.892800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891307+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:22:42.891307+0000","last_peered":"2024-05-14T15:22:42.891307+0000","last_clean":"2024-05-14T15:22:42.891307+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:22:42.891307+0000","last_undegraded":"2024-05-14T15:22:42.891307+0000","last_fullsized":"2024-05-14T15:22:42.891307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229522+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:22:43.229522+0000","last_peered":"2024-05-14T15:22:43.229522+0000","last_clean":"2024-05-14T15:22:43.229522+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:22:43.229522+0000","last_undegraded":"2024-05-14T15:22:43.229522+0000","last_fullsized":"2024-05-14T15:22:43.229522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889246+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:22:42.889246+0000","last_peered":"2024-05-14T15:22:42.889246+0000","last_clean":"2024-05-14T15:22:42.889246+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:22:42.889246+0000","last_undegraded":"2024-05-14T15:22:42.889246+0000","last_fullsized":"2024-05-14T15:22:42.889246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.227979+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:22:43.227979+0000","last_peered":"2024-05-14T15:22:43.227979+0000","last_clean":"2024-05-14T15:22:43.227979+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:22:43.227979+0000","last_undegraded":"2024-05-14T15:22:43.227979+0000","last_fullsized":"2024-05-14T15:22:43.227979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228055+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:22:43.228055+0000","last_peered":"2024-05-14T15:22:43.228055+0000","last_clean":"2024-05-14T15:22:43.228055+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:22:43.228055+0000","last_undegraded":"2024-05-14T15:22:43.228055+0000","last_fullsized":"2024-05-14T15:22:43.228055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895788+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:22:42.895788+0000","last_peered":"2024-05-14T15:22:42.895788+0000","last_clean":"2024-05-14T15:22:42.895788+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:22:42.895788+0000","last_undegraded":"2024-05-14T15:22:42.895788+0000","last_fullsized":"2024-05-14T15:22:42.895788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893247+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:22:42.893247+0000","last_peered":"2024-05-14T15:22:42.893247+0000","last_clean":"2024-05-14T15:22:42.893247+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:22:42.893247+0000","last_undegraded":"2024-05-14T15:22:42.893247+0000","last_fullsized":"2024-05-14T15:22:42.893247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229325+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:22:43.229325+0000","last_peered":"2024-05-14T15:22:43.229325+0000","last_clean":"2024-05-14T15:22:43.229325+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:22:43.229325+0000","last_undegraded":"2024-05-14T15:22:43.229325+0000","last_fullsized":"2024-05-14T15:22:43.229325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895766+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:22:42.895766+0000","last_peered":"2024-05-14T15:22:42.895766+0000","last_clean":"2024-05-14T15:22:42.895766+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:22:42.895766+0000","last_undegraded":"2024-05-14T15:22:42.895766+0000","last_fullsized":"2024-05-14T15:22:42.895766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895668+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:22:42.895668+0000","last_peered":"2024-05-14T15:22:42.895668+0000","last_clean":"2024-05-14T15:22:42.895668+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:22:42.895668+0000","last_undegraded":"2024-05-14T15:22:42.895668+0000","last_fullsized":"2024-05-14T15:22:42.895668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228749+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:22:43.228749+0000","last_peered":"2024-05-14T15:22:43.228749+0000","last_clean":"2024-05-14T15:22:43.228749+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:22:43.228749+0000","last_undegraded":"2024-05-14T15:22:43.228749+0000","last_fullsized":"2024-05-14T15:22:43.228749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892480+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:22:42.892480+0000","last_peered":"2024-05-14T15:22:42.892480+0000","last_clean":"2024-05-14T15:22:42.892480+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:22:42.892480+0000","last_undegraded":"2024-05-14T15:22:42.892480+0000","last_fullsized":"2024-05-14T15:22:42.892480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228543+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:22:43.228543+0000","last_peered":"2024-05-14T15:22:43.228543+0000","last_clean":"2024-05-14T15:22:43.228543+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:22:43.228543+0000","last_undegraded":"2024-05-14T15:22:43.228543+0000","last_fullsized":"2024-05-14T15:22:43.228543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892694+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:22:42.892694+0000","last_peered":"2024-05-14T15:22:42.892694+0000","last_clean":"2024-05-14T15:22:42.892694+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:22:42.892694+0000","last_undegraded":"2024-05-14T15:22:42.892694+0000","last_fullsized":"2024-05-14T15:22:42.892694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228669+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:22:43.228669+0000","last_peered":"2024-05-14T15:22:43.228669+0000","last_clean":"2024-05-14T15:22:43.228669+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:22:43.228669+0000","last_undegraded":"2024-05-14T15:22:43.228669+0000","last_fullsized":"2024-05-14T15:22:43.228669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228842+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:22:43.228842+0000","last_peered":"2024-05-14T15:22:43.228842+0000","last_clean":"2024-05-14T15:22:43.228842+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:22:43.228842+0000","last_undegraded":"2024-05-14T15:22:43.228842+0000","last_fullsized":"2024-05-14T15:22:43.228842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895009+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:22:42.895009+0000","last_peered":"2024-05-14T15:22:42.895009+0000","last_clean":"2024-05-14T15:22:42.895009+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:22:42.895009+0000","last_undegraded":"2024-05-14T15:22:42.895009+0000","last_fullsized":"2024-05-14T15:22:42.895009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895721+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:22:42.895721+0000","last_peered":"2024-05-14T15:22:42.895721+0000","last_clean":"2024-05-14T15:22:42.895721+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:22:42.895721+0000","last_undegraded":"2024-05-14T15:22:42.895721+0000","last_fullsized":"2024-05-14T15:22:42.895721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228817+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:22:43.228817+0000","last_peered":"2024-05-14T15:22:43.228817+0000","last_clean":"2024-05-14T15:22:43.228817+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:22:43.228817+0000","last_undegraded":"2024-05-14T15:22:43.228817+0000","last_fullsized":"2024-05-14T15:22:43.228817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894409+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:22:42.894409+0000","last_peered":"2024-05-14T15:22:42.894409+0000","last_clean":"2024-05-14T15:22:42.894409+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:22:42.894409+0000","last_undegraded":"2024-05-14T15:22:42.894409+0000","last_fullsized":"2024-05-14T15:22:42.894409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891060+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:22:42.891060+0000","last_peered":"2024-05-14T15:22:42.891060+0000","last_clean":"2024-05-14T15:22:42.891060+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:22:42.891060+0000","last_undegraded":"2024-05-14T15:22:42.891060+0000","last_fullsized":"2024-05-14T15:22:42.891060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895391+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:22:42.895391+0000","last_peered":"2024-05-14T15:22:42.895391+0000","last_clean":"2024-05-14T15:22:42.895391+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:22:42.895391+0000","last_undegraded":"2024-05-14T15:22:42.895391+0000","last_fullsized":"2024-05-14T15:22:42.895391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:33.926017+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:23:33.926017+0000","last_peered":"2024-05-14T15:23:33.926017+0000","last_clean":"2024-05-14T15:23:33.926017+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:23:33.926017+0000","last_undegraded":"2024-05-14T15:23:33.926017+0000","last_fullsized":"2024-05-14T15:23:33.926017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893810+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:22:42.893810+0000","last_peered":"2024-05-14T15:22:42.893810+0000","last_clean":"2024-05-14T15:22:42.893810+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:22:42.893810+0000","last_undegraded":"2024-05-14T15:22:42.893810+0000","last_fullsized":"2024-05-14T15:22:42.893810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895150+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:22:42.895150+0000","last_peered":"2024-05-14T15:22:42.895150+0000","last_clean":"2024-05-14T15:22:42.895150+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:22:42.895150+0000","last_undegraded":"2024-05-14T15:22:42.895150+0000","last_fullsized":"2024-05-14T15:22:42.895150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.233584+0000","last_change":"2024-05-14T15:22:43.233044+0000","last_active":"2024-05-14T15:22:43.233584+0000","last_peered":"2024-05-14T15:22:43.233584+0000","last_clean":"2024-05-14T15:22:43.233584+0000","last_became_active":"2024-05-14T15:22:43.229808+0000","last_became_peered":"2024-05-14T15:22:43.229808+0000","last_unstale":"2024-05-14T15:22:43.233584+0000","last_undegraded":"2024-05-14T15:22:43.233584+0000","last_fullsized":"2024-05-14T15:22:43.233584+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:07.226460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890849+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:22:42.890849+0000","last_peered":"2024-05-14T15:22:42.890849+0000","last_clean":"2024-05-14T15:22:42.890849+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:22:42.890849+0000","last_undegraded":"2024-05-14T15:22:42.890849+0000","last_fullsized":"2024-05-14T15:22:42.890849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892474+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:22:42.892474+0000","last_peered":"2024-05-14T15:22:42.892474+0000","last_clean":"2024-05-14T15:22:42.892474+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:22:42.892474+0000","last_undegraded":"2024-05-14T15:22:42.892474+0000","last_fullsized":"2024-05-14T15:22:42.892474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889261+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:22:42.889261+0000","last_peered":"2024-05-14T15:22:42.889261+0000","last_clean":"2024-05-14T15:22:42.889261+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:22:42.889261+0000","last_undegraded":"2024-05-14T15:22:42.889261+0000","last_fullsized":"2024-05-14T15:22:42.889261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892324+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:22:42.892324+0000","last_peered":"2024-05-14T15:22:42.892324+0000","last_clean":"2024-05-14T15:22:42.892324+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:22:42.892324+0000","last_undegraded":"2024-05-14T15:22:42.892324+0000","last_fullsized":"2024-05-14T15:22:42.892324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229134+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:22:43.229134+0000","last_peered":"2024-05-14T15:22:43.229134+0000","last_clean":"2024-05-14T15:22:43.229134+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:22:43.229134+0000","last_undegraded":"2024-05-14T15:22:43.229134+0000","last_fullsized":"2024-05-14T15:22:43.229134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.900544+0000","last_change":"2024-05-14T15:22:42.900544+0000","last_active":"2024-05-14T15:22:42.900544+0000","last_peered":"2024-05-14T15:22:42.900544+0000","last_clean":"2024-05-14T15:22:42.900544+0000","last_became_active":"2024-05-14T15:22:42.900015+0000","last_became_peered":"2024-05-14T15:22:42.900015+0000","last_unstale":"2024-05-14T15:22:42.900544+0000","last_undegraded":"2024-05-14T15:22:42.900544+0000","last_fullsized":"2024-05-14T15:22:42.900544+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:15.904051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228181+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:22:43.228181+0000","last_peered":"2024-05-14T15:22:43.228181+0000","last_clean":"2024-05-14T15:22:43.228181+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:22:43.228181+0000","last_undegraded":"2024-05-14T15:22:43.228181+0000","last_fullsized":"2024-05-14T15:22:43.228181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892276+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:22:42.892276+0000","last_peered":"2024-05-14T15:22:42.892276+0000","last_clean":"2024-05-14T15:22:42.892276+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:22:42.892276+0000","last_undegraded":"2024-05-14T15:22:42.892276+0000","last_fullsized":"2024-05-14T15:22:42.892276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:33.925418+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:23:33.925418+0000","last_peered":"2024-05-14T15:23:33.925418+0000","last_clean":"2024-05-14T15:23:33.925418+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:23:33.925418+0000","last_undegraded":"2024-05-14T15:23:33.925418+0000","last_fullsized":"2024-05-14T15:23:33.925418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896216+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:22:42.896216+0000","last_peered":"2024-05-14T15:22:42.896216+0000","last_clean":"2024-05-14T15:22:42.896216+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:22:42.896216+0000","last_undegraded":"2024-05-14T15:22:42.896216+0000","last_fullsized":"2024-05-14T15:22:42.896216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.227944+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:22:43.227944+0000","last_peered":"2024-05-14T15:22:43.227944+0000","last_clean":"2024-05-14T15:22:43.227944+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:22:43.227944+0000","last_undegraded":"2024-05-14T15:22:43.227944+0000","last_fullsized":"2024-05-14T15:22:43.227944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891487+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:22:42.891487+0000","last_peered":"2024-05-14T15:22:42.891487+0000","last_clean":"2024-05-14T15:22:42.891487+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:22:42.891487+0000","last_undegraded":"2024-05-14T15:22:42.891487+0000","last_fullsized":"2024-05-14T15:22:42.891487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228713+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:22:43.228713+0000","last_peered":"2024-05-14T15:22:43.228713+0000","last_clean":"2024-05-14T15:22:43.228713+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:22:43.228713+0000","last_undegraded":"2024-05-14T15:22:43.228713+0000","last_fullsized":"2024-05-14T15:22:43.228713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228987+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:22:43.228987+0000","last_peered":"2024-05-14T15:22:43.228987+0000","last_clean":"2024-05-14T15:22:43.228987+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:22:43.228987+0000","last_undegraded":"2024-05-14T15:22:43.228987+0000","last_fullsized":"2024-05-14T15:22:43.228987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890800+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:22:42.890800+0000","last_peered":"2024-05-14T15:22:42.890800+0000","last_clean":"2024-05-14T15:22:42.890800+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:22:42.890800+0000","last_undegraded":"2024-05-14T15:22:42.890800+0000","last_fullsized":"2024-05-14T15:22:42.890800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895782+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:22:42.895782+0000","last_peered":"2024-05-14T15:22:42.895782+0000","last_clean":"2024-05-14T15:22:42.895782+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:22:42.895782+0000","last_undegraded":"2024-05-14T15:22:42.895782+0000","last_fullsized":"2024-05-14T15:22:42.895782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228275+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:22:43.228275+0000","last_peered":"2024-05-14T15:22:43.228275+0000","last_clean":"2024-05-14T15:22:43.228275+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:22:43.228275+0000","last_undegraded":"2024-05-14T15:22:43.228275+0000","last_fullsized":"2024-05-14T15:22:43.228275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":282,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:34.926553+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:23:34.926553+0000","last_peered":"2024-05-14T15:23:34.926553+0000","last_clean":"2024-05-14T15:23:34.926553+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:23:34.926553+0000","last_undegraded":"2024-05-14T15:23:34.926553+0000","last_fullsized":"2024-05-14T15:23:34.926553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":164,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894843+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:22:42.894843+0000","last_peered":"2024-05-14T15:22:42.894843+0000","last_clean":"2024-05-14T15:22:42.894843+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:22:42.894843+0000","last_undegraded":"2024-05-14T15:22:42.894843+0000","last_fullsized":"2024-05-14T15:22:42.894843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891424+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:22:42.891424+0000","last_peered":"2024-05-14T15:22:42.891424+0000","last_clean":"2024-05-14T15:22:42.891424+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:22:42.891424+0000","last_undegraded":"2024-05-14T15:22:42.891424+0000","last_fullsized":"2024-05-14T15:22:42.891424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896121+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:22:42.896121+0000","last_peered":"2024-05-14T15:22:42.896121+0000","last_clean":"2024-05-14T15:22:42.896121+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:22:42.896121+0000","last_undegraded":"2024-05-14T15:22:42.896121+0000","last_fullsized":"2024-05-14T15:22:42.896121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889457+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:22:42.889457+0000","last_peered":"2024-05-14T15:22:42.889457+0000","last_clean":"2024-05-14T15:22:42.889457+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:22:42.889457+0000","last_undegraded":"2024-05-14T15:22:42.889457+0000","last_fullsized":"2024-05-14T15:22:42.889457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892668+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:22:42.892668+0000","last_peered":"2024-05-14T15:22:42.892668+0000","last_clean":"2024-05-14T15:22:42.892668+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:22:42.892668+0000","last_undegraded":"2024-05-14T15:22:42.892668+0000","last_fullsized":"2024-05-14T15:22:42.892668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890575+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:22:42.890575+0000","last_peered":"2024-05-14T15:22:42.890575+0000","last_clean":"2024-05-14T15:22:42.890575+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:22:42.890575+0000","last_undegraded":"2024-05-14T15:22:42.890575+0000","last_fullsized":"2024-05-14T15:22:42.890575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890147+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:22:42.890147+0000","last_peered":"2024-05-14T15:22:42.890147+0000","last_clean":"2024-05-14T15:22:42.890147+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:22:42.890147+0000","last_undegraded":"2024-05-14T15:22:42.890147+0000","last_fullsized":"2024-05-14T15:22:42.890147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228499+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:22:43.228499+0000","last_peered":"2024-05-14T15:22:43.228499+0000","last_clean":"2024-05-14T15:22:43.228499+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:22:43.228499+0000","last_undegraded":"2024-05-14T15:22:43.228499+0000","last_fullsized":"2024-05-14T15:22:43.228499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:34.931476+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:23:34.931476+0000","last_peered":"2024-05-14T15:23:34.931476+0000","last_clean":"2024-05-14T15:23:34.931476+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:23:34.931476+0000","last_undegraded":"2024-05-14T15:23:34.931476+0000","last_fullsized":"2024-05-14T15:23:34.931476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890455+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:22:42.890455+0000","last_peered":"2024-05-14T15:22:42.890455+0000","last_clean":"2024-05-14T15:22:42.890455+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:22:42.890455+0000","last_undegraded":"2024-05-14T15:22:42.890455+0000","last_fullsized":"2024-05-14T15:22:42.890455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889844+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:22:42.889844+0000","last_peered":"2024-05-14T15:22:42.889844+0000","last_clean":"2024-05-14T15:22:42.889844+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:22:42.889844+0000","last_undegraded":"2024-05-14T15:22:42.889844+0000","last_fullsized":"2024-05-14T15:22:42.889844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893796+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:22:42.893796+0000","last_peered":"2024-05-14T15:22:42.893796+0000","last_clean":"2024-05-14T15:22:42.893796+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:22:42.893796+0000","last_undegraded":"2024-05-14T15:22:42.893796+0000","last_fullsized":"2024-05-14T15:22:42.893796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890230+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:22:42.890230+0000","last_peered":"2024-05-14T15:22:42.890230+0000","last_clean":"2024-05-14T15:22:42.890230+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:22:42.890230+0000","last_undegraded":"2024-05-14T15:22:42.890230+0000","last_fullsized":"2024-05-14T15:22:42.890230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890053+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:22:42.890053+0000","last_peered":"2024-05-14T15:22:42.890053+0000","last_clean":"2024-05-14T15:22:42.890053+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:22:42.890053+0000","last_undegraded":"2024-05-14T15:22:42.890053+0000","last_fullsized":"2024-05-14T15:22:42.890053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894730+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:22:42.894730+0000","last_peered":"2024-05-14T15:22:42.894730+0000","last_clean":"2024-05-14T15:22:42.894730+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:22:42.894730+0000","last_undegraded":"2024-05-14T15:22:42.894730+0000","last_fullsized":"2024-05-14T15:22:42.894730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892793+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:22:42.892793+0000","last_peered":"2024-05-14T15:22:42.892793+0000","last_clean":"2024-05-14T15:22:42.892793+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:22:42.892793+0000","last_undegraded":"2024-05-14T15:22:42.892793+0000","last_fullsized":"2024-05-14T15:22:42.892793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893702+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:22:42.893702+0000","last_peered":"2024-05-14T15:22:42.893702+0000","last_clean":"2024-05-14T15:22:42.893702+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:22:42.893702+0000","last_undegraded":"2024-05-14T15:22:42.893702+0000","last_fullsized":"2024-05-14T15:22:42.893702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228983+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:22:43.228983+0000","last_peered":"2024-05-14T15:22:43.228983+0000","last_clean":"2024-05-14T15:22:43.228983+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:22:43.228983+0000","last_undegraded":"2024-05-14T15:22:43.228983+0000","last_fullsized":"2024-05-14T15:22:43.228983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892043+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:22:42.892043+0000","last_peered":"2024-05-14T15:22:42.892043+0000","last_clean":"2024-05-14T15:22:42.892043+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:22:42.892043+0000","last_undegraded":"2024-05-14T15:22:42.892043+0000","last_fullsized":"2024-05-14T15:22:42.892043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889458+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:22:42.889458+0000","last_peered":"2024-05-14T15:22:42.889458+0000","last_clean":"2024-05-14T15:22:42.889458+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:22:42.889458+0000","last_undegraded":"2024-05-14T15:22:42.889458+0000","last_fullsized":"2024-05-14T15:22:42.889458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890147+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:22:42.890147+0000","last_peered":"2024-05-14T15:22:42.890147+0000","last_clean":"2024-05-14T15:22:42.890147+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:22:42.890147+0000","last_undegraded":"2024-05-14T15:22:42.890147+0000","last_fullsized":"2024-05-14T15:22:42.890147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892446+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:22:42.892446+0000","last_peered":"2024-05-14T15:22:42.892446+0000","last_clean":"2024-05-14T15:22:42.892446+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:22:42.892446+0000","last_undegraded":"2024-05-14T15:22:42.892446+0000","last_fullsized":"2024-05-14T15:22:42.892446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889900+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:22:42.889900+0000","last_peered":"2024-05-14T15:22:42.889900+0000","last_clean":"2024-05-14T15:22:42.889900+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:22:42.889900+0000","last_undegraded":"2024-05-14T15:22:42.889900+0000","last_fullsized":"2024-05-14T15:22:42.889900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890875+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:22:42.890875+0000","last_peered":"2024-05-14T15:22:42.890875+0000","last_clean":"2024-05-14T15:22:42.890875+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:22:42.890875+0000","last_undegraded":"2024-05-14T15:22:42.890875+0000","last_fullsized":"2024-05-14T15:22:42.890875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:23:32.926980+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:23:32.926980+0000","last_peered":"2024-05-14T15:23:32.926980+0000","last_clean":"2024-05-14T15:23:32.926980+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:23:32.926980+0000","last_undegraded":"2024-05-14T15:23:32.926980+0000","last_fullsized":"2024-05-14T15:23:32.926980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891147+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:22:42.891147+0000","last_peered":"2024-05-14T15:22:42.891147+0000","last_clean":"2024-05-14T15:22:42.891147+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:22:42.891147+0000","last_undegraded":"2024-05-14T15:22:42.891147+0000","last_fullsized":"2024-05-14T15:22:42.891147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892930+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:22:42.892930+0000","last_peered":"2024-05-14T15:22:42.892930+0000","last_clean":"2024-05-14T15:22:42.892930+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:22:42.892930+0000","last_undegraded":"2024-05-14T15:22:42.892930+0000","last_fullsized":"2024-05-14T15:22:42.892930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228957+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:22:43.228957+0000","last_peered":"2024-05-14T15:22:43.228957+0000","last_clean":"2024-05-14T15:22:43.228957+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:22:43.228957+0000","last_undegraded":"2024-05-14T15:22:43.228957+0000","last_fullsized":"2024-05-14T15:22:43.228957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889825+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:22:42.889825+0000","last_peered":"2024-05-14T15:22:42.889825+0000","last_clean":"2024-05-14T15:22:42.889825+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:22:42.889825+0000","last_undegraded":"2024-05-14T15:22:42.889825+0000","last_fullsized":"2024-05-14T15:22:42.889825+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891290+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:22:42.891290+0000","last_peered":"2024-05-14T15:22:42.891290+0000","last_clean":"2024-05-14T15:22:42.891290+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:22:42.891290+0000","last_undegraded":"2024-05-14T15:22:42.891290+0000","last_fullsized":"2024-05-14T15:22:42.891290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889043+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:22:42.889043+0000","last_peered":"2024-05-14T15:22:42.889043+0000","last_clean":"2024-05-14T15:22:42.889043+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:22:42.889043+0000","last_undegraded":"2024-05-14T15:22:42.889043+0000","last_fullsized":"2024-05-14T15:22:42.889043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.903832+0000","last_change":"2024-05-14T15:22:42.903272+0000","last_active":"2024-05-14T15:22:42.903832+0000","last_peered":"2024-05-14T15:22:42.903832+0000","last_clean":"2024-05-14T15:22:42.903832+0000","last_became_active":"2024-05-14T15:22:42.900088+0000","last_became_peered":"2024-05-14T15:22:42.900088+0000","last_unstale":"2024-05-14T15:22:42.903832+0000","last_undegraded":"2024-05-14T15:22:42.903832+0000","last_fullsized":"2024-05-14T15:22:42.903832+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:23.975904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895414+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:22:42.895414+0000","last_peered":"2024-05-14T15:22:42.895414+0000","last_clean":"2024-05-14T15:22:42.895414+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:22:42.895414+0000","last_undegraded":"2024-05-14T15:22:42.895414+0000","last_fullsized":"2024-05-14T15:22:42.895414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894757+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:22:42.894757+0000","last_peered":"2024-05-14T15:22:42.894757+0000","last_clean":"2024-05-14T15:22:42.894757+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:22:42.894757+0000","last_undegraded":"2024-05-14T15:22:42.894757+0000","last_fullsized":"2024-05-14T15:22:42.894757+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889686+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:22:42.889686+0000","last_peered":"2024-05-14T15:22:42.889686+0000","last_clean":"2024-05-14T15:22:42.889686+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:22:42.889686+0000","last_undegraded":"2024-05-14T15:22:42.889686+0000","last_fullsized":"2024-05-14T15:22:42.889686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892174+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:22:42.892174+0000","last_peered":"2024-05-14T15:22:42.892174+0000","last_clean":"2024-05-14T15:22:42.892174+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:22:42.892174+0000","last_undegraded":"2024-05-14T15:22:42.892174+0000","last_fullsized":"2024-05-14T15:22:42.892174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229405+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:22:43.229405+0000","last_peered":"2024-05-14T15:22:43.229405+0000","last_clean":"2024-05-14T15:22:43.229405+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:22:43.229405+0000","last_undegraded":"2024-05-14T15:22:43.229405+0000","last_fullsized":"2024-05-14T15:22:43.229405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892912+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:22:42.892912+0000","last_peered":"2024-05-14T15:22:42.892912+0000","last_clean":"2024-05-14T15:22:42.892912+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:22:42.892912+0000","last_undegraded":"2024-05-14T15:22:42.892912+0000","last_fullsized":"2024-05-14T15:22:42.892912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895293+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:22:42.895293+0000","last_peered":"2024-05-14T15:22:42.895293+0000","last_clean":"2024-05-14T15:22:42.895293+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:22:42.895293+0000","last_undegraded":"2024-05-14T15:22:42.895293+0000","last_fullsized":"2024-05-14T15:22:42.895293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229439+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:22:43.229439+0000","last_peered":"2024-05-14T15:22:43.229439+0000","last_clean":"2024-05-14T15:22:43.229439+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:22:43.229439+0000","last_undegraded":"2024-05-14T15:22:43.229439+0000","last_fullsized":"2024-05-14T15:22:43.229439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229283+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:22:43.229283+0000","last_peered":"2024-05-14T15:22:43.229283+0000","last_clean":"2024-05-14T15:22:43.229283+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:22:43.229283+0000","last_undegraded":"2024-05-14T15:22:43.229283+0000","last_fullsized":"2024-05-14T15:22:43.229283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229394+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:22:43.229394+0000","last_peered":"2024-05-14T15:22:43.229394+0000","last_clean":"2024-05-14T15:22:43.229394+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:22:43.229394+0000","last_undegraded":"2024-05-14T15:22:43.229394+0000","last_fullsized":"2024-05-14T15:22:43.229394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895849+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:22:42.895849+0000","last_peered":"2024-05-14T15:22:42.895849+0000","last_clean":"2024-05-14T15:22:42.895849+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:22:42.895849+0000","last_undegraded":"2024-05-14T15:22:42.895849+0000","last_fullsized":"2024-05-14T15:22:42.895849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.894144+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:22:42.894144+0000","last_peered":"2024-05-14T15:22:42.894144+0000","last_clean":"2024-05-14T15:22:42.894144+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:22:42.894144+0000","last_undegraded":"2024-05-14T15:22:42.894144+0000","last_fullsized":"2024-05-14T15:22:42.894144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228624+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:22:43.228624+0000","last_peered":"2024-05-14T15:22:43.228624+0000","last_clean":"2024-05-14T15:22:43.228624+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:22:43.228624+0000","last_undegraded":"2024-05-14T15:22:43.228624+0000","last_fullsized":"2024-05-14T15:22:43.228624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892417+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:22:42.892417+0000","last_peered":"2024-05-14T15:22:42.892417+0000","last_clean":"2024-05-14T15:22:42.892417+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:22:42.892417+0000","last_undegraded":"2024-05-14T15:22:42.892417+0000","last_fullsized":"2024-05-14T15:22:42.892417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893361+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:22:42.893361+0000","last_peered":"2024-05-14T15:22:42.893361+0000","last_clean":"2024-05-14T15:22:42.893361+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:22:42.893361+0000","last_undegraded":"2024-05-14T15:22:42.893361+0000","last_fullsized":"2024-05-14T15:22:42.893361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229122+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:22:43.229122+0000","last_peered":"2024-05-14T15:22:43.229122+0000","last_clean":"2024-05-14T15:22:43.229122+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:22:43.229122+0000","last_undegraded":"2024-05-14T15:22:43.229122+0000","last_fullsized":"2024-05-14T15:22:43.229122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895168+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:22:42.895168+0000","last_peered":"2024-05-14T15:22:42.895168+0000","last_clean":"2024-05-14T15:22:42.895168+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:22:42.895168+0000","last_undegraded":"2024-05-14T15:22:42.895168+0000","last_fullsized":"2024-05-14T15:22:42.895168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892727+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:22:42.892727+0000","last_peered":"2024-05-14T15:22:42.892727+0000","last_clean":"2024-05-14T15:22:42.892727+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:22:42.892727+0000","last_undegraded":"2024-05-14T15:22:42.892727+0000","last_fullsized":"2024-05-14T15:22:42.892727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.229462+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:22:43.229462+0000","last_peered":"2024-05-14T15:22:43.229462+0000","last_clean":"2024-05-14T15:22:43.229462+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:22:43.229462+0000","last_undegraded":"2024-05-14T15:22:43.229462+0000","last_fullsized":"2024-05-14T15:22:43.229462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.902298+0000","last_change":"2024-05-14T15:22:42.901814+0000","last_active":"2024-05-14T15:22:42.902298+0000","last_peered":"2024-05-14T15:22:42.902298+0000","last_clean":"2024-05-14T15:22:42.902298+0000","last_became_active":"2024-05-14T15:22:42.896795+0000","last_became_peered":"2024-05-14T15:22:42.896795+0000","last_unstale":"2024-05-14T15:22:42.902298+0000","last_undegraded":"2024-05-14T15:22:42.902298+0000","last_fullsized":"2024-05-14T15:22:42.902298+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:15.981532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.890195+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:22:42.890195+0000","last_peered":"2024-05-14T15:22:42.890195+0000","last_clean":"2024-05-14T15:22:42.890195+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:22:42.890195+0000","last_undegraded":"2024-05-14T15:22:42.890195+0000","last_fullsized":"2024-05-14T15:22:42.890195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895408+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:22:42.895408+0000","last_peered":"2024-05-14T15:22:42.895408+0000","last_clean":"2024-05-14T15:22:42.895408+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:22:42.895408+0000","last_undegraded":"2024-05-14T15:22:42.895408+0000","last_fullsized":"2024-05-14T15:22:42.895408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895524+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:22:42.895524+0000","last_peered":"2024-05-14T15:22:42.895524+0000","last_clean":"2024-05-14T15:22:42.895524+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:22:42.895524+0000","last_undegraded":"2024-05-14T15:22:42.895524+0000","last_fullsized":"2024-05-14T15:22:42.895524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892220+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:22:42.892220+0000","last_peered":"2024-05-14T15:22:42.892220+0000","last_clean":"2024-05-14T15:22:42.892220+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:22:42.892220+0000","last_undegraded":"2024-05-14T15:22:42.892220+0000","last_fullsized":"2024-05-14T15:22:42.892220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228424+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:22:43.228424+0000","last_peered":"2024-05-14T15:22:43.228424+0000","last_clean":"2024-05-14T15:22:43.228424+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:22:43.228424+0000","last_undegraded":"2024-05-14T15:22:43.228424+0000","last_fullsized":"2024-05-14T15:22:43.228424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891192+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:22:42.891192+0000","last_peered":"2024-05-14T15:22:42.891192+0000","last_clean":"2024-05-14T15:22:42.891192+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:22:42.891192+0000","last_undegraded":"2024-05-14T15:22:42.891192+0000","last_fullsized":"2024-05-14T15:22:42.891192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891158+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:22:42.891158+0000","last_peered":"2024-05-14T15:22:42.891158+0000","last_clean":"2024-05-14T15:22:42.891158+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:22:42.891158+0000","last_undegraded":"2024-05-14T15:22:42.891158+0000","last_fullsized":"2024-05-14T15:22:42.891158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889807+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:22:42.889807+0000","last_peered":"2024-05-14T15:22:42.889807+0000","last_clean":"2024-05-14T15:22:42.889807+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:22:42.889807+0000","last_undegraded":"2024-05-14T15:22:42.889807+0000","last_fullsized":"2024-05-14T15:22:42.889807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.236629+0000","last_change":"2024-05-14T15:22:43.235879+0000","last_active":"2024-05-14T15:22:43.236629+0000","last_peered":"2024-05-14T15:22:43.236629+0000","last_clean":"2024-05-14T15:22:43.236629+0000","last_became_active":"2024-05-14T15:22:43.230116+0000","last_became_peered":"2024-05-14T15:22:43.230116+0000","last_unstale":"2024-05-14T15:22:43.236629+0000","last_undegraded":"2024-05-14T15:22:43.236629+0000","last_fullsized":"2024-05-14T15:22:43.236629+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:25:50.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.891625+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:22:42.891625+0000","last_peered":"2024-05-14T15:22:42.891625+0000","last_clean":"2024-05-14T15:22:42.891625+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:22:42.891625+0000","last_undegraded":"2024-05-14T15:22:42.891625+0000","last_fullsized":"2024-05-14T15:22:42.891625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893515+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:22:42.893515+0000","last_peered":"2024-05-14T15:22:42.893515+0000","last_clean":"2024-05-14T15:22:42.893515+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:22:42.893515+0000","last_undegraded":"2024-05-14T15:22:42.893515+0000","last_fullsized":"2024-05-14T15:22:42.893515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896064+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:22:42.896064+0000","last_peered":"2024-05-14T15:22:42.896064+0000","last_clean":"2024-05-14T15:22:42.896064+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:22:42.896064+0000","last_undegraded":"2024-05-14T15:22:42.896064+0000","last_fullsized":"2024-05-14T15:22:42.896064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895071+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:22:42.895071+0000","last_peered":"2024-05-14T15:22:42.895071+0000","last_clean":"2024-05-14T15:22:42.895071+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:22:42.895071+0000","last_undegraded":"2024-05-14T15:22:42.895071+0000","last_fullsized":"2024-05-14T15:22:42.895071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:43.228542+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:22:43.228542+0000","last_peered":"2024-05-14T15:22:43.228542+0000","last_clean":"2024-05-14T15:22:43.228542+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:22:43.228542+0000","last_undegraded":"2024-05-14T15:22:43.228542+0000","last_fullsized":"2024-05-14T15:22:43.228542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889722+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:22:42.889722+0000","last_peered":"2024-05-14T15:22:42.889722+0000","last_clean":"2024-05-14T15:22:42.889722+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:22:42.889722+0000","last_undegraded":"2024-05-14T15:22:42.889722+0000","last_fullsized":"2024-05-14T15:22:42.889722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893428+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:22:42.893428+0000","last_peered":"2024-05-14T15:22:42.893428+0000","last_clean":"2024-05-14T15:22:42.893428+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:22:42.893428+0000","last_undegraded":"2024-05-14T15:22:42.893428+0000","last_fullsized":"2024-05-14T15:22:42.893428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.896524+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:22:42.896524+0000","last_peered":"2024-05-14T15:22:42.896524+0000","last_clean":"2024-05-14T15:22:42.896524+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:22:42.896524+0000","last_undegraded":"2024-05-14T15:22:42.896524+0000","last_fullsized":"2024-05-14T15:22:42.896524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893046+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:22:42.893046+0000","last_peered":"2024-05-14T15:22:42.893046+0000","last_clean":"2024-05-14T15:22:42.893046+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:22:42.893046+0000","last_undegraded":"2024-05-14T15:22:42.893046+0000","last_fullsized":"2024-05-14T15:22:42.893046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889979+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:22:42.889979+0000","last_peered":"2024-05-14T15:22:42.889979+0000","last_clean":"2024-05-14T15:22:42.889979+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:22:42.889979+0000","last_undegraded":"2024-05-14T15:22:42.889979+0000","last_fullsized":"2024-05-14T15:22:42.889979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.892587+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:22:42.892587+0000","last_peered":"2024-05-14T15:22:42.892587+0000","last_clean":"2024-05-14T15:22:42.892587+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:22:42.892587+0000","last_undegraded":"2024-05-14T15:22:42.892587+0000","last_fullsized":"2024-05-14T15:22:42.892587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.895659+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:22:42.895659+0000","last_peered":"2024-05-14T15:22:42.895659+0000","last_clean":"2024-05-14T15:22:42.895659+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:22:42.895659+0000","last_undegraded":"2024-05-14T15:22:42.895659+0000","last_fullsized":"2024-05-14T15:22:42.895659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.889808+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:22:42.889808+0000","last_peered":"2024-05-14T15:22:42.889808+0000","last_clean":"2024-05-14T15:22:42.889808+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:22:42.889808+0000","last_undegraded":"2024-05-14T15:22:42.889808+0000","last_fullsized":"2024-05-14T15:22:42.889808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-14T15:22:42.893157+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:22:42.893157+0000","last_peered":"2024-05-14T15:22:42.893157+0000","last_clean":"2024-05-14T15:22:42.893157+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:22:42.893157+0000","last_undegraded":"2024-05-14T15:22:42.893157+0000","last_fullsized":"2024-05-14T15:22:42.893157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283950,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66658,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":616,"num_read_kb":378,"num_write":168,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678065,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":518237,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874293,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":1168162,"data_compressed":68468,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":522569,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234048,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":518393,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430277,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":1114302,"data_compressed":68468,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593801,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":513649,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":576216,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":515037,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":577957,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":513843,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953556,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":518299,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790030,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1109890,"data_compressed":68468,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34480,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:23:40.443 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T15:23:40.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-14T15:23:40.443 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-14T15:23:40.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-14T15:23:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:40.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:40 smithi005 ceph-mon[31559]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:40.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:40 smithi005 ceph-mon[31559]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:23:41.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:40 smithi144 ceph-mon[35650]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:41.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:40 smithi144 ceph-mon[35650]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:23:41.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:40 smithi193 ceph-mon[36001]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:41.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:40 smithi193 ceph-mon[36001]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:23:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 -- 172.21.15.5:0/4106743874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34108f10 msgr2=0x7fdf34109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 --2- 172.21.15.5:0/4106743874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34108f10 0x7fdf34109320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0099c0 tx=0x7fdf1c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 -- 172.21.15.5:0/4106743874 shutdown_connections 2024-05-14T15:23:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 --2- 172.21.15.5:0/4106743874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 0x7fdf341111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 --2- 172.21.15.5:0/4106743874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf341045e0 0x7fdf341049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 --2- 172.21.15.5:0/4106743874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34108f10 0x7fdf34109320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 -- 172.21.15.5:0/4106743874 >> 172.21.15.5:0/4106743874 conn(0x7fdf340ffed0 msgr2=0x7fdf34102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 -- 172.21.15.5:0/4106743874 shutdown_connections 2024-05-14T15:23:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.367+0000 7fdf38896700 1 -- 172.21.15.5:0/4106743874 wait complete. 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.368+0000 7fdf38896700 1 Processor -- start 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 -- start start 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 0x7fdf3419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf34108f10 0x7fdf3419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 0x7fdf3419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34114470 con 0x7fdf341045e0 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341142f0 con 0x7fdf34108f10 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf38896700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341145f0 con 0x7fdf34109860 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 0x7fdf3419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 0x7fdf3419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48070/0 (socket says 172.21.15.5:48070) 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 -- 172.21.15.5:0/2539736980 learned_addr learned my addr 172.21.15.5:0/2539736980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf31d9b700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf34108f10 0x7fdf3419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf32d9d700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 0x7fdf3419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 -- 172.21.15.5:0/2539736980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 msgr2=0x7fdf3419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 0x7fdf3419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 -- 172.21.15.5:0/2539736980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf34108f10 msgr2=0x7fdf3419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf34108f10 0x7fdf3419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf3259c700 1 -- 172.21.15.5:0/2539736980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419fc60 con 0x7fdf341045e0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.369+0000 7fdf32d9d700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 0x7fdf3419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.370+0000 7fdf3259c700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 0x7fdf3419ae40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d950 tx=0x7fdf2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.370+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24016070 con 0x7fdf341045e0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.370+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419ff50 con 0x7fdf341045e0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.370+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf24009df0 con 0x7fdf341045e0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.370+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400f3c0 con 0x7fdf341045e0 2024-05-14T15:23:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.370+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf341a0460 con 0x7fdf341045e0 2024-05-14T15:23:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.371+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf340512c0 con 0x7fdf341045e0 2024-05-14T15:23:42.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.373+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdf24019070 con 0x7fdf341045e0 2024-05-14T15:23:42.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.374+0000 7fdf2b7fe700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdf200b2880 0x7fdf200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:42.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.374+0000 7fdf31d9b700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdf200b2880 0x7fdf200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:42.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.374+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fdf240e21d0 con 0x7fdf341045e0 2024-05-14T15:23:42.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.375+0000 7fdf31d9b700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdf200b2880 0x7fdf200b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006010 tx=0x7fdf1c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:42.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.378+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf240a9b20 con 0x7fdf341045e0 2024-05-14T15:23:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.642+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdf3402e5f0 con 0x7fdf341045e0 2024-05-14T15:23:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.642+0000 7fdf2b7fe700 1 -- 172.21.15.5:0/2539736980 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdf240a9420 con 0x7fdf341045e0 2024-05-14T15:23:42.643 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:42.643 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T15:23:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.644+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdf200b2880 msgr2=0x7fdf200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.644+0000 7fdf38896700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdf200b2880 0x7fdf200b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006010 tx=0x7fdf1c000bc0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.644+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 msgr2=0x7fdf3419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.644+0000 7fdf38896700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 0x7fdf3419ae40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d950 tx=0x7fdf2400dc60 comp rx=0 tx=0).stop 2024-05-14T15:23:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 shutdown_connections 2024-05-14T15:23:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdf200b2880 0x7fdf200b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf34109860 0x7fdf3419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf341045e0 0x7fdf3419ae40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 --2- 172.21.15.5:0/2539736980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf34108f10 0x7fdf3419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:42.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 >> 172.21.15.5:0/2539736980 conn(0x7fdf340ffed0 msgr2=0x7fdf3410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:42.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 shutdown_connections 2024-05-14T15:23:42.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:42.645+0000 7fdf38896700 1 -- 172.21.15.5:0/2539736980 wait complete. 2024-05-14T15:23:42.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:42 smithi005 ceph-mon[31559]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:42.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:42 smithi005 ceph-mon[31559]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-14T15:23:42.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:42 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2539736980' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T15:23:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:42 smithi144 ceph-mon[35650]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:42 smithi144 ceph-mon[35650]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-14T15:23:43.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:42 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2539736980' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T15:23:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:42 smithi193 ceph-mon[36001]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:23:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:42 smithi193 ceph-mon[36001]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-14T15:23:43.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:42 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2539736980' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T15:23:43.371 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-14T15:23:43.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:23:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:44.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:44 smithi005 ceph-mon[31559]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:45.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:44 smithi144 ceph-mon[35650]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:45.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:44 smithi193 ceph-mon[36001]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:23:45.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 -- 172.21.15.5:0/2499046828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 msgr2=0x7fa3301100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:45.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 --2- 172.21.15.5:0/2499046828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa3301100c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b0d0 tx=0x7fa32000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 -- 172.21.15.5:0/2499046828 shutdown_connections 2024-05-14T15:23:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 --2- 172.21.15.5:0/2499046828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa3301100c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 --2- 172.21.15.5:0/2499046828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3301034e0 0x7fa3301038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 --2- 172.21.15.5:0/2499046828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa330107e10 0x7fa330108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.432+0000 7fa33487b700 1 -- 172.21.15.5:0/2499046828 >> 172.21.15.5:0/2499046828 conn(0x7fa3300fef30 msgr2=0x7fa330101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.433+0000 7fa33487b700 1 -- 172.21.15.5:0/2499046828 shutdown_connections 2024-05-14T15:23:45.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.433+0000 7fa33487b700 1 -- 172.21.15.5:0/2499046828 wait complete. 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.433+0000 7fa33487b700 1 Processor -- start 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.433+0000 7fa33487b700 1 -- start start 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa33487b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3301034e0 0x7fa33019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa33487b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330107e10 0x7fa33019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa33487b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa33019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa33487b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa330113480 con 0x7fa330108760 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa33487b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa330113300 con 0x7fa3301034e0 2024-05-14T15:23:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa33487b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa330113600 con 0x7fa330107e10 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa33019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa33019c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48086/0 (socket says 172.21.15.5:48086) 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 -- 172.21.15.5:0/2828957422 learned_addr learned my addr 172.21.15.5:0/2828957422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32e59c700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3301034e0 0x7fa33019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32dd9b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330107e10 0x7fa33019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 -- 172.21.15.5:0/2828957422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330107e10 msgr2=0x7fa33019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330107e10 0x7fa33019c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 -- 172.21.15.5:0/2828957422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3301034e0 msgr2=0x7fa33019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3301034e0 0x7fa33019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.434+0000 7fa32ed9d700 1 -- 172.21.15.5:0/2828957422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3301a7150 con 0x7fa330108760 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.435+0000 7fa32ed9d700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa33019c8a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa320009fd0 tx=0x7fa320007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.435+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32001f070 con 0x7fa330108760 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.435+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3301a73e0 con 0x7fa330108760 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.435+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa320003c00 con 0x7fa330108760 2024-05-14T15:23:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.435+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32001b8e0 con 0x7fa330108760 2024-05-14T15:23:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.435+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a7840 con 0x7fa330108760 2024-05-14T15:23:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.436+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3300512c0 con 0x7fa330108760 2024-05-14T15:23:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.437+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa320022070 con 0x7fa330108760 2024-05-14T15:23:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.438+0000 7fa3277fe700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa31c0b2880 0x7fa31c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.438+0000 7fa32e59c700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa31c0b2880 0x7fa31c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.438+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fa32000f020 con 0x7fa330108760 2024-05-14T15:23:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.439+0000 7fa32e59c700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa31c0b2880 0x7fa31c0b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa328006fd0 tx=0x7fa328008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.442+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa320029020 con 0x7fa330108760 2024-05-14T15:23:45.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.705+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa33002e5f0 con 0x7fa330108760 2024-05-14T15:23:45.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.706+0000 7fa3277fe700 1 -- 172.21.15.5:0/2828957422 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11864 (secure 0 0 0) 0x7fa3200ab760 con 0x7fa330108760 2024-05-14T15:23:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:45.707 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:22:00.688182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24487,24515]},"id":1}]} 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa31c0b2880 msgr2=0x7fa31c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa31c0b2880 0x7fa31c0b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa328006fd0 tx=0x7fa328008040 comp rx=0 tx=0).stop 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 msgr2=0x7fa33019c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa33019c8a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa320009fd0 tx=0x7fa320007ad0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 shutdown_connections 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fa31c0b2880 0x7fa31c0b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa330108760 0x7fa33019c8a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3301034e0 0x7fa33019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 --2- 172.21.15.5:0/2828957422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa330107e10 0x7fa33019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:45.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 >> 172.21.15.5:0/2828957422 conn(0x7fa3300fef30 msgr2=0x7fa3301075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 shutdown_connections 2024-05-14T15:23:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:45.709+0000 7fa33487b700 1 -- 172.21.15.5:0/2828957422 wait complete. 2024-05-14T15:23:45.711 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:23:46.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:23:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:46.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:46 smithi005 ceph-mon[31559]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:46.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:46 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2828957422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:47.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:46 smithi144 ceph-mon[35650]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:47.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:46 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2828957422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:47.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:46 smithi193 ceph-mon[36001]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:23:47.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:46 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2828957422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 -- 172.21.15.5:0/968133372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed81034c0 msgr2=0x7f3ed81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 --2- 172.21.15.5:0/968133372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed81034c0 0x7f3ed81038d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0099c0 tx=0x7f3ecc009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 -- 172.21.15.5:0/968133372 shutdown_connections 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 --2- 172.21.15.5:0/968133372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed8108740 0x7f3ed81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 --2- 172.21.15.5:0/968133372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed81034c0 0x7f3ed81038d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 --2- 172.21.15.5:0/968133372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8107df0 0x7f3ed8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 -- 172.21.15.5:0/968133372 >> 172.21.15.5:0/968133372 conn(0x7f3ed80fedc0 msgr2=0x7f3ed81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 -- 172.21.15.5:0/968133372 shutdown_connections 2024-05-14T15:23:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.570+0000 7f3edf198700 1 -- 172.21.15.5:0/968133372 wait complete. 2024-05-14T15:23:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 Processor -- start 2024-05-14T15:23:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 -- start start 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed81034c0 0x7f3ed819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 0x7f3ed819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8108740 0x7f3ed81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed81133c0 con 0x7f3ed8107df0 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8113240 con 0x7f3ed8108740 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edf198700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed8113540 con 0x7f3ed81034c0 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 0x7f3ed819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 0x7f3ed819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47420/0 (socket says 172.21.15.5:47420) 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 -- 172.21.15.5:0/4223269943 learned_addr learned my addr 172.21.15.5:0/4223269943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edcf34700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed81034c0 0x7f3ed819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edd735700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8108740 0x7f3ed81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 -- 172.21.15.5:0/4223269943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed81034c0 msgr2=0x7f3ed819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed81034c0 0x7f3ed819d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 -- 172.21.15.5:0/4223269943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8108740 msgr2=0x7f3ed81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8108740 0x7f3ed81a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3ed7fff700 1 -- 172.21.15.5:0/4223269943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed81a1f10 con 0x7f3ed8107df0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.571+0000 7f3edd735700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8108740 0x7f3ed81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.572+0000 7f3ed7fff700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 0x7f3ed819d5e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d980 tx=0x7f3ec400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.572+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4017070 con 0x7f3ed8107df0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.572+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec4009de0 con 0x7f3ed8107df0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.572+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed81a2200 con 0x7f3ed8107df0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.572+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400f3c0 con 0x7f3ed8107df0 2024-05-14T15:23:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.572+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a9b20 con 0x7f3ed8107df0 2024-05-14T15:23:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.573+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed80512c0 con 0x7f3ed8107df0 2024-05-14T15:23:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.574+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3ec4009930 con 0x7f3ed8107df0 2024-05-14T15:23:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.575+0000 7f3ed5ffb700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec80b2880 0x7f3ec80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.575+0000 7f3edcf34700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec80b2880 0x7f3ec80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.575+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f3ec40e2130 con 0x7f3ed8107df0 2024-05-14T15:23:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.576+0000 7f3edcf34700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec80b2880 0x7f3ec80b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b580 tx=0x7f3ecc0036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.580+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ec40a9a80 con 0x7f3ed8107df0 2024-05-14T15:23:48.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:48 smithi005 ceph-mon[31559]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:23:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.842+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed802e5f0 con 0x7f3ed8107df0 2024-05-14T15:23:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.843+0000 7f3ed5ffb700 1 -- 172.21.15.5:0/4223269943 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11864 (secure 0 0 0) 0x7f3ec40a9380 con 0x7f3ed8107df0 2024-05-14T15:23:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:48.844 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:22:00.688182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24487,24515]},"id":1}]} 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec80b2880 msgr2=0x7f3ec80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec80b2880 0x7f3ec80b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b580 tx=0x7f3ecc0036e0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 msgr2=0x7f3ed819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 0x7f3ed819d5e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400d980 tx=0x7f3ec400dc90 comp rx=0 tx=0).stop 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 shutdown_connections 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3ec80b2880 0x7f3ec80b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ed8108740 0x7f3ed81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed81034c0 0x7f3ed819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 --2- 172.21.15.5:0/4223269943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed8107df0 0x7f3ed819d5e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 >> 172.21.15.5:0/4223269943 conn(0x7f3ed80fedc0 msgr2=0x7f3ed8106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 shutdown_connections 2024-05-14T15:23:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:48.845+0000 7f3edf198700 1 -- 172.21.15.5:0/4223269943 wait complete. 2024-05-14T15:23:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:23:49.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:48 smithi144 ceph-mon[35650]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:23:49.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:48 smithi193 ceph-mon[36001]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:23:49.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:23:49.882 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:50.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:49 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4223269943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:50.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:49 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4223269943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:50.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:49 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4223269943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:50 smithi005 ceph-mon[31559]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:23:51.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:50 smithi144 ceph-mon[35650]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:23:51.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:50 smithi193 ceph-mon[36001]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:23:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4146511954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8108740 msgr2=0x7f37d81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4146511954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8108740 0x7f37d81100a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f37d00099c0 tx=0x7f37d0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4146511954 shutdown_connections 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4146511954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8108740 0x7f37d81100a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4146511954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d81034c0 0x7f37d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4146511954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37d8107df0 0x7f37d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4146511954 >> 172.21.15.5:0/4146511954 conn(0x7f37d80fedf0 msgr2=0x7f37d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4146511954 shutdown_connections 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4146511954 wait complete. 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.582+0000 7f37dd6a7700 1 Processor -- start 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 -- start start 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37d81034c0 0x7f37d819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 0x7f37d819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d8108740 0x7f37d819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81133c0 con 0x7f37d8107df0 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8113240 con 0x7f37d8108740 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37dd6a7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8113540 con 0x7f37d81034c0 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 0x7f37d819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 0x7f37d819c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47446/0 (socket says 172.21.15.5:47446) 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 -- 172.21.15.5:0/4220461445 learned_addr learned my addr 172.21.15.5:0/4220461445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d6ffd700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37d81034c0 0x7f37d819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d77fe700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d8108740 0x7f37d819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 -- 172.21.15.5:0/4220461445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37d81034c0 msgr2=0x7f37d819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37d81034c0 0x7f37d819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 -- 172.21.15.5:0/4220461445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d8108740 msgr2=0x7f37d819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d8108740 0x7f37d819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d67fc700 1 -- 172.21.15.5:0/4220461445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d81a6ee0 con 0x7f37d8107df0 2024-05-14T15:23:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.583+0000 7f37d77fe700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d8108740 0x7f37d819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.584+0000 7f37d67fc700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 0x7f37d819c140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d980 tx=0x7f37cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:51.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.584+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc017070 con 0x7f37d8107df0 2024-05-14T15:23:51.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.584+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d81a71d0 con 0x7f37d8107df0 2024-05-14T15:23:51.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.584+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37cc009de0 con 0x7f37d8107df0 2024-05-14T15:23:51.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.584+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00f3c0 con 0x7f37d8107df0 2024-05-14T15:23:51.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.584+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a7710 con 0x7f37d8107df0 2024-05-14T15:23:51.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.585+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d80512c0 con 0x7f37d8107df0 2024-05-14T15:23:51.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.587+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f37cc009930 con 0x7f37d8107df0 2024-05-14T15:23:51.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.587+0000 7f37bffff700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37c00b2880 0x7f37c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:51.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.588+0000 7f37d6ffd700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37c00b2880 0x7f37c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:51.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.588+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f37cc0e2020 con 0x7f37d8107df0 2024-05-14T15:23:51.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.588+0000 7f37d6ffd700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37c00b2880 0x7f37c00b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f37c800f440 tx=0x7f37c80079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.591+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37cc0a9970 con 0x7f37d8107df0 2024-05-14T15:23:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.805+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37d802e5f0 con 0x7f37d8107df0 2024-05-14T15:23:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.807+0000 7f37bffff700 1 -- 172.21.15.5:0/4220461445 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24761 (secure 0 0 0) 0x7f37cc0a9270 con 0x7f37d8107df0 2024-05-14T15:23:51.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:51.808 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:22:42.885032+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:23:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.809+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37c00b2880 msgr2=0x7f37c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.809+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37c00b2880 0x7f37c00b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f37c800f440 tx=0x7f37c80079e0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 msgr2=0x7f37d819c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 0x7f37d819c140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d980 tx=0x7f37cc00dc90 comp rx=0 tx=0).stop 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 shutdown_connections 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f37c00b2880 0x7f37c00b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d8108740 0x7f37d819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37d81034c0 0x7f37d819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 --2- 172.21.15.5:0/4220461445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8107df0 0x7f37d819c140 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 >> 172.21.15.5:0/4220461445 conn(0x7f37d80fedf0 msgr2=0x7f37d8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 shutdown_connections 2024-05-14T15:23:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:51.810+0000 7f37dd6a7700 1 -- 172.21.15.5:0/4220461445 wait complete. 2024-05-14T15:23:52.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:23:52.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:52 smithi005 ceph-mon[31559]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T15:23:52.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:52 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4220461445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:23:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:52 smithi144 ceph-mon[35650]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T15:23:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:52 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4220461445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:23:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:52 smithi193 ceph-mon[36001]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-05-14T15:23:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:52 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4220461445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:23:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3781902661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac107e30 msgr2=0x7fbcac108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3781902661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac107e30 0x7fbcac108240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc940099c0 tx=0x7fbc94009cd0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3781902661 shutdown_connections 2024-05-14T15:23:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3781902661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcac108780 0x7fbcac1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3781902661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcac103550 0x7fbcac103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3781902661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac107e30 0x7fbcac108240 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3781902661 >> 172.21.15.5:0/3781902661 conn(0x7fbcac078e00 msgr2=0x7fbcac079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3781902661 shutdown_connections 2024-05-14T15:23:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.573+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3781902661 wait complete. 2024-05-14T15:23:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 Processor -- start 2024-05-14T15:23:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 -- start start 2024-05-14T15:23:54.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcac103550 0x7fbcac19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcac107e30 0x7fbcac19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 0x7fbcac1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1144c0 con 0x7fbcac108780 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac114340 con 0x7fbcac107e30 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcb14d1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac114640 con 0x7fbcac103550 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 0x7fbcac1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 0x7fbcac1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47472/0 (socket says 172.21.15.5:47472) 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcab7fe700 1 -- 172.21.15.5:0/3404778142 learned_addr learned my addr 172.21.15.5:0/3404778142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.574+0000 7fbcaaffd700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcac103550 0x7fbcac19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcaa7fc700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcac107e30 0x7fbcac19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcab7fe700 1 -- 172.21.15.5:0/3404778142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcac103550 msgr2=0x7fbcac19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcab7fe700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcac103550 0x7fbcac19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcab7fe700 1 -- 172.21.15.5:0/3404778142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcac107e30 msgr2=0x7fbcac19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcab7fe700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcac107e30 0x7fbcac19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcab7fe700 1 -- 172.21.15.5:0/3404778142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac1a1e10 con 0x7fbcac108780 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcab7fe700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 0x7fbcac1a1820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d980 tx=0x7fbca400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4017070 con 0x7fbcac108780 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcac1a2100 con 0x7fbcac108780 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4005020 con 0x7fbcac108780 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40099b0 con 0x7fbcac108780 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.575+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcac1a2610 con 0x7fbcac108780 2024-05-14T15:23:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.576+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcac0512c0 con 0x7fbcac108780 2024-05-14T15:23:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.579+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbca4009b70 con 0x7fbcac108780 2024-05-14T15:23:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.579+0000 7fbca3fff700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc980b2880 0x7fbc980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.580+0000 7fbcaaffd700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc980b2880 0x7fbc980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.580+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fbca40e2880 con 0x7fbcac108780 2024-05-14T15:23:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.580+0000 7fbcaaffd700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc980b2880 0x7fbc980b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f440 tx=0x7fbc9c0079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.583+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbca4020020 con 0x7fbcac108780 2024-05-14T15:23:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.846+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcac1a9b30 con 0x7fbcac108780 2024-05-14T15:23:54.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:54 smithi005 ceph-mon[31559]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:23:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.847+0000 7fbca3fff700 1 -- 172.21.15.5:0/3404778142 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11864 (secure 0 0 0) 0x7fbca40aa1d0 con 0x7fbcac108780 2024-05-14T15:23:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:23:54.848 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:22:00.688182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24487,24515]},"id":1}]} 2024-05-14T15:23:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.849+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc980b2880 msgr2=0x7fbc980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.849+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc980b2880 0x7fbc980b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f440 tx=0x7fbc9c0079e0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 msgr2=0x7fbcac1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 0x7fbcac1a1820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d980 tx=0x7fbca400dc90 comp rx=0 tx=0).stop 2024-05-14T15:23:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 shutdown_connections 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fbc980b2880 0x7fbc980b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcac108780 0x7fbcac1a1820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcac103550 0x7fbcac19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 --2- 172.21.15.5:0/3404778142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcac107e30 0x7fbcac19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 >> 172.21.15.5:0/3404778142 conn(0x7fbcac078e00 msgr2=0x7fbcac10e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 shutdown_connections 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:54.850+0000 7fbcb14d1700 1 -- 172.21.15.5:0/3404778142 wait complete. 2024-05-14T15:23:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:23:55.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:54 smithi144 ceph-mon[35650]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:23:55.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:54 smithi193 ceph-mon[36001]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T15:23:55.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T15:20:57.810761+0000', 'modified': '2024-05-14T15:22:00.688182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24515, 'mds_1': 24491, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/4153907810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 4153907810}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 4153907810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6837/1910760961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1910760961}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1910760961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3352720235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3352720235}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3352720235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.193:6839/570860802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 570860802}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 570860802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.144:6837/214149479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 214149479}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 214149479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6839/4183436774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 4183436774}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 4183436774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24491, 24487, 24515]} 2024-05-14T15:23:55.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T15:23:55.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T15:23:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:56.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:55 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3404778142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:55 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3404778142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:55 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3404778142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:23:57.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:56 smithi144 ceph-mon[35650]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T15:23:57.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:56 smithi193 ceph-mon[36001]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T15:23:57.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:56 smithi005 ceph-mon[31559]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T15:23:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 -- 172.21.15.5:0/1751747687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185010e8e0 msgr2=0x7f1850110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 --2- 172.21.15.5:0/1751747687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185010e8e0 0x7f1850110dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f184800b0a0 tx=0x7f184800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 -- 172.21.15.5:0/1751747687 shutdown_connections 2024-05-14T15:23:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 --2- 172.21.15.5:0/1751747687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185010e8e0 0x7f1850110dc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 --2- 172.21.15.5:0/1751747687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850105500 0x7f1850105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 --2- 172.21.15.5:0/1751747687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1850105e50 0x7f185010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 -- 172.21.15.5:0/1751747687 >> 172.21.15.5:0/1751747687 conn(0x7f18500fedc0 msgr2=0x7f18501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.729+0000 7f1856ccd700 1 -- 172.21.15.5:0/1751747687 shutdown_connections 2024-05-14T15:23:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.730+0000 7f1856ccd700 1 -- 172.21.15.5:0/1751747687 wait complete. 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.730+0000 7f1856ccd700 1 Processor -- start 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 -- start start 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850105500 0x7f185019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 0x7f185019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185010e8e0 0x7f185019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18501144e0 con 0x7f1850105e50 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1850114360 con 0x7f185010e8e0 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1856ccd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1850114660 con 0x7f1850105500 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 0x7f185019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 0x7f185019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36520/0 (socket says 172.21.15.5:36520) 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f184ffff700 1 -- 172.21.15.5:0/371994107 learned_addr learned my addr 172.21.15.5:0/371994107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f1854a69700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850105500 0x7f185019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.731+0000 7f185526a700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185010e8e0 0x7f185019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184ffff700 1 -- 172.21.15.5:0/371994107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850105500 msgr2=0x7f185019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184ffff700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850105500 0x7f185019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184ffff700 1 -- 172.21.15.5:0/371994107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185010e8e0 msgr2=0x7f185019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184ffff700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185010e8e0 0x7f185019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184ffff700 1 -- 172.21.15.5:0/371994107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185019fca0 con 0x7f1850105e50 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184ffff700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 0x7f185019b370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f184400b530 tx=0x7f184400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184dffb700 1 -- 172.21.15.5:0/371994107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844015020 con 0x7f1850105e50 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184dffb700 1 -- 172.21.15.5:0/371994107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1844011ce0 con 0x7f1850105e50 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f184dffb700 1 -- 172.21.15.5:0/371994107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844010dd0 con 0x7f1850105e50 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185019ff90 con 0x7f1850105e50 2024-05-14T15:23:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.732+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1850103120 con 0x7f1850105e50 2024-05-14T15:23:57.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.735+0000 7f184dffb700 1 -- 172.21.15.5:0/371994107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1844018020 con 0x7f1850105e50 2024-05-14T15:23:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.736+0000 7f184dffb700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f18380b2880 0x7f18380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.736+0000 7f1854a69700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f18380b2880 0x7f18380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.736+0000 7f184dffb700 1 -- 172.21.15.5:0/371994107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f18440e1ce0 con 0x7f1850105e50 2024-05-14T15:23:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.736+0000 7f18656e1b80 1 Processor -- start 2024-05-14T15:23:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.736+0000 7f18656e1b80 1 -- start start 2024-05-14T15:23:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.736+0000 7f1854a69700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f18380b2880 0x7f18380b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1840005950 tx=0x7f184000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f18656e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564df12c9810 con 0x564df13a7c10 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f18656e1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564df1268770 con 0x564df1229df0 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f18656e1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564df12c57d0 con 0x564df13ac420 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36524/0 (socket says 172.21.15.5:36524) 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 -- 172.21.15.5:0/263154541 learned_addr learned my addr 172.21.15.5:0/263154541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f185526a700 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f1854a69700 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 -- 172.21.15.5:0/263154541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 msgr2=0x564df13ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 -- 172.21.15.5:0/263154541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 msgr2=0x564df13a76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13a76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 -- 172.21.15.5:0/263154541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df1216880 con 0x564df13a7c10 2024-05-14T15:23:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.737+0000 7f184ffff700 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13abee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f18440a6160 tx=0x7f18440a8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd0fbfc4f5cd5ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.738+0000 7f1836ffd700 1 -- 172.21.15.5:0/263154541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440a95f0 con 0x564df13a7c10 2024-05-14T15:23:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.738+0000 7f1836ffd700 1 -- 172.21.15.5:0/263154541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440a8ef0 con 0x564df13a7c10 2024-05-14T15:23:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.738+0000 7f1836ffd700 1 -- 172.21.15.5:0/263154541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440a8150 con 0x564df13a7c10 2024-05-14T15:23:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.738+0000 7f18656e1b80 1 -- 172.21.15.5:0/263154541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 msgr2=0x564df13abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.738+0000 7f18656e1b80 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13abee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f18440a6160 tx=0x7f18440a8930 comp rx=0 tx=0).stop 2024-05-14T15:23:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 -- 172.21.15.5:0/263154541 shutdown_connections 2024-05-14T15:23:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 --2- 172.21.15.5:0/263154541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13abee0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 -- 172.21.15.5:0/263154541 >> 172.21.15.5:0/263154541 conn(0x564df12715b0 msgr2=0x564df13a2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 -- 172.21.15.5:0/263154541 shutdown_connections 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 -- 172.21.15.5:0/263154541 wait complete. 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 -- start start 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.739+0000 7f18656e1b80 1 Processor -- start 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564df13b4370 con 0x564df13a7c10 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564df122fbe0 con 0x564df1229df0 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564df13b44f0 con 0x564df13ac420 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f18656e1b80 10 client.0 _ll_register_callbacks cb 0x564df12618d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36530/0 (socket says 172.21.15.5:36530) 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 -- 172.21.15.5:0/490013771 learned_addr learned my addr 172.21.15.5:0/490013771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f1854a69700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f185526a700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 -- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 msgr2=0x564df13b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 -- 172.21.15.5:0/490013771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 msgr2=0x564df13b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 -- 172.21.15.5:0/490013771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df1265640 con 0x564df13a7c10 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f184ffff700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b1760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f18440a4f70 tx=0x7f18440106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2962792eda7ac256 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.740+0000 7f182ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f182ffff700 1 -- 172.21.15.5:0/490013771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184406e020 con 0x564df13a7c10 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f182ffff700 1 -- 172.21.15.5:0/490013771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f1844011830 con 0x564df13a7c10 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18440a8ef0 con 0x564df13a7c10 2024-05-14T15:23:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f182ffff700 1 -- 172.21.15.5:0/490013771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f18440a95f0 con 0x564df13a7c10 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 10 client.15066 fetch_fsmap learned FSMap version 25 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564df13b4370 con 0x564df13a7c10 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f182ffff700 1 -- 172.21.15.5:0/490013771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440a8ef0 con 0x564df13a7c10 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f182ffff700 1 -- 172.21.15.5:0/490013771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f18440a4940 con 0x564df13a7c10 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 20 client.15066 populate_metadata read hostname 'smithi005' 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df13c11a0 0x564df13c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 4 client.15066 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.741+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564df12d71f0 con 0x564df13c11a0 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.742+0000 7f1854a69700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df13c11a0 0x564df13c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.742+0000 7f182ffff700 10 client.15066 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.742+0000 7f1854a69700 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df13c11a0 0x564df13c2610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36ef7d2d6e49334d server_cookie=b3f70d8b715b129d in_seq=0 out_seq=0 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f182ffff700 1 -- 172.21.15.5:0/490013771 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564df12d71f0 con 0x564df13c11a0 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f182ffff700 10 client.15066 handle_command_reply: tid=0 2024-05-14T15:23:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 client.15066 shutdown 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 msgr2=0x564df13b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b1760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f18440a4f70 tx=0x7f18440106d0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 shutdown_connections 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df13c11a0 0x564df13c2610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df13b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 --2- 172.21.15.5:0/490013771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b1760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 >> 172.21.15.5:0/490013771 conn(0x564df12715b0 msgr2=0x564df13a6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.743+0000 7f182ffff700 0 client.15066 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.744+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 shutdown_connections 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.744+0000 7f18656e1b80 1 -- 172.21.15.5:0/490013771 wait complete. 2024-05-14T15:23:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.744+0000 7f18656e1b80 20 client.15066 trim_cache size 0 max 16384 2024-05-14T15:23:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 Processor -- start 2024-05-14T15:23:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 -- start start 2024-05-14T15:23:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df139c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564df13c0950 con 0x564df13a7c10 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564df13c1020 con 0x564df1229df0 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f18656e1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564df13c16f0 con 0x564df13ac420 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f184ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36538/0 (socket says 172.21.15.5:36538) 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f184ffff700 1 -- 172.21.15.5:0/1311786100 learned_addr learned my addr 172.21.15.5:0/1311786100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f1854a69700 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df139c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.765+0000 7f185526a700 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f184ffff700 1 -- 172.21.15.5:0/1311786100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 msgr2=0x564df13bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f184ffff700 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f184ffff700 1 -- 172.21.15.5:0/1311786100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 msgr2=0x564df139c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f184ffff700 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df139c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f184ffff700 1 -- 172.21.15.5:0/1311786100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df139cde0 con 0x564df13a7c10 2024-05-14T15:23:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f1854a69700 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df139c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f184ffff700 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b0b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18440a6160 tx=0x7f1844004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0514f55d31d5708 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f18367fc700 1 -- 172.21.15.5:0/1311786100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184406e070 con 0x564df13a7c10 2024-05-14T15:23:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f18367fc700 1 -- 172.21.15.5:0/1311786100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440a8ef0 con 0x564df13a7c10 2024-05-14T15:23:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.766+0000 7f18367fc700 1 -- 172.21.15.5:0/1311786100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440a4ac0 con 0x564df13a7c10 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 -- 172.21.15.5:0/1311786100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 msgr2=0x564df13b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b0b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f18440a6160 tx=0x7f1844004860 comp rx=0 tx=0).stop 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 -- 172.21.15.5:0/1311786100 shutdown_connections 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df13ac420 0x564df13bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df1229df0 0x564df139c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 --2- 172.21.15.5:0/1311786100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13a7c10 0x564df13b0b20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 -- 172.21.15.5:0/1311786100 >> 172.21.15.5:0/1311786100 conn(0x564df12715b0 msgr2=0x564df139c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 -- 172.21.15.5:0/1311786100 shutdown_connections 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 -- 172.21.15.5:0/1311786100 wait complete. 2024-05-14T15:23:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 -- start start 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.767+0000 7f18656e1b80 1 Processor -- start 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df1229df0 0x564df13c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df13a7c10 0x564df13c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 0x564df14245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564df13c2490 con 0x564df13ac420 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564df13c1dc0 con 0x564df13a7c10 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564df13c2b60 con 0x564df1229df0 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f18656e1b80 10 client.0 _ll_register_callbacks cb 0x564df12618d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 0x564df14245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 0x564df14245d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36540/0 (socket says 172.21.15.5:36540) 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 -- 172.21.15.5:0/874855297 learned_addr learned my addr 172.21.15.5:0/874855297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f1854a69700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df1229df0 0x564df13c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f184ffff700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df13a7c10 0x564df13c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 -- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df1229df0 msgr2=0x564df13c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df1229df0 0x564df13c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 -- 172.21.15.5:0/874855297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df13a7c10 msgr2=0x564df13c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df13a7c10 0x564df13c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 -- 172.21.15.5:0/874855297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df1424bc0 con 0x564df13ac420 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f184ffff700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df13a7c10 0x564df13c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f185526a700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 0x564df14245d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f184800b0a0 tx=0x7f1848003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf757375f20b93b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.768+0000 7f182ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f182ffff700 1 -- 172.21.15.5:0/874855297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184801cad0 con 0x564df13ac420 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f182ffff700 1 -- 172.21.15.5:0/874855297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f184801cc70 con 0x564df13ac420 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18440a8ef0 con 0x564df13ac420 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f182ffff700 1 -- 172.21.15.5:0/874855297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1848030710 con 0x564df13ac420 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 10 client.15072 fetch_fsmap learned FSMap version 25 2024-05-14T15:23:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564df142bec0 con 0x564df13ac420 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f182ffff700 1 -- 172.21.15.5:0/874855297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440a8ef0 con 0x564df13ac420 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f182ffff700 1 -- 172.21.15.5:0/874855297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1848011070 con 0x564df13ac420 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 20 client.15072 populate_metadata read hostname 'smithi005' 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df142e470 0x564df142f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 4 client.15072 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.769+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "status"}) -- 0x564df12d1290 con 0x564df142e470 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.770+0000 7f1854a69700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df142e470 0x564df142f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.770+0000 7f182ffff700 10 client.15072 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.770+0000 7f1854a69700 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df142e470 0x564df142f8e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cd1b9d4c6b82632 server_cookie=1316f109c1fa2020 in_seq=0 out_seq=0 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.770+0000 7f182ffff700 1 -- 172.21.15.5:0/874855297 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564df12d1290 con 0x564df142e470 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.770+0000 7f182ffff700 10 client.15072 handle_command_reply: tid=0 2024-05-14T15:23:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.770+0000 7f18656e1b80 1 client.15072 shutdown 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 msgr2=0x564df14245d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 0x564df14245d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f184800b0a0 tx=0x7f1848003e60 comp rx=0 tx=0).stop 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 shutdown_connections 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df13ac420 0x564df14245d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564df1229df0 0x564df13c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x564df142e470 0x564df142f8e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 --2- 172.21.15.5:0/874855297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564df13a7c10 0x564df13c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 >> 172.21.15.5:0/874855297 conn(0x564df12715b0 msgr2=0x564df13a1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f182ffff700 0 client.15072 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 shutdown_connections 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 1 -- 172.21.15.5:0/874855297 wait complete. 2024-05-14T15:23:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.771+0000 7f18656e1b80 20 client.15072 trim_cache size 0 max 16384 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24487, 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 122.01040574699999, 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-14T15:23:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T15:23:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 207.880779574 2024-05-14T15:23:57.778 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:23:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f18380b2880 msgr2=0x7f18380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f18380b2880 0x7f18380b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1840005950 tx=0x7f184000a300 comp rx=0 tx=0).stop 2024-05-14T15:23:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 msgr2=0x7f185019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 0x7f185019b370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f184400b530 tx=0x7f184400b840 comp rx=0 tx=0).stop 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 shutdown_connections 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f18380b2880 0x7f18380b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185010e8e0 0x7f185019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1850105500 0x7f185019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 --2- 172.21.15.5:0/371994107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1850105e50 0x7f185019b370 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 >> 172.21.15.5:0/371994107 conn(0x7f18500fedc0 msgr2=0x7f18500ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 shutdown_connections 2024-05-14T15:23:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:23:57.772+0000 7f1856ccd700 1 -- 172.21.15.5:0/371994107 wait complete. 2024-05-14T15:23:58.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T15:23:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:23:58.979 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:23:58 smithi005 ceph-mon[31559]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:23:59.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:23:58 smithi144 ceph-mon[35650]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:23:59.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:23:58 smithi193 ceph-mon[36001]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T15:24:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 -- 172.21.15.5:0/42812437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 msgr2=0x7f6ba81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/42812437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba81096e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009990 tx=0x7f6b98009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 -- 172.21.15.5:0/42812437 shutdown_connections 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/42812437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba8104330 0x7f6ba8110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/42812437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba81096e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/42812437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba81039e0 0x7f6ba8103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 -- 172.21.15.5:0/42812437 >> 172.21.15.5:0/42812437 conn(0x7f6ba80fe540 msgr2=0x7f6ba8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 -- 172.21.15.5:0/42812437 shutdown_connections 2024-05-14T15:24:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.626+0000 7f6bad5e8700 1 -- 172.21.15.5:0/42812437 wait complete. 2024-05-14T15:24:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 Processor -- start 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 -- start start 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba81039e0 0x7f6ba819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba8104330 0x7f6ba819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81133c0 con 0x7f6ba81092d0 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8113240 con 0x7f6ba81039e0 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6bad5e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8113540 con 0x7f6ba8104330 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6ba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6ba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36550/0 (socket says 172.21.15.5:36550) 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6ba77fe700 1 -- 172.21.15.5:0/3216359794 learned_addr learned my addr 172.21.15.5:0/3216359794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba81039e0 0x7f6ba819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.627+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba8104330 0x7f6ba819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba77fe700 1 -- 172.21.15.5:0/3216359794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba8104330 msgr2=0x7f6ba819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba8104330 0x7f6ba819c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba77fe700 1 -- 172.21.15.5:0/3216359794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba81039e0 msgr2=0x7f6ba819bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba81039e0 0x7f6ba819bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba77fe700 1 -- 172.21.15.5:0/3216359794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba81a7000 con 0x7f6ba81092d0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba819c760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0007ee0 tx=0x7f6ba000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba4ff9700 1 -- 172.21.15.5:0/3216359794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000dbd0 con 0x7f6ba81092d0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba4ff9700 1 -- 172.21.15.5:0/3216359794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba000dd70 con 0x7f6ba81092d0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81a72f0 con 0x7f6ba81092d0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6ba4ff9700 1 -- 172.21.15.5:0/3216359794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005990 con 0x7f6ba81092d0 2024-05-14T15:24:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.628+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a77e0 con 0x7f6ba81092d0 2024-05-14T15:24:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.631+0000 7f6ba4ff9700 1 -- 172.21.15.5:0/3216359794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6ba001a020 con 0x7f6ba81092d0 2024-05-14T15:24:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6ba4ff9700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6b940b2880 0x7f6b940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 Processor -- start 2024-05-14T15:24:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 -- start start 2024-05-14T15:24:00.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6ba4ff9700 1 -- 172.21.15.5:0/3216359794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f6ba00e1da0 con 0x7f6ba81092d0 2024-05-14T15:24:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6b940b2880 0x7f6b940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab2430e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24308880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab243091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab2422a8f0 con 0x55ab2418adf0 2024-05-14T15:24:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab2418fd50 con 0x55ab24308dc0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.632+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab242268b0 con 0x55ab2430ed30 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab2430e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab2430e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36552/0 (socket says 172.21.15.5:36552) 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/4060715587 learned_addr learned my addr 172.21.15.5:0/4060715587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6b940b2880 0x7f6b940b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800b2e0 tx=0x7f6b98005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24308880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab243091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/4060715587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 msgr2=0x55ab24308880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24308880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/4060715587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 msgr2=0x55ab243091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab243091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/4060715587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab24177880 con 0x55ab2418adf0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.633+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab2430e7f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6b98015000 tx=0x7f6b9801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119df248eaa3b7d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:00.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.634+0000 7f6b9d7fa700 1 -- 172.21.15.5:0/4060715587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98003680 con 0x55ab2418adf0 2024-05-14T15:24:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.634+0000 7f6b9d7fa700 1 -- 172.21.15.5:0/4060715587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b98003cd0 con 0x55ab2418adf0 2024-05-14T15:24:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.634+0000 7f6b9d7fa700 1 -- 172.21.15.5:0/4060715587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800fbf0 con 0x55ab2418adf0 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.634+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/4060715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 msgr2=0x55ab2430e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab2430e7f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6b98015000 tx=0x7f6b9801e000 comp rx=0 tx=0).stop 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/4060715587 shutdown_connections 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab243091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab2430e7f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/4060715587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24308880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/4060715587 >> 172.21.15.5:0/4060715587 conn(0x55ab241d25e0 msgr2=0x55ab243039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/4060715587 shutdown_connections 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/4060715587 wait complete. 2024-05-14T15:24:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 -- start start 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.635+0000 7f6bbbffcb80 1 Processor -- start 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2418adf0 0x55ab2430f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 0x55ab2430f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab2430ed30 0x55ab2430fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab243156b0 con 0x55ab24308dc0 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab241e3e80 con 0x55ab2430ed30 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab24315830 con 0x55ab2418adf0 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6bbbffcb80 10 client.0 _ll_register_callbacks cb 0x55ab241c2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 0x55ab2430f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 0x55ab2430f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36560/0 (socket says 172.21.15.5:36560) 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 -- 172.21.15.5:0/2460736962 learned_addr learned my addr 172.21.15.5:0/2460736962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2418adf0 0x55ab2430f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 -- 172.21.15.5:0/2460736962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2418adf0 msgr2=0x55ab2430f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab2430ed30 0x55ab2430fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2418adf0 0x55ab2430f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 -- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab2430ed30 msgr2=0x55ab2430fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab2430ed30 0x55ab2430fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 -- 172.21.15.5:0/2460736962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab241e1710 con 0x55ab24308dc0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab2430ed30 0x55ab2430fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 0x55ab2430f800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b90012700 tx=0x7f6b90012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9147664b883575b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.636+0000 7f6b8e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/2460736962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90004320 con 0x55ab24308dc0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/2460736962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f6b900044c0 con 0x55ab24308dc0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b98003cd0 con 0x55ab24308dc0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/2460736962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b90025050 con 0x55ab24308dc0 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 10 client.15084 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:00.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab24328c90 con 0x55ab24308dc0 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/2460736962 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b90003b30 con 0x55ab24308dc0 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/2460736962 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b900119f0 con 0x55ab24308dc0 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 20 client.15084 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab2432c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 4 client.15084 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.637+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6b98003cd0 con 0x55ab2432b340 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.638+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab2432c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.638+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab2432c7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f4c5eee60a840ab server_cookie=a7b456d6c29396b7 in_seq=0 out_seq=0 2024-05-14T15:24:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.638+0000 7f6b8e7fc700 10 client.15084 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/2460736962 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6b98003cd0 con 0x55ab2432b340 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6b8e7fc700 10 client.15084 handle_command_reply: tid=0 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 client.15084 shutdown 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 msgr2=0x55ab2430f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 0x55ab2430f800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b90012700 tx=0x7f6b90012a10 comp rx=0 tx=0).stop 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 shutdown_connections 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab2432c7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6b8e7fc700 0 client.15084 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab2430ed30 0x55ab2430fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2418adf0 0x55ab2430f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/2460736962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab24308dc0 0x55ab2430f800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.639+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 >> 172.21.15.5:0/2460736962 conn(0x55ab241d25e0 msgr2=0x55ab24307870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.640+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 shutdown_connections 2024-05-14T15:24:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.640+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/2460736962 wait complete. 2024-05-14T15:24:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.640+0000 7f6bbbffcb80 20 client.15084 trim_cache size 0 max 16384 2024-05-14T15:24:00.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 Processor -- start 2024-05-14T15:24:00.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 -- start start 2024-05-14T15:24:00.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24308880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab2430b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab2432aaf0 con 0x55ab2418adf0 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab2432b1c0 con 0x55ab24308dc0 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab2432b890 con 0x55ab2430ed30 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6ba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24308880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6ba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24308880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36574/0 (socket says 172.21.15.5:36574) 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/3072539563 learned_addr learned my addr 172.21.15.5:0/3072539563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab2430b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/3072539563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 msgr2=0x55ab24335c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.662+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/3072539563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 msgr2=0x55ab2430b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab2430b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/3072539563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab243046d0 con 0x55ab2418adf0 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab2430b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24308880 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800bce0 tx=0x7f6b98009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fb369ea794b3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6b9cff9700 1 -- 172.21.15.5:0/3072539563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800f840 con 0x55ab2418adf0 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6b9cff9700 1 -- 172.21.15.5:0/3072539563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9803d070 con 0x55ab2418adf0 2024-05-14T15:24:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.663+0000 7f6b9cff9700 1 -- 172.21.15.5:0/3072539563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800fc00 con 0x55ab2418adf0 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/3072539563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 msgr2=0x55ab24308880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24308880 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800bce0 tx=0x7f6b98009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/3072539563 shutdown_connections 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24308880 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/3072539563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab2430b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/3072539563 >> 172.21.15.5:0/3072539563 conn(0x55ab241d25e0 msgr2=0x55ab2430bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/3072539563 shutdown_connections 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/3072539563 wait complete. 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.664+0000 7f6bbbffcb80 1 -- start start 2024-05-14T15:24:00.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 Processor -- start 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24327ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab24335510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab2432c630 con 0x55ab2418adf0 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab2432bf60 con 0x55ab24308dc0 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab2432cd00 con 0x55ab2430ed30 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6bbbffcb80 10 client.0 _ll_register_callbacks cb 0x55ab241c2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24327ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24327ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36582/0 (socket says 172.21.15.5:36582) 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/1936060389 learned_addr learned my addr 172.21.15.5:0/1936060389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba77fe700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab24335510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/1936060389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 msgr2=0x55ab24335a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 msgr2=0x55ab24335510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab24335510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.665+0000 7f6ba6ffd700 1 -- 172.21.15.5:0/1936060389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab24336130 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6ba6ffd700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24327ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098e0 tx=0x7f6b980092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e784a981d2d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6b8e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/1936060389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98046060 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9803d070 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/1936060389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f6b9802b040 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/1936060389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b9800fa80 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6bbbffcb80 10 client.15090 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab24396590 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/1936060389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9803d070 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/1936060389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b98037ce0 con 0x55ab2418adf0 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.666+0000 7f6bbbffcb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:00.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6bbbffcb80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6bbbffcb80 20 client.15090 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab24399820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6bbbffcb80 4 client.15090 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "status"}) -- 0x55ab242382d0 con 0x55ab2432b340 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab24399820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6b9e7fc700 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab24399820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=593b180808626c70 server_cookie=5721d4a37f9881bc in_seq=0 out_seq=0 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.667+0000 7f6b8e7fc700 10 client.15090 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6b8e7fc700 1 -- 172.21.15.5:0/1936060389 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ab242382d0 con 0x55ab2432b340 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6b8e7fc700 10 client.15090 handle_command_reply: tid=0 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 client.15090 shutdown 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 msgr2=0x55ab24327ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24327ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098e0 tx=0x7f6b980092d0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 shutdown_connections 2024-05-14T15:24:00.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55ab2432b340 0x55ab24399820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab2430ed30 0x55ab24335a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab2418adf0 0x55ab24327ad0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 --2- 172.21.15.5:0/1936060389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab24308dc0 0x55ab24335510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 >> 172.21.15.5:0/1936060389 conn(0x55ab241d25e0 msgr2=0x55ab2430cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6b8e7fc700 0 client.15090 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 shutdown_connections 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 1 -- 172.21.15.5:0/1936060389 wait complete. 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.668+0000 7f6bbbffcb80 20 client.15090 trim_cache size 0 max 16384 2024-05-14T15:24:00.673 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24491, 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 125.916085927, 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 213.565560667 2024-05-14T15:24:00.674 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6b940b2880 msgr2=0x7f6b940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6b940b2880 0x7f6b940b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800b2e0 tx=0x7f6b98005fd0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 msgr2=0x7f6ba819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba819c760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0007ee0 tx=0x7f6ba000d3b0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 shutdown_connections 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f6b940b2880 0x7f6b940b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba81092d0 0x7f6ba819c760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ba81039e0 0x7f6ba819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 --2- 172.21.15.5:0/3216359794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba8104330 0x7f6ba819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 >> 172.21.15.5:0/3216359794 conn(0x7f6ba80fe540 msgr2=0x7f6ba8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 shutdown_connections 2024-05-14T15:24:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:00.669+0000 7f6bad5e8700 1 -- 172.21.15.5:0/3216359794 wait complete. 2024-05-14T15:24:00.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:00 smithi005 ceph-mon[31559]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:01.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:00 smithi144 ceph-mon[35650]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:01.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:00 smithi193 ceph-mon[36001]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:01.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T15:24:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:03.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:02 smithi005 ceph-mon[31559]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:03.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:02 smithi144 ceph-mon[35650]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:03.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:02 smithi193 ceph-mon[36001]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/1791983265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc4105ec0 msgr2=0x7f8fc410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/1791983265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc4105ec0 0x7f8fc410e410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009990 tx=0x7f8fb4009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/1791983265 shutdown_connections 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/1791983265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc410e950 0x7f8fc4110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/1791983265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4105570 0x7f8fc4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/1791983265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc4105ec0 0x7f8fc410e410 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/1791983265 >> 172.21.15.5:0/1791983265 conn(0x7f8fc4078e00 msgr2=0x7f8fc4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/1791983265 shutdown_connections 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.448+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/1791983265 wait complete. 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.449+0000 7f8fcc3ac700 1 Processor -- start 2024-05-14T15:24:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.449+0000 7f8fcc3ac700 1 -- start start 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.449+0000 7f8fcc3ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4105570 0x7f8fc419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.449+0000 7f8fcc3ac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4105ec0 0x7f8fc419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.449+0000 7f8fcc3ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 0x7f8fc419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fcc3ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41144a0 con 0x7f8fc410e950 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fcc3ac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4114320 con 0x7f8fc4105570 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fcc3ac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4114620 con 0x7f8fc4105ec0 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 0x7f8fc419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 0x7f8fc419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36602/0 (socket says 172.21.15.5:36602) 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 -- 172.21.15.5:0/2113881497 learned_addr learned my addr 172.21.15.5:0/2113881497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca148700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4105570 0x7f8fc419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fc9947700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4105ec0 0x7f8fc419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 -- 172.21.15.5:0/2113881497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4105ec0 msgr2=0x7f8fc419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4105ec0 0x7f8fc419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 -- 172.21.15.5:0/2113881497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4105570 msgr2=0x7f8fc419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4105570 0x7f8fc419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 -- 172.21.15.5:0/2113881497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc419fd40 con 0x7f8fc410e950 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fca949700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 0x7f8fc419f6a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc007ee0 tx=0x7f8fbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fbb7fe700 1 -- 172.21.15.5:0/2113881497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00ddd0 con 0x7f8fc410e950 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fbb7fe700 1 -- 172.21.15.5:0/2113881497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc004020 con 0x7f8fc410e950 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fbb7fe700 1 -- 172.21.15.5:0/2113881497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc015660 con 0x7f8fc410e950 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.450+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41a0030 con 0x7f8fc410e950 2024-05-14T15:24:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.451+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a7950 con 0x7f8fc410e950 2024-05-14T15:24:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.454+0000 7f8fbb7fe700 1 -- 172.21.15.5:0/2113881497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8fbc0041d0 con 0x7f8fc410e950 2024-05-14T15:24:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.455+0000 7f8fbb7fe700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8fb00b2880 0x7f8fb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.455+0000 7f8fdadc0b80 1 Processor -- start 2024-05-14T15:24:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.455+0000 7f8fca148700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8fb00b2880 0x7f8fb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.455+0000 7f8fbb7fe700 1 -- 172.21.15.5:0/2113881497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f8fbc0e1ff0 con 0x7f8fc410e950 2024-05-14T15:24:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.455+0000 7f8fdadc0b80 1 -- start start 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8fb00b2880 0x7f8fb00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0004630 tx=0x7f8fc000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff171c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff1716880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff17171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dff16388f0 con 0x55dff1598df0 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dff159dd50 con 0x55dff1716dc0 2024-05-14T15:24:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dff16348b0 con 0x55dff171cd30 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff171c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff171c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36608/0 (socket says 172.21.15.5:36608) 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 -- 172.21.15.5:0/3110397479 learned_addr learned my addr 172.21.15.5:0/3110397479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fc9947700 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff1716880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca949700 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff17171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 -- 172.21.15.5:0/3110397479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 msgr2=0x55dff1716880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff1716880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 -- 172.21.15.5:0/3110397479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 msgr2=0x55dff17171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff17171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 -- 172.21.15.5:0/3110397479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff1585880 con 0x55dff1598df0 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca949700 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff17171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fca148700 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff171c7f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0003b10 tx=0x7f8fc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f9345401d3852d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.456+0000 7f8fb8d47700 1 -- 172.21.15.5:0/3110397479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00118d0 con 0x55dff1598df0 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.457+0000 7f8fb8d47700 1 -- 172.21.15.5:0/3110397479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc001e420 con 0x55dff1598df0 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.457+0000 7f8fb8d47700 1 -- 172.21.15.5:0/3110397479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001c610 con 0x55dff1598df0 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.457+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/3110397479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 msgr2=0x55dff171c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.457+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff171c7f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0003b10 tx=0x7f8fc0013000 comp rx=0 tx=0).stop 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/3110397479 shutdown_connections 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff17171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff171c7f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/3110397479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff1716880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/3110397479 >> 172.21.15.5:0/3110397479 conn(0x55dff15e05e0 msgr2=0x55dff17119b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/3110397479 shutdown_connections 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/3110397479 wait complete. 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 -- start start 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.458+0000 7f8fdadc0b80 1 Processor -- start 2024-05-14T15:24:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff1720490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1716dc0 0x55dff17209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff171cd30 0x55dff1720f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dff17237d0 con 0x55dff1598df0 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dff15f1e80 con 0x55dff171cd30 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dff1723950 con 0x55dff1716dc0 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fdadc0b80 10 client.0 _ll_register_callbacks cb 0x55dff15d0900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff1720490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff1720490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36614/0 (socket says 172.21.15.5:36614) 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 -- 172.21.15.5:0/4214985454 learned_addr learned my addr 172.21.15.5:0/4214985454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fc9947700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1716dc0 0x55dff17209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca949700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff171cd30 0x55dff1720f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 -- 172.21.15.5:0/4214985454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1716dc0 msgr2=0x55dff17209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1716dc0 0x55dff17209d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 -- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff171cd30 msgr2=0x55dff1720f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff171cd30 0x55dff1720f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 -- 172.21.15.5:0/4214985454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff15ef710 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fca148700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff1720490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8fc001eda0 tx=0x7f8fc001e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8c7f3f0fbcc46b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fadffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.459+0000 7f8fadffb700 1 -- 172.21.15.5:0/4214985454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0035060 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fadffb700 1 -- 172.21.15.5:0/4214985454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f8fc001ee00 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc001e420 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fadffb700 1 -- 172.21.15.5:0/4214985454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8fc003c050 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dff172e310 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fadffb700 1 -- 172.21.15.5:0/4214985454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc001e420 con 0x55dff1598df0 2024-05-14T15:24:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fadffb700 1 -- 172.21.15.5:0/4214985454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8fc0030a90 con 0x55dff1598df0 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 20 client.15102 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff1730560 0x55dff17319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 4 client.15102 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.460+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff16462d0 con 0x55dff1730560 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.461+0000 7f8fc9947700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff1730560 0x55dff17319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.461+0000 7f8fc9947700 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff1730560 0x55dff17319d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1750b7478b1eeabc server_cookie=8a1006fb1c36ceb7 in_seq=0 out_seq=0 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.461+0000 7f8fadffb700 10 client.15102 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fadffb700 1 -- 172.21.15.5:0/4214985454 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dff16462d0 con 0x55dff1730560 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fadffb700 10 client.15102 handle_command_reply: tid=0 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 client.15102 shutdown 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 msgr2=0x55dff1720490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff1720490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8fc001eda0 tx=0x7f8fc001e680 comp rx=0 tx=0).stop 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 shutdown_connections 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff1730560 0x55dff17319d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff171cd30 0x55dff1720f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1598df0 0x55dff1720490 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/4214985454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1716dc0 0x55dff17209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 >> 172.21.15.5:0/4214985454 conn(0x55dff15e05e0 msgr2=0x55dff1715870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fadffb700 0 client.15102 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 shutdown_connections 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/4214985454 wait complete. 2024-05-14T15:24:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.462+0000 7f8fdadc0b80 20 client.15102 trim_cache size 0 max 16384 2024-05-14T15:24:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.483+0000 7f8fdadc0b80 1 Processor -- start 2024-05-14T15:24:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.483+0000 7f8fdadc0b80 1 -- start start 2024-05-14T15:24:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1598df0 0x55dff1716880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff172cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 0x55dff1720d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dff172fd10 con 0x55dff171cd30 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dff17303e0 con 0x55dff1716dc0 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dff1730ab0 con 0x55dff1598df0 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 0x55dff1720d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 0x55dff1720d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36624/0 (socket says 172.21.15.5:36624) 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 -- 172.21.15.5:0/1611111892 learned_addr learned my addr 172.21.15.5:0/1611111892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca148700 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1598df0 0x55dff1716880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 -- 172.21.15.5:0/1611111892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1598df0 msgr2=0x55dff1716880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1598df0 0x55dff1716880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 -- 172.21.15.5:0/1611111892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 msgr2=0x55dff172cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff172cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 -- 172.21.15.5:0/1611111892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff1712930 con 0x55dff171cd30 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca148700 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1598df0 0x55dff1716880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8fca949700 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 0x55dff1720d50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00d7b0 tx=0x7f8fbc0a8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821e337511120fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8faffff700 1 -- 172.21.15.5:0/1611111892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a9240 con 0x55dff171cd30 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8faffff700 1 -- 172.21.15.5:0/1611111892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc0ad5f0 con 0x55dff171cd30 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.484+0000 7f8faffff700 1 -- 172.21.15.5:0/1611111892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a9700 con 0x55dff171cd30 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1611111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 msgr2=0x55dff1720d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 0x55dff1720d50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00d7b0 tx=0x7f8fbc0a8a80 comp rx=0 tx=0).stop 2024-05-14T15:24:03.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1611111892 shutdown_connections 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff171cd30 0x55dff1720d50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff1598df0 0x55dff1716880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1611111892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1716dc0 0x55dff172cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1611111892 >> 172.21.15.5:0/1611111892 conn(0x55dff15e05e0 msgr2=0x55dff171ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1611111892 shutdown_connections 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.485+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1611111892 wait complete. 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 -- start start 2024-05-14T15:24:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 Processor -- start 2024-05-14T15:24:03.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1598df0 0x55dff173b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 0x55dff172bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff172c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dff1731850 con 0x55dff1716dc0 2024-05-14T15:24:03.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dff1731180 con 0x55dff1598df0 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dff1731f20 con 0x55dff171cd30 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fdadc0b80 10 client.0 _ll_register_callbacks cb 0x55dff15d0900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fc9947700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 0x55dff172bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fc9947700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 0x55dff172bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36634/0 (socket says 172.21.15.5:36634) 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fc9947700 1 -- 172.21.15.5:0/1579901426 learned_addr learned my addr 172.21.15.5:0/1579901426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fca148700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1598df0 0x55dff173b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.486+0000 7f8fca949700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff172c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fc9947700 1 -- 172.21.15.5:0/1579901426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 msgr2=0x55dff172c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fc9947700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff172c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fc9947700 1 -- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1598df0 msgr2=0x55dff173b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fc9947700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1598df0 0x55dff173b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fc9947700 1 -- 172.21.15.5:0/1579901426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff172ca00 con 0x55dff1716dc0 2024-05-14T15:24:03.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fc9947700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 0x55dff172bbe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0f2b3ab3848033 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fadffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fadffb700 1 -- 172.21.15.5:0/1579901426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003d00 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fadffb700 1 -- 172.21.15.5:0/1579901426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f8fb40050d0 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc0ad5f0 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fadffb700 1 -- 172.21.15.5:0/1579901426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8fb4036050 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fdadc0b80 10 client.15108 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dff179b9d0 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.487+0000 7f8fadffb700 1 -- 172.21.15.5:0/1579901426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc0ad5f0 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fadffb700 1 -- 172.21.15.5:0/1579901426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8fb40291d0 con 0x55dff1716dc0 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fdadc0b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:03.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fdadc0b80 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fdadc0b80 20 client.15108 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff179de80 0x55dff179f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fdadc0b80 4 client.15108 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "status"}) -- 0x55dff1640370 con 0x55dff179de80 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fca148700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff179de80 0x55dff179f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fca148700 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff179de80 0x55dff179f2f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fb4816428654dbb server_cookie=e4888e4c704e042b in_seq=0 out_seq=0 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.488+0000 7f8fadffb700 10 client.15108 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:03.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fadffb700 1 -- 172.21.15.5:0/1579901426 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dff1640370 con 0x55dff179de80 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fadffb700 10 client.15108 handle_command_reply: tid=0 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 client.15108 shutdown 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 msgr2=0x55dff172bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 0x55dff172bbe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb4009f70 comp rx=0 tx=0).stop 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 shutdown_connections 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dff179de80 0x55dff179f2f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dff171cd30 0x55dff172c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dff1598df0 0x55dff173b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 --2- 172.21.15.5:0/1579901426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dff1716dc0 0x55dff172bbe0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fadffb700 0 client.15108 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 >> 172.21.15.5:0/1579901426 conn(0x55dff15e05e0 msgr2=0x55dff1711220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:03.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 shutdown_connections 2024-05-14T15:24:03.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 1 -- 172.21.15.5:0/1579901426 wait complete. 2024-05-14T15:24:03.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fdadc0b80 20 client.15108 trim_cache size 0 max 16384 2024-05-14T15:24:03.498 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:24:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:24:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-14T15:24:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24515, 2024-05-14T15:24:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:24:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:24:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:24:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 185.57127662799999, 2024-05-14T15:24:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-05-14T15:24:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-14T15:24:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T15:24:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 203.122280571 2024-05-14T15:24:03.500 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:24:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8fb00b2880 msgr2=0x7f8fb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8fb00b2880 0x7f8fb00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0004630 tx=0x7f8fc000b3c0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 msgr2=0x7f8fc419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.489+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 0x7f8fc419f6a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc007ee0 tx=0x7f8fbc00d3b0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 shutdown_connections 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8fb00b2880 0x7f8fb00b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fc410e950 0x7f8fc419f6a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fc4105570 0x7f8fc419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 --2- 172.21.15.5:0/2113881497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fc4105ec0 0x7f8fc419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 >> 172.21.15.5:0/2113881497 conn(0x7f8fc4078e00 msgr2=0x7f8fc41000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 shutdown_connections 2024-05-14T15:24:03.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:03.490+0000 7f8fcc3ac700 1 -- 172.21.15.5:0/2113881497 wait complete. 2024-05-14T15:24:04.229 INFO:tasks.cephadm:Setup complete, yielding 2024-05-14T15:24:04.229 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T15:24:04.240 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:24:04.240 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-14T15:24:04.492 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:04.870 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:04 smithi005 ceph-mon[31559]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:05.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:04 smithi144 ceph-mon[35650]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:05.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:04 smithi193 ceph-mon[36001]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 -- 172.21.15.5:0/1856005943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c06c2b0 msgr2=0x7f129c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 --2- 172.21.15.5:0/1856005943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c06c2b0 0x7f129c106ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f129800b0a0 tx=0x7f129800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 -- 172.21.15.5:0/1856005943 shutdown_connections 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 --2- 172.21.15.5:0/1856005943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c109ff0 0x7f129c10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 --2- 172.21.15.5:0/1856005943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c06c2b0 0x7f129c106ed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 --2- 172.21.15.5:0/1856005943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f129c1074a0 0x7f129c109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 -- 172.21.15.5:0/1856005943 >> 172.21.15.5:0/1856005943 conn(0x7f129c0fe880 msgr2=0x7f129c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:06.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.208+0000 7f12a4103700 1 -- 172.21.15.5:0/1856005943 shutdown_connections 2024-05-14T15:24:06.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.209+0000 7f12a4103700 1 -- 172.21.15.5:0/1856005943 wait complete. 2024-05-14T15:24:06.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.209+0000 7f12a4103700 1 Processor -- start 2024-05-14T15:24:06.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 -- start start 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c06c2b0 0x7f129c104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f129c1074a0 0x7f129c102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 0x7f129c103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f129c10fd70 con 0x7f129c109ff0 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f129c10fbf0 con 0x7f129c1074a0 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a4103700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f129c10fef0 con 0x7f129c06c2b0 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 0x7f129c103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 0x7f129c103310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45340/0 (socket says 172.21.15.5:45340) 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a1e9f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c06c2b0 0x7f129c104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:06.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 -- 172.21.15.5:0/1719718132 learned_addr learned my addr 172.21.15.5:0/1719718132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a169e700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f129c1074a0 0x7f129c102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 -- 172.21.15.5:0/1719718132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c06c2b0 msgr2=0x7f129c104720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c06c2b0 0x7f129c104720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 -- 172.21.15.5:0/1719718132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f129c1074a0 msgr2=0x7f129c102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f129c1074a0 0x7f129c102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a26a0700 1 -- 172.21.15.5:0/1719718132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c103a70 con 0x7f129c109ff0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.210+0000 7f12a1e9f700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c06c2b0 0x7f129c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.211+0000 7f12a26a0700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 0x7f129c103310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f129400d980 tx=0x7f129400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.211+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294017070 con 0x7f129c109ff0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.211+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1294005020 con 0x7f129c109ff0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.211+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c1a49e0 con 0x7f129c109ff0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.211+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129400fa10 con 0x7f129c109ff0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.211+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c1a4ec0 con 0x7f129c109ff0 2024-05-14T15:24:06.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.212+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c0512c0 con 0x7f129c109ff0 2024-05-14T15:24:06.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.214+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1294004a30 con 0x7f129c109ff0 2024-05-14T15:24:06.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.214+0000 7f1292ffd700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12880b2900 0x7f12880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:06.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.214+0000 7f12a1e9f700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12880b2900 0x7f12880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:06.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.214+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f12940e27b0 con 0x7f129c109ff0 2024-05-14T15:24:06.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.215+0000 7f12a1e9f700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12880b2900 0x7f12880b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1298009900 tx=0x7f129800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:06.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.218+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1294020020 con 0x7f129c109ff0 2024-05-14T15:24:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.449+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f129c063c00 con 0x7f12880b2900 2024-05-14T15:24:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.451+0000 7f1292ffd700 1 -- 172.21.15.5:0/1719718132 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f129c063c00 con 0x7f12880b2900 2024-05-14T15:24:06.452 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-05-14T15:24:06.452 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-05-14T15:24:06.453 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12880b2900 msgr2=0x7f12880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12880b2900 0x7f12880b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1298009900 tx=0x7f129800b7b0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 msgr2=0x7f129c103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 0x7f129c103310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f129400d980 tx=0x7f129400dc90 comp rx=0 tx=0).stop 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 shutdown_connections 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f12880b2900 0x7f12880b4e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f129c109ff0 0x7f129c103310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f129c06c2b0 0x7f129c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 --2- 172.21.15.5:0/1719718132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f129c1074a0 0x7f129c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 >> 172.21.15.5:0/1719718132 conn(0x7f129c0fe880 msgr2=0x7f129c1056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 shutdown_connections 2024-05-14T15:24:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:06.453+0000 7f12a4103700 1 -- 172.21.15.5:0/1719718132 wait complete. 2024-05-14T15:24:07.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:06 smithi144 ceph-mon[35650]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:07.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:06 smithi193 ceph-mon[36001]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:07.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:06 smithi005 ceph-mon[31559]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:07.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-14T15:24:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:07.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:07 smithi005 ceph-mon[31559]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:08.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:07 smithi144 ceph-mon[35650]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:08.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:07 smithi193 ceph-mon[36001]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:08.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:08 smithi005 ceph-mon[31559]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:09.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:08 smithi144 ceph-mon[35650]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:09.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:08 smithi193 ceph-mon[36001]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 -- 172.21.15.5:0/4106007549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c10a100 msgr2=0x7fda6c10c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 --2- 172.21.15.5:0/4106007549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c10a100 0x7fda6c10c520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda64009990 tx=0x7fda64009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 -- 172.21.15.5:0/4106007549 shutdown_connections 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 --2- 172.21.15.5:0/4106007549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c10a100 0x7fda6c10c520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 --2- 172.21.15.5:0/4106007549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c104e40 0x7fda6c107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 --2- 172.21.15.5:0/4106007549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c1077a0 0x7fda6c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 -- 172.21.15.5:0/4106007549 >> 172.21.15.5:0/4106007549 conn(0x7fda6c0fe880 msgr2=0x7fda6c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.159+0000 7fda712cc700 1 -- 172.21.15.5:0/4106007549 shutdown_connections 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.160+0000 7fda712cc700 1 -- 172.21.15.5:0/4106007549 wait complete. 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.160+0000 7fda712cc700 1 Processor -- start 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.160+0000 7fda712cc700 1 -- start start 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda712cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c104e40 0x7fda6c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda712cc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 0x7fda6c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda712cc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c10a100 0x7fda6c1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda712cc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10fda0 con 0x7fda6c1077a0 2024-05-14T15:24:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda712cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10fc20 con 0x7fda6c104e40 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda712cc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10ff20 con 0x7fda6c10a100 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 0x7fda6c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 0x7fda6c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45356/0 (socket says 172.21.15.5:45356) 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 -- 172.21.15.5:0/1424434181 learned_addr learned my addr 172.21.15.5:0/1424434181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6b7fe700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c10a100 0x7fda6c1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6affd700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c104e40 0x7fda6c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 -- 172.21.15.5:0/1424434181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c10a100 msgr2=0x7fda6c1a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c10a100 0x7fda6c1a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 -- 172.21.15.5:0/1424434181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c104e40 msgr2=0x7fda6c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c104e40 0x7fda6c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 -- 172.21.15.5:0/1424434181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1a2360 con 0x7fda6c1077a0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6b7fe700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c10a100 0x7fda6c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.161+0000 7fda6a7fc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 0x7fda6c19add0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c9b0 tx=0x7fda6000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.162+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60007b00 con 0x7fda6c1077a0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.162+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda60007ca0 con 0x7fda6c1077a0 2024-05-14T15:24:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.162+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c1a2650 con 0x7fda6c1077a0 2024-05-14T15:24:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.162+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60018730 con 0x7fda6c1077a0 2024-05-14T15:24:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.162+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c1a2b90 con 0x7fda6c1077a0 2024-05-14T15:24:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.163+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c0512c0 con 0x7fda6c1077a0 2024-05-14T15:24:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.165+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fda6000f450 con 0x7fda6c1077a0 2024-05-14T15:24:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.166+0000 7fda53fff700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fda540b2880 0x7fda540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.166+0000 7fda6affd700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fda540b2880 0x7fda540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.166+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fda600e29b0 con 0x7fda6c1077a0 2024-05-14T15:24:09.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.167+0000 7fda6affd700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fda540b2880 0x7fda540b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0060b0 tx=0x7fda5c007b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.170+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda60014030 con 0x7fda6c1077a0 2024-05-14T15:24:09.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.394+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fda6c063c00 con 0x7fda540b2880 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.407+0000 7fda53fff700 1 -- 172.21.15.5:0/1424434181 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fda6c063c00 con 0x7fda540b2880 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (3m) 34s ago 3m 84.5M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 29d56efefc52 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi144 running (3m) 74s ago 3m 93.0M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 099fd1712df5 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi193 running (3m) 74s ago 3m 89.4M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 9e98e3d9fb01 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (3m) 34s ago 3m 82.2M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 5536fdb611a4 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi144 running (3m) 74s ago 3m 88.6M - 19.0.0-3588-ge2b755a6 bc14c6192b1a e8a99d2c1340 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi193 running (3m) 74s ago 3m 83.9M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 2edd9b4dfe16 2024-05-14T15:24:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (3m) 34s ago 3m 84.7M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 95767b1fd925 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi144 running (3m) 74s ago 3m 85.4M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 4b34cf157ad6 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi193 running (3m) 74s ago 3m 86.0M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 037ab062289e 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (3m) 34s ago 3m 81.1M - 19.0.0-3588-ge2b755a6 bc14c6192b1a e865c0ba7472 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi144 running (4m) 74s ago 4m 81.4M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 9cb95246a8f0 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi193 running (4m) 74s ago 4m 82.2M - 19.0.0-3588-ge2b755a6 bc14c6192b1a c93296aea431 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (11m) 34s ago 11m 598M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 4921cde61604 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi144 *:8443,8765 running (9m) 74s ago 9m 534M - 19.0.0-3588-ge2b755a6 bc14c6192b1a 03589df7a821 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi193 *:8443,8765 running (9m) 74s ago 9m 537M - 19.0.0-3588-ge2b755a6 bc14c6192b1a a288382abd54 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (11m) 34s ago 11m 120M 2048M 19.0.0-3588-ge2b755a6 bc14c6192b1a 1757122c7e39 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi144 running (9m) 74s ago 9m 108M 2048M 19.0.0-3588-ge2b755a6 bc14c6192b1a baa0403d2691 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi193 running (9m) 74s ago 9m 109M 2048M 19.0.0-3588-ge2b755a6 bc14c6192b1a 7f993fef78a0 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (8m) 34s ago 8m 126M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 6d21e4de0452 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi144 running (8m) 74s ago 8m 127M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a 2adf885dae46 2024-05-14T15:24:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi193 running (8m) 74s ago 8m 115M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a 54d65e28ca4d 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (7m) 34s ago 7m 122M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a f9c261300a33 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi144 running (7m) 74s ago 7m 111M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a 86e2a6259ff7 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi193 running (6m) 74s ago 6m 124M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a b9deea07315f 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (6m) 34s ago 6m 110M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 7c9cce180cd6 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi144 running (6m) 74s ago 6m 124M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a c444e109f298 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi193 running (5m) 74s ago 5m 110M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a 9be3e15fe630 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (5m) 34s ago 5m 122M 4273M 19.0.0-3588-ge2b755a6 bc14c6192b1a 29c2fe55651c 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi144 running (4m) 74s ago 4m 110M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a b983405ec52c 2024-05-14T15:24:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi193 running (4m) 74s ago 4m 123M 4272M 19.0.0-3588-ge2b755a6 bc14c6192b1a 9bb683fee1a5 2024-05-14T15:24:09.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fda540b2880 msgr2=0x7fda540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:09.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fda540b2880 0x7fda540b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0060b0 tx=0x7fda5c007b80 comp rx=0 tx=0).stop 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 msgr2=0x7fda6c19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 0x7fda6c19add0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda6000c9b0 tx=0x7fda6000cd70 comp rx=0 tx=0).stop 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 shutdown_connections 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fda540b2880 0x7fda540b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda6c10a100 0x7fda6c1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c104e40 0x7fda6c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 --2- 172.21.15.5:0/1424434181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda6c1077a0 0x7fda6c19add0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 >> 172.21.15.5:0/1424434181 conn(0x7fda6c0fe880 msgr2=0x7fda6c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 shutdown_connections 2024-05-14T15:24:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:09.411+0000 7fda712cc700 1 -- 172.21.15.5:0/1424434181 wait complete. 2024-05-14T15:24:10.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-14T15:24:10.365 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:10.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:10 smithi005 ceph-mon[31559]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:10.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:10 smithi005 ceph-mon[31559]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:11.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:10 smithi144 ceph-mon[35650]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:11.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:10 smithi144 ceph-mon[35650]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:11.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:10 smithi193 ceph-mon[36001]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:11.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:10 smithi193 ceph-mon[36001]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 -- 172.21.15.5:0/1002553489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f722810a000 msgr2=0x7f722810c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 --2- 172.21.15.5:0/1002553489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f722810a000 0x7f722810c510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7220009990 tx=0x7f7220009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 -- 172.21.15.5:0/1002553489 shutdown_connections 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 --2- 172.21.15.5:0/1002553489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f722810a000 0x7f722810c510 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 --2- 172.21.15.5:0/1002553489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280766b0 0x7f7228106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 --2- 172.21.15.5:0/1002553489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7228107490 0x7f7228109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 -- 172.21.15.5:0/1002553489 >> 172.21.15.5:0/1002553489 conn(0x7f72280fe890 msgr2=0x7f7228100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 -- 172.21.15.5:0/1002553489 shutdown_connections 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.037+0000 7f722cded700 1 -- 172.21.15.5:0/1002553489 wait complete. 2024-05-14T15:24:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.038+0000 7f722cded700 1 Processor -- start 2024-05-14T15:24:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.038+0000 7f722cded700 1 -- start start 2024-05-14T15:24:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722cded700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280766b0 0x7f7228104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722cded700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 0x7f7228102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722cded700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f722810a000 0x7f7228103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722cded700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f722810fde0 con 0x7f7228107490 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722cded700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f722810fc60 con 0x7f722810a000 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722cded700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f722810ff60 con 0x7f72280766b0 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 0x7f7228102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 0x7f7228102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45370/0 (socket says 172.21.15.5:45370) 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 -- 172.21.15.5:0/1063273261 learned_addr learned my addr 172.21.15.5:0/1063273261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f722659c700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280766b0 0x7f7228104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7226d9d700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f722810a000 0x7f7228103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 -- 172.21.15.5:0/1063273261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280766b0 msgr2=0x7f7228104710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280766b0 0x7f7228104710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 -- 172.21.15.5:0/1063273261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f722810a000 msgr2=0x7f7228103300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f722810a000 0x7f7228103300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 -- 172.21.15.5:0/1063273261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72281039d0 con 0x7f7228107490 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7226d9d700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f722810a000 0x7f7228103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.039+0000 7f7225d9b700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 0x7f7228102dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f721000c9b0 tx=0x7f721000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.040+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210007b00 con 0x7f7228107490 2024-05-14T15:24:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.040+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7210007ca0 con 0x7f7228107490 2024-05-14T15:24:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.040+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210018660 con 0x7f7228107490 2024-05-14T15:24:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.040+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72281a4980 con 0x7f7228107490 2024-05-14T15:24:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.040+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72281a4d70 con 0x7f7228107490 2024-05-14T15:24:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.041+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722802e5f0 con 0x7f7228107490 2024-05-14T15:24:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.043+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f721000f450 con 0x7f7228107490 2024-05-14T15:24:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.043+0000 7f721f7fe700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f72140b2880 0x7f72140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.043+0000 7f722659c700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f72140b2880 0x7f72140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.044+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f72100e2880 con 0x7f7228107490 2024-05-14T15:24:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.044+0000 7f722659c700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f72140b2880 0x7f72140b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f72180060b0 tx=0x7f721800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.047+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7210014030 con 0x7f7228107490 2024-05-14T15:24:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.274+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7228063c00 con 0x7f72140b2880 2024-05-14T15:24:12.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.280+0000 7f721f7fe700 1 -- 172.21.15.5:0/1063273261 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7228063c00 con 0x7f72140b2880 2024-05-14T15:24:12.281 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-14T15:24:12.282 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 76s ago 4m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-05-14T15:24:12.282 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 76s ago 9m smithi005=x;smithi144=y;smithi193=z;count:3 2024-05-14T15:24:12.282 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 76s ago 9m smithi005:172.21.15.5=a;smithi144:172.21.15.144=b;smithi193:172.21.15.193=c;count:3 2024-05-14T15:24:12.282 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 76s ago - 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f72140b2880 msgr2=0x7f72140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f72140b2880 0x7f72140b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f72180060b0 tx=0x7f721800c040 comp rx=0 tx=0).stop 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 msgr2=0x7f7228102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 0x7f7228102dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f721000c9b0 tx=0x7f721000ccc0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 shutdown_connections 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f72140b2880 0x7f72140b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f722810a000 0x7f7228103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280766b0 0x7f7228104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 --2- 172.21.15.5:0/1063273261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7228107490 0x7f7228102dc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 >> 172.21.15.5:0/1063273261 conn(0x7f72280fe890 msgr2=0x7f7228105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 shutdown_connections 2024-05-14T15:24:12.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:12.282+0000 7f722cded700 1 -- 172.21.15.5:0/1063273261 wait complete. 2024-05-14T15:24:12.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:12 smithi005 ceph-mon[31559]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:12.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-14T15:24:13.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:12 smithi144 ceph-mon[35650]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:13.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:12 smithi193 ceph-mon[36001]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:14.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:13 smithi005 ceph-mon[31559]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:14.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:13 smithi144 ceph-mon[35650]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:14.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:13 smithi193 ceph-mon[36001]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 -- 172.21.15.5:0/2309755508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981059b0 msgr2=0x7f3e98105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/2309755508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981059b0 0x7f3e98105dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009990 tx=0x7f3e8c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 -- 172.21.15.5:0/2309755508 shutdown_connections 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/2309755508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e981049e0 0x7f3e9810c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/2309755508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981059b0 0x7f3e98105dc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/2309755508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 0x7f3e98104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.915+0000 7f3e9fe75700 1 -- 172.21.15.5:0/2309755508 >> 172.21.15.5:0/2309755508 conn(0x7f3e980ff9d0 msgr2=0x7f3e98101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.916+0000 7f3e9fe75700 1 -- 172.21.15.5:0/2309755508 shutdown_connections 2024-05-14T15:24:14.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.916+0000 7f3e9fe75700 1 -- 172.21.15.5:0/2309755508 wait complete. 2024-05-14T15:24:14.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:14 smithi005 ceph-mon[31559]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 Processor -- start 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 -- start start 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 0x7f3e9819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 0x7f3e9819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e981059b0 0x7f3e981a2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9810ffa0 con 0x7f3e981049e0 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9810fe20 con 0x7f3e98103f60 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9fe75700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e98110120 con 0x7f3e981059b0 2024-05-14T15:24:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9d410700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 0x7f3e9819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9d410700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 0x7f3e9819b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45396/0 (socket says 172.21.15.5:45396) 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9d410700 1 -- 172.21.15.5:0/1365714138 learned_addr learned my addr 172.21.15.5:0/1365714138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9dc11700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 0x7f3e9819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.917+0000 7f3e9e412700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e981059b0 0x7f3e981a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9d410700 1 -- 172.21.15.5:0/1365714138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e981059b0 msgr2=0x7f3e981a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9d410700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e981059b0 0x7f3e981a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9d410700 1 -- 172.21.15.5:0/1365714138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 msgr2=0x7f3e9819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9d410700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 0x7f3e9819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9d410700 1 -- 172.21.15.5:0/1365714138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e981a2700 con 0x7f3e981049e0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9dc11700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 0x7f3e9819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9d410700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 0x7f3e9819b0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c920 tx=0x7f3e9400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94004a30 con 0x7f3e981049e0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e940054d0 con 0x7f3e981049e0 2024-05-14T15:24:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e981a29f0 con 0x7f3e981049e0 2024-05-14T15:24:14.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94007c70 con 0x7f3e981049e0 2024-05-14T15:24:14.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.918+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e981a2f30 con 0x7f3e981049e0 2024-05-14T15:24:14.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.919+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e980512c0 con 0x7f3e981049e0 2024-05-14T15:24:14.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.921+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3e94004c60 con 0x7f3e981049e0 2024-05-14T15:24:14.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.921+0000 7f3e8affd700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e840b2880 0x7f3e840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:14.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.922+0000 7f3e9dc11700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e840b2880 0x7f3e840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:14.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.922+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f3e940e2500 con 0x7f3e981049e0 2024-05-14T15:24:14.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.922+0000 7f3e9dc11700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e840b2880 0x7f3e840b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0038a0 tx=0x7f3e8c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:14.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:14.925+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e94014030 con 0x7f3e981049e0 2024-05-14T15:24:15.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:14 smithi144 ceph-mon[35650]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:15.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:14 smithi193 ceph-mon[36001]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.146+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3e980008d0 con 0x7f3e840b2880 2024-05-14T15:24:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.148+0000 7f3e8affd700 1 -- 172.21.15.5:0/1365714138 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3e980008d0 con 0x7f3e840b2880 2024-05-14T15:24:15.148 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-05-14T15:24:15.149 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-05-14T15:24:15.149 INFO:teuthology.orchestra.run.smithi005.stdout:smithi144 172.21.15.144 2024-05-14T15:24:15.149 INFO:teuthology.orchestra.run.smithi005.stdout:smithi193 172.21.15.193 2024-05-14T15:24:15.149 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-05-14T15:24:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e840b2880 msgr2=0x7f3e840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e840b2880 0x7f3e840b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0038a0 tx=0x7f3e8c01a040 comp rx=0 tx=0).stop 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 msgr2=0x7f3e9819b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 0x7f3e9819b0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c920 tx=0x7f3e9400cc30 comp rx=0 tx=0).stop 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 shutdown_connections 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f3e840b2880 0x7f3e840b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e981059b0 0x7f3e981a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e98103f60 0x7f3e9819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 --2- 172.21.15.5:0/1365714138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e981049e0 0x7f3e9819b0e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.150+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 >> 172.21.15.5:0/1365714138 conn(0x7f3e980ff9d0 msgr2=0x7f3e981086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.151+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 shutdown_connections 2024-05-14T15:24:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:15.151+0000 7f3e9fe75700 1 -- 172.21.15.5:0/1365714138 wait complete. 2024-05-14T15:24:15.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-14T15:24:16.122 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:16.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:16 smithi005 ceph-mon[31559]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:16.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:16 smithi005 ceph-mon[31559]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:17.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:16 smithi144 ceph-mon[35650]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:17.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:16 smithi144 ceph-mon[35650]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:17.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:16 smithi193 ceph-mon[36001]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:17.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:16 smithi193 ceph-mon[36001]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/4269774004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 msgr2=0x7f8bec103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/4269774004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec103c20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009990 tx=0x7f8bdc009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/4269774004 shutdown_connections 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/4269774004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 0x7f8bec10b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/4269774004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec103c20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/4269774004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bec1041f0 0x7f8bec104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/4269774004 >> 172.21.15.5:0/4269774004 conn(0x7f8bec0fe020 msgr2=0x7f8bec1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/4269774004 shutdown_connections 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.848+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/4269774004 wait complete. 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.849+0000 7f8bf1ad5700 1 Processor -- start 2024-05-14T15:24:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.849+0000 7f8bf1ad5700 1 -- start start 2024-05-14T15:24:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bf1ad5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bf1ad5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bec1041f0 0x7f8bec19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bf1ad5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 0x7f8bec1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bf1ad5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10ecb0 con 0x7f8bec103810 2024-05-14T15:24:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bf1ad5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10eb30 con 0x7f8bec104cf0 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bf1ad5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10ee30 con 0x7f8bec1041f0 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51988/0 (socket says 172.21.15.5:51988) 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 -- 172.21.15.5:0/1510024254 learned_addr learned my addr 172.21.15.5:0/1510024254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beaffd700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bec1041f0 0x7f8bec19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bebfff700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 0x7f8bec1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 -- 172.21.15.5:0/1510024254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bec1041f0 msgr2=0x7f8bec19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bec1041f0 0x7f8bec19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 -- 172.21.15.5:0/1510024254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 msgr2=0x7f8bec1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 0x7f8bec1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 -- 172.21.15.5:0/1510024254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec1a24b0 con 0x7f8bec103810 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8bebfff700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 0x7f8bec1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.850+0000 7f8beb7fe700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec102ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00b5c0 tx=0x7f8bdc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.851+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc004830 con 0x7f8bec103810 2024-05-14T15:24:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.851+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec1a2740 con 0x7f8bec103810 2024-05-14T15:24:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.851+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1a2b90 con 0x7f8bec103810 2024-05-14T15:24:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.851+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bdc003c00 con 0x7f8bec103810 2024-05-14T15:24:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.851+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc005280 con 0x7f8bec103810 2024-05-14T15:24:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.852+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bec0512c0 con 0x7f8bec103810 2024-05-14T15:24:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.853+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8bdc003fd0 con 0x7f8bec103810 2024-05-14T15:24:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.854+0000 7f8be8ff9700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bd80b2900 0x7f8bd80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.854+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f8bdc0e33b0 con 0x7f8bec103810 2024-05-14T15:24:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.854+0000 7f8beaffd700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bd80b2900 0x7f8bd80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.855+0000 7f8beaffd700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bd80b2900 0x7f8bd80b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bec102880 tx=0x7f8bd400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:17.859+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8bdc013070 con 0x7f8bec103810 2024-05-14T15:24:18.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.082+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8bec063c00 con 0x7f8bd80b2900 2024-05-14T15:24:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.084+0000 7f8be8ff9700 1 -- 172.21.15.5:0/1510024254 <== mgr.14150 v2:172.21.15.5:6800/887023568 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8bec063c00 con 0x7f8bd80b2900 2024-05-14T15:24:18.084 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-14T15:24:18.084 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-05-14T15:24:18.084 INFO:teuthology.orchestra.run.smithi005.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T15:24:18.084 INFO:teuthology.orchestra.run.smithi005.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bd80b2900 msgr2=0x7f8bd80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bd80b2900 0x7f8bd80b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bec102880 tx=0x7f8bd400b410 comp rx=0 tx=0).stop 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 msgr2=0x7f8bec102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec102ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00b5c0 tx=0x7f8bdc0047b0 comp rx=0 tx=0).stop 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 shutdown_connections 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f8bd80b2900 0x7f8bd80b4e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bec104cf0 0x7f8bec1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bec103810 0x7f8bec102ed0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 --2- 172.21.15.5:0/1510024254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bec1041f0 0x7f8bec19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 >> 172.21.15.5:0/1510024254 conn(0x7f8bec0fe020 msgr2=0x7f8bec1077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 shutdown_connections 2024-05-14T15:24:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:18.086+0000 7f8bf1ad5700 1 -- 172.21.15.5:0/1510024254 wait complete. 2024-05-14T15:24:18.917 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T15:24:18.926 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:24:18.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-14T15:24:19.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:18 smithi144 ceph-mon[35650]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:19.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:18 smithi193 ceph-mon[36001]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:19.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:18 smithi005 ceph-mon[31559]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:19.183 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:19.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:19 smithi005 ceph-mon[31559]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:19 smithi144 ceph-mon[35650]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:19 smithi193 ceph-mon[36001]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:20.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:20 smithi005 ceph-mon[31559]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 -- 172.21.15.5:0/822845351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 msgr2=0x7f4d7c077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 --2- 172.21.15.5:0/822845351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c077690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009990 tx=0x7f4d6c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 -- 172.21.15.5:0/822845351 shutdown_connections 2024-05-14T15:24:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 --2- 172.21.15.5:0/822845351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 0x7f4d7c10e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 --2- 172.21.15.5:0/822845351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d7c0787f0 0x7f4d7c076cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 --2- 172.21.15.5:0/822845351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c077690 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 -- 172.21.15.5:0/822845351 >> 172.21.15.5:0/822845351 conn(0x7f4d7c0ff9d0 msgr2=0x7f4d7c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 -- 172.21.15.5:0/822845351 shutdown_connections 2024-05-14T15:24:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.876+0000 7f4d83a57700 1 -- 172.21.15.5:0/822845351 wait complete. 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 Processor -- start 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 -- start start 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d7c0787f0 0x7f4d7c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 0x7f4d7c1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c111fd0 con 0x7f4d7c077200 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c111e50 con 0x7f4d7c0787f0 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d83a57700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c112150 con 0x7f4d7c10c220 2024-05-14T15:24:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d817f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d817f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52010/0 (socket says 172.21.15.5:52010) 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.877+0000 7f4d817f3700 1 -- 172.21.15.5:0/4253232450 learned_addr learned my addr 172.21.15.5:0/4253232450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d80ff2700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d7c0787f0 0x7f4d7c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d81ff4700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 0x7f4d7c1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d817f3700 1 -- 172.21.15.5:0/4253232450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 msgr2=0x7f4d7c1a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d817f3700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 0x7f4d7c1a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d817f3700 1 -- 172.21.15.5:0/4253232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d7c0787f0 msgr2=0x7f4d7c19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d817f3700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d7c0787f0 0x7f4d7c19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d817f3700 1 -- 172.21.15.5:0/4253232450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c1a4650 con 0x7f4d7c077200 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d81ff4700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 0x7f4d7c1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d817f3700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c19caf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800d980 tx=0x7f4d7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78017070 con 0x7f4d7c077200 2024-05-14T15:24:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d78009de0 con 0x7f4d7c077200 2024-05-14T15:24:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c1a4940 con 0x7f4d7c077200 2024-05-14T15:24:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800f3c0 con 0x7f4d7c077200 2024-05-14T15:24:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.878+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c1a4e80 con 0x7f4d7c077200 2024-05-14T15:24:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.879+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c0512c0 con 0x7f4d7c077200 2024-05-14T15:24:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.881+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4d78009930 con 0x7f4d7c077200 2024-05-14T15:24:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.881+0000 7f4d727fc700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4d680b2880 0x7f4d680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.882+0000 7f4d80ff2700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4d680b2880 0x7f4d680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.882+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f4d780e24c0 con 0x7f4d7c077200 2024-05-14T15:24:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.882+0000 7f4d80ff2700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4d680b2880 0x7f4d680b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c19dba0 tx=0x7f4d6c00b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:20.885+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d780a9dd0 con 0x7f4d7c077200 2024-05-14T15:24:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:20 smithi144 ceph-mon[35650]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:21.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:20 smithi193 ceph-mon[36001]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.145+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4d7c02e5f0 con 0x7f4d7c077200 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.145+0000 7f4d727fc700 1 -- 172.21.15.5:0/4253232450 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3046 (secure 0 0 0) 0x7f4d780a96d0 con 0x7f4d7c077200 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout:e25 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-05-14T15:24:21.146 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 25 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-14T15:20:57.810761+0000 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-05-14T15:22:00.688182+0000 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-05-14T15:24:21.147 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 3 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24515,1=24491,2=24487} 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3,4] 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-05-14T15:24:21.148 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24515 members: 24491,24487,24515 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24515} state up:active seq 6 addr [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{0:24464} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/3352720235,v1:172.21.15.144:6835/3352720235] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{1:24491} state up:active seq 23 addr [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{1:24463} state up:standby-replay seq 1 addr [v2:172.21.15.193:6836/1910760961,v1:172.21.15.193:6837/1910760961] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{2:24487} state up:active seq 23 addr [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{2:24457} state up:standby-replay seq 1 addr [v2:172.21.15.193:6834/4153907810,v1:172.21.15.193:6835/4153907810] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:21.149 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.5:6834/1133415705,v1:172.21.15.5:6835/1133415705] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.150 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.5:6836/3415932983,v1:172.21.15.5:6837/3415932983] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.150 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{-1:14598} state up:standby seq 1 addr [v2:172.21.15.5:6838/45851617,v1:172.21.15.5:6839/45851617] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.150 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.5:6840/1161784438,v1:172.21.15.5:6841/1161784438] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.150 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.193:6832/2582549673,v1:172.21.15.193:6833/2582549673] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.150 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.144:6832/3788502407,v1:172.21.15.144:6833/3788502407] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4d680b2880 msgr2=0x7f4d680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4d680b2880 0x7f4d680b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c19dba0 tx=0x7f4d6c00b720 comp rx=0 tx=0).stop 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 msgr2=0x7f4d7c19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c19caf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800d980 tx=0x7f4d7800dc90 comp rx=0 tx=0).stop 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 shutdown_connections 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4d680b2880 0x7f4d680b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d7c10c220 0x7f4d7c1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d7c077200 0x7f4d7c19caf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 --2- 172.21.15.5:0/4253232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d7c0787f0 0x7f4d7c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.147+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 >> 172.21.15.5:0/4253232450 conn(0x7f4d7c0ff9d0 msgr2=0x7f4d7c1081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.148+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 shutdown_connections 2024-05-14T15:24:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:21.148+0000 7f4d83a57700 1 -- 172.21.15.5:0/4253232450 wait complete. 2024-05-14T15:24:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:24:21.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:21 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4253232450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T15:24:21.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-14T15:24:22.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:21 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4253232450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T15:24:22.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:21 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4253232450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T15:24:22.083 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:22.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:22 smithi005 ceph-mon[31559]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:23.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:22 smithi144 ceph-mon[35650]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:23.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:22 smithi193 ceph-mon[36001]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 -- 172.21.15.5:0/71796444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68104370 msgr2=0x7f4e68104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/71796444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68104370 0x7f4e68104780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009990 tx=0x7f4e5c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 -- 172.21.15.5:0/71796444 shutdown_connections 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/71796444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 0x7f4e6810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/71796444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e68103240 0x7f4e68103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/71796444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68104370 0x7f4e68104780 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 -- 172.21.15.5:0/71796444 >> 172.21.15.5:0/71796444 conn(0x7f4e680fe880 msgr2=0x7f4e68100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 -- 172.21.15.5:0/71796444 shutdown_connections 2024-05-14T15:24:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.743+0000 7f4e6c80a700 1 -- 172.21.15.5:0/71796444 wait complete. 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 Processor -- start 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 -- start start 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 0x7f4e6819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e68104370 0x7f4e6819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 0x7f4e681a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810eca0 con 0x7f4e68103240 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810eb20 con 0x7f4e68104d10 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6c80a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810ee20 con 0x7f4e68104370 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 0x7f4e6819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 0x7f4e6819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52018/0 (socket says 172.21.15.5:52018) 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.744+0000 7f4e6659c700 1 -- 172.21.15.5:0/1995643863 learned_addr learned my addr 172.21.15.5:0/1995643863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e65d9b700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e68104370 0x7f4e6819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e66d9d700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 0x7f4e681a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6659c700 1 -- 172.21.15.5:0/1995643863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e68104370 msgr2=0x7f4e6819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6659c700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e68104370 0x7f4e6819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6659c700 1 -- 172.21.15.5:0/1995643863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 msgr2=0x7f4e681a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6659c700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 0x7f4e681a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6659c700 1 -- 172.21.15.5:0/1995643863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e681a2460 con 0x7f4e68103240 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e66d9d700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 0x7f4e681a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6659c700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 0x7f4e6819a900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800c970 tx=0x7f4e5800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800ce60 con 0x7f4e68103240 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e681a2750 con 0x7f4e68103240 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e58007900 con 0x7f4e68103240 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58003c70 con 0x7f4e68103240 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.745+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681a2c90 con 0x7f4e68103240 2024-05-14T15:24:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.746+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e680512c0 con 0x7f4e68103240 2024-05-14T15:24:23.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.749+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4e58004220 con 0x7f4e68103240 2024-05-14T15:24:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.750+0000 7f4e577fe700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e500b2880 0x7f4e500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.750+0000 7f4e65d9b700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e500b2880 0x7f4e500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.750+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f4e580e24e0 con 0x7f4e68103240 2024-05-14T15:24:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.750+0000 7f4e65d9b700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e500b2880 0x7f4e500b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b5c0 tx=0x7f4e5c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.754+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e58014030 con 0x7f4e68103240 2024-05-14T15:24:23.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.971+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4e681a2fb0 con 0x7f4e68103240 2024-05-14T15:24:23.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.972+0000 7f4e577fe700 1 -- 172.21.15.5:0/1995643863 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5690 (secure 0 0 0) 0x7f4e58004510 con 0x7f4e68103240 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 90 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-14T15:12:38.043878+0000 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-05-14T15:22:42.885032+0000 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 27 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-05-14T15:24:23.973 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/1440842370,v1:172.21.15.5:6803/1440842370] [v2:172.21.15.5:6804/1440842370,v1:172.21.15.5:6805/1440842370] exists,up ca22e76e-dbda-4a5c-be55-9740b288c6d4 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/3346749766,v1:172.21.15.144:6801/3346749766] [v2:172.21.15.144:6802/3346749766,v1:172.21.15.144:6803/3346749766] exists,up f8e57534-cfcb-479f-bcfc-64cadc8937cd 2024-05-14T15:24:23.974 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/3480718798,v1:172.21.15.193:6801/3480718798] [v2:172.21.15.193:6802/3480718798,v1:172.21.15.193:6803/3480718798] exists,up 58e105d1-e047-41db-a978-c3baed2b8d25 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/431512675,v1:172.21.15.5:6811/431512675] [v2:172.21.15.5:6812/431512675,v1:172.21.15.5:6813/431512675] exists,up 7530ff13-b342-47a6-9bf9-fa376975ab8f 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 32 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/3402609207,v1:172.21.15.144:6809/3402609207] [v2:172.21.15.144:6810/3402609207,v1:172.21.15.144:6811/3402609207] exists,up c8db2181-5e0b-407f-9ce9-3a23a0898b57 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2216132149,v1:172.21.15.193:6809/2216132149] [v2:172.21.15.193:6810/2216132149,v1:172.21.15.193:6811/2216132149] exists,up 3301b223-bb64-4c9e-a532-508e7e2e2e8f 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 43 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/1238248767,v1:172.21.15.5:6819/1238248767] [v2:172.21.15.5:6820/1238248767,v1:172.21.15.5:6821/1238248767] exists,up 024a549a-dc28-482e-a03d-4e4d5987e94b 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/3028125205,v1:172.21.15.144:6817/3028125205] [v2:172.21.15.144:6818/3028125205,v1:172.21.15.144:6819/3028125205] exists,up 3aebc627-3769-4770-b617-721385d1cb23 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 54 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/1885686165,v1:172.21.15.193:6817/1885686165] [v2:172.21.15.193:6818/1885686165,v1:172.21.15.193:6819/1885686165] exists,up bb0f1f87-2738-4488-8c2b-eac7983424ef 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/2591214668,v1:172.21.15.5:6827/2591214668] [v2:172.21.15.5:6828/2591214668,v1:172.21.15.5:6829/2591214668] exists,up 038f4941-b7d8-4fb3-8e34-dda9b11b012a 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 65 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/366654361,v1:172.21.15.144:6825/366654361] [v2:172.21.15.144:6826/366654361,v1:172.21.15.144:6827/366654361] exists,up a017373b-14d7-440d-a525-b6bb24b2690d 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/1840598125,v1:172.21.15.193:6825/1840598125] [v2:172.21.15.193:6826/1840598125,v1:172.21.15.193:6827/1840598125] exists,up f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.0 [8,1] 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.4 [2,11] 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.b [2,6] 2024-05-14T15:24:23.975 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.e [2,4] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.18 [5,11] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1a [8,4] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.6 [4,6] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,8] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.22 [3,9] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.25 [5,9] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.36 [4,9] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.18 [9,8] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.1b [9,8,5,10] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.1f [9,10] 2024-05-14T15:24:23.976 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/2154399460 expires 2024-05-15T15:13:41.367574+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/2401136202 expires 2024-05-15T15:13:13.551868+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2075859388 expires 2024-05-15T15:13:41.367574+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1587175256 expires 2024-05-15T15:13:13.551868+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4018221162 expires 2024-05-15T15:13:41.367574+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/2401136202 expires 2024-05-15T15:13:13.551868+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2357267352 expires 2024-05-15T15:13:41.367574+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1647912546 expires 2024-05-15T15:13:13.551868+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/2154399460 expires 2024-05-15T15:13:41.367574+0000 2024-05-14T15:24:23.977 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1991676154 expires 2024-05-15T15:13:13.551868+0000 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e500b2880 msgr2=0x7f4e500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e500b2880 0x7f4e500b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b5c0 tx=0x7f4e5c01a040 comp rx=0 tx=0).stop 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 msgr2=0x7f4e6819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 0x7f4e6819a900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800c970 tx=0x7f4e5800cc80 comp rx=0 tx=0).stop 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 shutdown_connections 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f4e500b2880 0x7f4e500b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e68104d10 0x7f4e681a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e68103240 0x7f4e6819a900 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 --2- 172.21.15.5:0/1995643863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e68104370 0x7f4e6819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 >> 172.21.15.5:0/1995643863 conn(0x7f4e680fe880 msgr2=0x7f4e681001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 shutdown_connections 2024-05-14T15:24:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:23.975+0000 7f4e6c80a700 1 -- 172.21.15.5:0/1995643863 wait complete. 2024-05-14T15:24:24.639 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-14T15:24:24.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:24:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:25.025 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:24 smithi005 ceph-mon[31559]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:25.025 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:24 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1995643863' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T15:24:25.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:24 smithi144 ceph-mon[35650]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:25.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:24 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1995643863' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T15:24:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:24 smithi193 ceph-mon[36001]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:25.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:24 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1995643863' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 -- 172.21.15.5:0/3451155481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90108780 msgr2=0x7f2c901100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 --2- 172.21.15.5:0/3451155481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90108780 0x7f2c901100e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b0d0 tx=0x7f2c8800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 -- 172.21.15.5:0/3451155481 shutdown_connections 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 --2- 172.21.15.5:0/3451155481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90108780 0x7f2c901100e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 --2- 172.21.15.5:0/3451155481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c90103550 0x7f2c90103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 --2- 172.21.15.5:0/3451155481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90107e30 0x7f2c90108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 -- 172.21.15.5:0/3451155481 >> 172.21.15.5:0/3451155481 conn(0x7f2c90078e00 msgr2=0x7f2c90079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:26.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 -- 172.21.15.5:0/3451155481 shutdown_connections 2024-05-14T15:24:26.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.591+0000 7f2c95426700 1 -- 172.21.15.5:0/3451155481 wait complete. 2024-05-14T15:24:26.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 Processor -- start 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 -- start start 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 0x7f2c9019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90107e30 0x7f2c9019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c90108780 0x7f2c901a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90113400 con 0x7f2c90107e30 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90113280 con 0x7f2c90103550 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c95426700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90113580 con 0x7f2c90108780 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 0x7f2c9019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 0x7f2c9019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34396/0 (socket says 172.21.15.5:34396) 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c8e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90107e30 0x7f2c9019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c8effd700 1 -- 172.21.15.5:0/1406492765 learned_addr learned my addr 172.21.15.5:0/1406492765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.592+0000 7f2c8f7fe700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c90108780 0x7f2c901a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8effd700 1 -- 172.21.15.5:0/1406492765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c90108780 msgr2=0x7f2c901a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8effd700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c90108780 0x7f2c901a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8effd700 1 -- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90107e30 msgr2=0x7f2c9019c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8effd700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90107e30 0x7f2c9019c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8effd700 1 -- 172.21.15.5:0/1406492765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9019c980 con 0x7f2c90103550 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8e7fc700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90107e30 0x7f2c9019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:24:26.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c8effd700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 0x7f2c9019bc20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c9b0 tx=0x7f2c7800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78007b00 con 0x7f2c90103550 2024-05-14T15:24:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c901a7240 con 0x7f2c90103550 2024-05-14T15:24:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7800ce50 con 0x7f2c90103550 2024-05-14T15:24:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78018660 con 0x7f2c90103550 2024-05-14T15:24:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.593+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a7780 con 0x7f2c90103550 2024-05-14T15:24:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.595+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c900512c0 con 0x7f2c90103550 2024-05-14T15:24:26.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.596+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2c7800f450 con 0x7f2c90103550 2024-05-14T15:24:26.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.597+0000 7f2c87fff700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c7c0b2880 0x7f2c7c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:26.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.597+0000 7f2c8e7fc700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c7c0b2880 0x7f2c7c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:26.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.597+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f2c780e27e0 con 0x7f2c90103550 2024-05-14T15:24:26.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.598+0000 7f2c8e7fc700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c7c0b2880 0x7f2c7c0b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c9010b200 tx=0x7f2c80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.601+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c78014030 con 0x7f2c90103550 2024-05-14T15:24:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.863+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c901a7a70 con 0x7f2c90103550 2024-05-14T15:24:26.864 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:26 smithi005 ceph-mon[31559]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.864+0000 7f2c87fff700 1 -- 172.21.15.5:0/1406492765 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11864 (secure 0 0 0) 0x7f2c780aa1c0 con 0x7f2c90103550 2024-05-14T15:24:26.865 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:22:00.688182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24487,24515]},"id":1}]} 2024-05-14T15:24:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c7c0b2880 msgr2=0x7f2c7c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c7c0b2880 0x7f2c7c0b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c9010b200 tx=0x7f2c80009260 comp rx=0 tx=0).stop 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 msgr2=0x7f2c9019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 0x7f2c9019bc20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c9b0 tx=0x7f2c7800ccc0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 shutdown_connections 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c7c0b2880 0x7f2c7c0b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c90108780 0x7f2c901a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90103550 0x7f2c9019bc20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 --2- 172.21.15.5:0/1406492765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c90107e30 0x7f2c9019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 >> 172.21.15.5:0/1406492765 conn(0x7f2c90078e00 msgr2=0x7f2c90100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 shutdown_connections 2024-05-14T15:24:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:26.867+0000 7f2c95426700 1 -- 172.21.15.5:0/1406492765 wait complete. 2024-05-14T15:24:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:24:27.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:26 smithi144 ceph-mon[35650]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:27.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:26 smithi193 ceph-mon[36001]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:27.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:24:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:28.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:27 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1406492765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:28.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:27 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1406492765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:28.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:27 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1406492765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:28.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:28 smithi005 ceph-mon[31559]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:29.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:28 smithi144 ceph-mon[35650]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:29.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:28 smithi193 ceph-mon[36001]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 -- 172.21.15.5:0/2768576630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34108f20 msgr2=0x7fce34109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/2768576630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34108f20 0x7fce34109330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fce280099c0 tx=0x7fce28009cd0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 -- 172.21.15.5:0/2768576630 shutdown_connections 2024-05-14T15:24:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/2768576630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce34109870 0x7fce341111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/2768576630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce341045f0 0x7fce34104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/2768576630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34108f20 0x7fce34109330 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 -- 172.21.15.5:0/2768576630 >> 172.21.15.5:0/2768576630 conn(0x7fce340ffec0 msgr2=0x7fce34102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 -- 172.21.15.5:0/2768576630 shutdown_connections 2024-05-14T15:24:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.486+0000 7fce3a5d1700 1 -- 172.21.15.5:0/2768576630 wait complete. 2024-05-14T15:24:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.487+0000 7fce3a5d1700 1 Processor -- start 2024-05-14T15:24:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.487+0000 7fce3a5d1700 1 -- start start 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce3a5d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce341045f0 0x7fce3419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce3a5d1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce34108f20 0x7fce3419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce3a5d1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 0x7fce3419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce3a5d1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce34114480 con 0x7fce34109870 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce3a5d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce34114300 con 0x7fce341045f0 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce3a5d1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce34114600 con 0x7fce34108f20 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 0x7fce3419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 0x7fce3419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44934/0 (socket says 172.21.15.5:44934) 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 -- 172.21.15.5:0/46339809 learned_addr learned my addr 172.21.15.5:0/46339809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce33fff700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce341045f0 0x7fce3419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce337fe700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce34108f20 0x7fce3419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 -- 172.21.15.5:0/46339809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce34108f20 msgr2=0x7fce3419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce34108f20 0x7fce3419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 -- 172.21.15.5:0/46339809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce341045f0 msgr2=0x7fce3419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce341045f0 0x7fce3419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 -- 172.21.15.5:0/46339809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3419fd00 con 0x7fce34109870 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.488+0000 7fce38b6e700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 0x7fce3419f660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce2c007ee0 tx=0x7fce2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.489+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c004bb0 con 0x7fce34109870 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.489+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2c004d50 con 0x7fce34109870 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.489+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3419fff0 con 0x7fce34109870 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.489+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c005920 con 0x7fce34109870 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.489+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce341a7910 con 0x7fce34109870 2024-05-14T15:24:29.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.490+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce34105aa0 con 0x7fce34109870 2024-05-14T15:24:29.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.491+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fce2c01a020 con 0x7fce34109870 2024-05-14T15:24:29.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.493+0000 7fce317fa700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fce1c0b2880 0x7fce1c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:29.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.493+0000 7fce33fff700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fce1c0b2880 0x7fce1c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:29.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.493+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fce2c0e2240 con 0x7fce34109870 2024-05-14T15:24:29.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.493+0000 7fce33fff700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fce1c0b2880 0x7fce1c0b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce24005950 tx=0x7fce2400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.496+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce2c010070 con 0x7fce34109870 2024-05-14T15:24:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.757+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce3419c190 con 0x7fce34109870 2024-05-14T15:24:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.758+0000 7fce317fa700 1 -- 172.21.15.5:0/46339809 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11864 (secure 0 0 0) 0x7fce2c0a9b90 con 0x7fce34109870 2024-05-14T15:24:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:29.759 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:22:00.688182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24487,24515]},"id":1}]} 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fce1c0b2880 msgr2=0x7fce1c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fce1c0b2880 0x7fce1c0b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce24005950 tx=0x7fce2400b410 comp rx=0 tx=0).stop 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 msgr2=0x7fce3419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 0x7fce3419f660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce2c007ee0 tx=0x7fce2c00d3b0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 shutdown_connections 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fce1c0b2880 0x7fce1c0b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce34109870 0x7fce3419f660 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce341045f0 0x7fce3419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 --2- 172.21.15.5:0/46339809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce34108f20 0x7fce3419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 >> 172.21.15.5:0/46339809 conn(0x7fce340ffec0 msgr2=0x7fce341022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:29.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 shutdown_connections 2024-05-14T15:24:29.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:29.761+0000 7fce3a5d1700 1 -- 172.21.15.5:0/46339809 wait complete. 2024-05-14T15:24:29.763 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-05-14T15:24:30.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:29 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/46339809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:30.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:29 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/46339809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:30.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:29 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/46339809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:30.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:24:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:31.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:30 smithi144 ceph-mon[35650]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:31.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:30 smithi193 ceph-mon[36001]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:31.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:30 smithi005 ceph-mon[31559]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 -- 172.21.15.5:0/4177962326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 msgr2=0x7fdc401100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/4177962326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc401100c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800b0d0 tx=0x7fdc3800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 -- 172.21.15.5:0/4177962326 shutdown_connections 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/4177962326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc401100c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/4177962326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 0x7fdc401038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/4177962326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 0x7fdc40108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 -- 172.21.15.5:0/4177962326 >> 172.21.15.5:0/4177962326 conn(0x7fdc400fef30 msgr2=0x7fdc40101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 -- 172.21.15.5:0/4177962326 shutdown_connections 2024-05-14T15:24:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.444+0000 7fdc44c3b700 1 -- 172.21.15.5:0/4177962326 wait complete. 2024-05-14T15:24:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.445+0000 7fdc44c3b700 1 Processor -- start 2024-05-14T15:24:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.445+0000 7fdc44c3b700 1 -- start start 2024-05-14T15:24:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc44c3b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 0x7fdc4019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc44c3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 0x7fdc4019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc44c3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc4019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc44c3b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40113390 con 0x7fdc40108760 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc44c3b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40113210 con 0x7fdc40107e10 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc44c3b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc40113510 con 0x7fdc401034e0 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc4019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc4019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44950/0 (socket says 172.21.15.5:44950) 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 -- 172.21.15.5:0/309843039 learned_addr learned my addr 172.21.15.5:0/309843039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3e59c700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 0x7fdc4019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3dd9b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 0x7fdc4019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 -- 172.21.15.5:0/309843039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 msgr2=0x7fdc4019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 0x7fdc4019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 -- 172.21.15.5:0/309843039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 msgr2=0x7fdc4019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 0x7fdc4019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 -- 172.21.15.5:0/309843039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc401a6fd0 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3e59c700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 0x7fdc4019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3dd9b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 0x7fdc4019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.446+0000 7fdc3ed9d700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc4019c7b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdc380095c0 tx=0x7fdc38007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.447+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801f070 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.447+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc38003c00 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.447+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc401a7260 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.447+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3801b6a0 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.447+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401a76b0 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.448+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc400512c0 con 0x7fdc40108760 2024-05-14T15:24:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.449+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdc38022070 con 0x7fdc40108760 2024-05-14T15:24:32.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.451+0000 7fdc2f7fe700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdc280b2880 0x7fdc280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:32.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.451+0000 7fdc3e59c700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdc280b2880 0x7fdc280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:32.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.451+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fdc3800f020 con 0x7fdc40108760 2024-05-14T15:24:32.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.451+0000 7fdc3e59c700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdc280b2880 0x7fdc280b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005950 tx=0x7fdc300058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.454+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc380e3e20 con 0x7fdc40108760 2024-05-14T15:24:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.668+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc4002e7b0 con 0x7fdc40108760 2024-05-14T15:24:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.670+0000 7fdc2f7fe700 1 -- 172.21.15.5:0/309843039 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24761 (secure 0 0 0) 0x7fdc38029020 con 0x7fdc40108760 2024-05-14T15:24:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:32.671 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:22:42.885032+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:24:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdc280b2880 msgr2=0x7fdc280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdc280b2880 0x7fdc280b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005950 tx=0x7fdc300058e0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 msgr2=0x7fdc4019c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc4019c7b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdc380095c0 tx=0x7fdc38007ad0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 shutdown_connections 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fdc280b2880 0x7fdc280b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc40108760 0x7fdc4019c7b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc401034e0 0x7fdc4019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.672+0000 7fdc44c3b700 1 --2- 172.21.15.5:0/309843039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc40107e10 0x7fdc4019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.673+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 >> 172.21.15.5:0/309843039 conn(0x7fdc400fef30 msgr2=0x7fdc40107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.673+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 shutdown_connections 2024-05-14T15:24:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:32.673+0000 7fdc44c3b700 1 -- 172.21.15.5:0/309843039 wait complete. 2024-05-14T15:24:33.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:32 smithi144 ceph-mon[35650]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:33.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:32 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/309843039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:24:33.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:32 smithi005 ceph-mon[31559]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:33.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:32 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/309843039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:24:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:32 smithi193 ceph-mon[36001]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:32 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/309843039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:24:33.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T15:20:57.810761+0000', 'modified': '2024-05-14T15:22:00.688182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24515, 'mds_1': 24491, 'mds_2': 24487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/4153907810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 4153907810}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 4153907810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6837/1910760961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1910760961}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1910760961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3352720235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 3352720235}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3352720235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.193:6839/570860802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 570860802}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 570860802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.144:6837/214149479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 214149479}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 214149479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6839/4183436774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 4183436774}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 4183436774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24491, 24487, 24515]} 2024-05-14T15:24:33.367 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T15:24:33.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-14T15:24:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:34.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:34 smithi005 ceph-mon[31559]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:34 smithi144 ceph-mon[35650]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:35.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:34 smithi193 ceph-mon[36001]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 -- 172.21.15.5:0/999067268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 msgr2=0x7f70c8110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 --2- 172.21.15.5:0/999067268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c8110070 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70c000b0a0 tx=0x7f70c000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 -- 172.21.15.5:0/999067268 shutdown_connections 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 --2- 172.21.15.5:0/999067268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c8110070 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 --2- 172.21.15.5:0/999067268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c8108200 0x7f70c8108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 --2- 172.21.15.5:0/999067268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8103960 0x7f70c8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 -- 172.21.15.5:0/999067268 >> 172.21.15.5:0/999067268 conn(0x7f70c80fe520 msgr2=0x7f70c8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 -- 172.21.15.5:0/999067268 shutdown_connections 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.361+0000 7f70cf182700 1 -- 172.21.15.5:0/999067268 wait complete. 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 Processor -- start 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 -- start start 2024-05-14T15:24:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c8103960 0x7f70c819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8108200 0x7f70c819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8113390 con 0x7f70c81042b0 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8113210 con 0x7f70c8103960 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70cf182700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8113510 con 0x7f70c8108200 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.362+0000 7f70c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c819c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44962/0 (socket says 172.21.15.5:44962) 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 -- 172.21.15.5:0/3050373755 learned_addr learned my addr 172.21.15.5:0/3050373755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c8103960 0x7f70c819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70cd71f700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8108200 0x7f70c819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 -- 172.21.15.5:0/3050373755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8108200 msgr2=0x7f70c819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8108200 0x7f70c819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 -- 172.21.15.5:0/3050373755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c8103960 msgr2=0x7f70c819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c8103960 0x7f70c819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 -- 172.21.15.5:0/3050373755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c81a6ee0 con 0x7f70c81042b0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70cd71f700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8108200 0x7f70c819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c7fff700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c819c0b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d980 tx=0x7f70b400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c5ffb700 1 -- 172.21.15.5:0/3050373755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4016070 con 0x7f70c81042b0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c5ffb700 1 -- 172.21.15.5:0/3050373755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b40044d0 con 0x7f70c81042b0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c81a71d0 con 0x7f70c81042b0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.363+0000 7f70c5ffb700 1 -- 172.21.15.5:0/3050373755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4004d00 con 0x7f70c81042b0 2024-05-14T15:24:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.364+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81a7710 con 0x7f70c81042b0 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.366+0000 7f70c5ffb700 1 -- 172.21.15.5:0/3050373755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f70b4004020 con 0x7f70c81042b0 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.367+0000 7f70c5ffb700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f70b80b2880 0x7f70b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.367+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f70b80b2880 0x7f70b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.367+0000 7f70c5ffb700 1 -- 172.21.15.5:0/3050373755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f70b40e24b0 con 0x7f70c81042b0 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.367+0000 7f70ddb96b80 1 Processor -- start 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.367+0000 7f70ddb96b80 1 -- start start 2024-05-14T15:24:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.367+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f70b80b2880 0x7f70b80b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70bc005950 tx=0x7f70bc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a30ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a31830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7f394e810 con 0x55f7f3a31420 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7f38ed770 con 0x55f7f3a2cc10 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f7f394a7d0 con 0x55f7f38aedf0 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a31830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a31830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44968/0 (socket says 172.21.15.5:44968) 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 -- 172.21.15.5:0/2173317956 learned_addr learned my addr 172.21.15.5:0/2173317956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70c7fff700 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a30ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 -- 172.21.15.5:0/2173317956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 msgr2=0x55f7f3a2c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a2c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 -- 172.21.15.5:0/2173317956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 msgr2=0x55f7f3a30ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a30ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 -- 172.21.15.5:0/2173317956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f389b880 con 0x55f7f3a31420 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70c7fff700 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a30ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70cd71f700 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a31830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f70c000b070 tx=0x7f70c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d6e6bc9c34f858 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.368+0000 7f70b2ffd700 1 -- 172.21.15.5:0/2173317956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0020070 con 0x55f7f3a31420 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.369+0000 7f70b2ffd700 1 -- 172.21.15.5:0/2173317956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c00092d0 con 0x55f7f3a31420 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.369+0000 7f70b2ffd700 1 -- 172.21.15.5:0/2173317956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007e00 con 0x55f7f3a31420 2024-05-14T15:24:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.369+0000 7f70ddb96b80 1 -- 172.21.15.5:0/2173317956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 msgr2=0x55f7f3a31830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.369+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a31830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f70c000b070 tx=0x7f70c0009c00 comp rx=0 tx=0).stop 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 -- 172.21.15.5:0/2173317956 shutdown_connections 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a31830 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a2c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/2173317956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a30ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 -- 172.21.15.5:0/2173317956 >> 172.21.15.5:0/2173317956 conn(0x55f7f38f65b0 msgr2=0x55f7f3a27460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 -- 172.21.15.5:0/2173317956 shutdown_connections 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 -- 172.21.15.5:0/2173317956 wait complete. 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 -- start start 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.370+0000 7f70ddb96b80 1 Processor -- start 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a36610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a36b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a39370 con 0x55f7f3a31420 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7f38b4be0 con 0x55f7f3a2cc10 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a394f0 con 0x55f7f38aedf0 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ddb96b80 10 client.0 _ll_register_callbacks cb 0x55f7f38e68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a36b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a36b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44978/0 (socket says 172.21.15.5:44978) 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 -- 172.21.15.5:0/796389735 learned_addr learned my addr 172.21.15.5:0/796389735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70c7fff700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a36610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 -- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 msgr2=0x55f7f3a360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 -- 172.21.15.5:0/796389735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 msgr2=0x55f7f3a36610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a36610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 -- 172.21.15.5:0/796389735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f38ea640 con 0x55f7f3a31420 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70cd71f700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a36b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70c0007940 tx=0x7f70c0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f735c3b378b33c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.371+0000 7f70a7fff700 1 -- 172.21.15.5:0/796389735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0020040 con 0x55f7f3a31420 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70a7fff700 1 -- 172.21.15.5:0/796389735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f70c0007db0 con 0x55f7f3a31420 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c00092d0 con 0x55f7f3a31420 2024-05-14T15:24:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70a7fff700 1 -- 172.21.15.5:0/796389735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70c0025450 con 0x55f7f3a31420 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 10 client.15180 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7f3a43ec0 con 0x55f7f3a31420 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70a7fff700 1 -- 172.21.15.5:0/796389735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c00092d0 con 0x55f7f3a31420 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70a7fff700 1 -- 172.21.15.5:0/796389735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70c0011070 con 0x55f7f3a31420 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 20 client.15180 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3a45210 0x55f7f3a473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 4 client.15180 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.372+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7f395c1f0 con 0x55f7f3a45210 2024-05-14T15:24:35.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.373+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3a45210 0x55f7f3a473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.373+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3a45210 0x55f7f3a473d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ec8952bea5300b9 server_cookie=6553793f012df964 in_seq=0 out_seq=0 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.373+0000 7f70a7fff700 10 client.15180 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70a7fff700 1 -- 172.21.15.5:0/796389735 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7f395c1f0 con 0x55f7f3a45210 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70a7fff700 10 client.15180 handle_command_reply: tid=0 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 client.15180 shutdown 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 msgr2=0x55f7f3a36b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a36b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f70c0007940 tx=0x7f70c0004db0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 shutdown_connections 2024-05-14T15:24:35.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a31420 0x55f7f3a36b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3a45210 0x55f7f3a473d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/796389735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a36610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70a7fff700 0 client.15180 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.374+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 >> 172.21.15.5:0/796389735 conn(0x55f7f38f65b0 msgr2=0x55f7f3a2b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.375+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 shutdown_connections 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.375+0000 7f70ddb96b80 1 -- 172.21.15.5:0/796389735 wait complete. 2024-05-14T15:24:35.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.375+0000 7f70ddb96b80 20 client.15180 trim_cache size 0 max 16384 2024-05-14T15:24:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.396+0000 7f70ddb96b80 1 Processor -- start 2024-05-14T15:24:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.396+0000 7f70ddb96b80 1 -- start start 2024-05-14T15:24:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a42790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a45710 con 0x55f7f3a2cc10 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a45de0 con 0x55f7f3a31420 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a464b0 con 0x55f7f38aedf0 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a42790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a42790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44984/0 (socket says 172.21.15.5:44984) 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 -- 172.21.15.5:0/36141650 learned_addr learned my addr 172.21.15.5:0/36141650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70cd71f700 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 -- 172.21.15.5:0/36141650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 msgr2=0x55f7f3a349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 -- 172.21.15.5:0/36141650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 msgr2=0x55f7f3a30de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a30de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 -- 172.21.15.5:0/36141650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f3a23a90 con 0x55f7f3a2cc10 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70c7fff700 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a42790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f70b400b540 tx=0x7f70b40a8f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460732a86d022ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70b27fc700 1 -- 172.21.15.5:0/36141650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40c3070 con 0x55f7f3a2cc10 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70b27fc700 1 -- 172.21.15.5:0/36141650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b40a9210 con 0x55f7f3a2cc10 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.397+0000 7f70b27fc700 1 -- 172.21.15.5:0/36141650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4091320 con 0x55f7f3a2cc10 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 -- 172.21.15.5:0/36141650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 msgr2=0x55f7f3a42790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a42790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f70b400b540 tx=0x7f70b40a8f40 comp rx=0 tx=0).stop 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 -- 172.21.15.5:0/36141650 shutdown_connections 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a30de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/36141650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a42790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 -- 172.21.15.5:0/36141650 >> 172.21.15.5:0/36141650 conn(0x55f7f38f65b0 msgr2=0x55f7f3a306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 -- 172.21.15.5:0/36141650 shutdown_connections 2024-05-14T15:24:35.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.398+0000 7f70ddb96b80 1 -- 172.21.15.5:0/36141650 wait complete. 2024-05-14T15:24:35.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 -- start start 2024-05-14T15:24:35.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 Processor -- start 2024-05-14T15:24:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a50a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a4cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a4d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a47250 con 0x55f7f3a2cc10 2024-05-14T15:24:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a46b80 con 0x55f7f3a31420 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f7f3a47920 con 0x55f7f38aedf0 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70ddb96b80 10 client.0 _ll_register_callbacks cb 0x55f7f38e68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.399+0000 7f70c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a4cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a4cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44992/0 (socket says 172.21.15.5:44992) 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 -- 172.21.15.5:0/628057856 learned_addr learned my addr 172.21.15.5:0/628057856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70cd71f700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a4d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a50a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 -- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 msgr2=0x55f7f3a50a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a50a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 -- 172.21.15.5:0/628057856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 msgr2=0x55f7f3a4d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a4d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 -- 172.21.15.5:0/628057856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f3a4d860 con 0x55f7f3a2cc10 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a50a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70c7fff700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a4cde0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70b40a5730 tx=0x7f70b400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c19194db7360216 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70a7fff700 1 -- 172.21.15.5:0/628057856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b406a050 con 0x55f7f3a2cc10 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70a7fff700 1 -- 172.21.15.5:0/628057856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f70b40b22e0 con 0x55f7f3a2cc10 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b40a9210 con 0x55f7f3a2cc10 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70a7fff700 1 -- 172.21.15.5:0/628057856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70b40c3040 con 0x55f7f3a2cc10 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70ddb96b80 10 client.15186 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.400+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7f3ab1fc0 con 0x55f7f3a2cc10 2024-05-14T15:24:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70a7fff700 1 -- 172.21.15.5:0/628057856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b40a9210 con 0x55f7f3a2cc10 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70a7fff700 1 -- 172.21.15.5:0/628057856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70b4091080 con 0x55f7f3a2cc10 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ddb96b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ddb96b80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ddb96b80 20 client.15186 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3ab3ed0 0x55f7f3ab5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ddb96b80 4 client.15186 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "status"}) -- 0x55f7f3956290 con 0x55f7f3ab3ed0 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3ab3ed0 0x55f7f3ab5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70ccf1e700 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3ab3ed0 0x55f7f3ab5340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad1a32b771894e4b server_cookie=439c4ba005e20833 in_seq=0 out_seq=0 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.401+0000 7f70a7fff700 10 client.15186 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70a7fff700 1 -- 172.21.15.5:0/628057856 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f7f3956290 con 0x55f7f3ab3ed0 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70a7fff700 10 client.15186 handle_command_reply: tid=0 2024-05-14T15:24:35.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 client.15186 shutdown 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 msgr2=0x55f7f3a4cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a4cde0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70b40a5730 tx=0x7f70b400bd90 comp rx=0 tx=0).stop 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 shutdown_connections 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55f7f3ab3ed0 0x55f7f3ab5340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7f3a31420 0x55f7f3a4d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7f38aedf0 0x55f7f3a50a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70a7fff700 0 client.15186 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 --2- 172.21.15.5:0/628057856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7f3a2cc10 0x55f7f3a4cde0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 >> 172.21.15.5:0/628057856 conn(0x55f7f38f65b0 msgr2=0x55f7f3a306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 shutdown_connections 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 1 -- 172.21.15.5:0/628057856 wait complete. 2024-05-14T15:24:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.402+0000 7f70ddb96b80 20 client.15186 trim_cache size 0 max 16384 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24487, 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 159.64169740099999, 2024-05-14T15:24:35.410 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 245.512073904 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f70b80b2880 msgr2=0x7f70b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f70b80b2880 0x7f70b80b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70bc005950 tx=0x7f70bc0058e0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 msgr2=0x7f70c819c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c819c0b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d980 tx=0x7f70b400dd40 comp rx=0 tx=0).stop 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 shutdown_connections 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f70b80b2880 0x7f70b80b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70c8108200 0x7f70c819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c8103960 0x7f70c819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 --2- 172.21.15.5:0/3050373755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70c81042b0 0x7f70c819c0b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 >> 172.21.15.5:0/3050373755 conn(0x7f70c80fe520 msgr2=0x7f70c8107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 shutdown_connections 2024-05-14T15:24:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:35.403+0000 7f70cf182700 1 -- 172.21.15.5:0/3050373755 wait complete. 2024-05-14T15:24:36.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:35 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:36.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T15:24:36.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:35 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:36.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:35 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:36.385 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:37.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:36 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:36 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T15:24:37.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:36 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:38.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:37 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:38.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:37 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:38.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:37 smithi005 ceph-mon[31559]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 -- 172.21.15.5:0/3478421470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08109850 msgr2=0x7f9c081111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3478421470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08109850 0x7f9c081111b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b0d0 tx=0x7f9c0000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 -- 172.21.15.5:0/3478421470 shutdown_connections 2024-05-14T15:24:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3478421470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08109850 0x7f9c081111b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3478421470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081045d0 0x7f9c081049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3478421470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08108f00 0x7f9c08109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 -- 172.21.15.5:0/3478421470 >> 172.21.15.5:0/3478421470 conn(0x7f9c080ffea0 msgr2=0x7f9c081022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 -- 172.21.15.5:0/3478421470 shutdown_connections 2024-05-14T15:24:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.080+0000 7f9c0e792700 1 -- 172.21.15.5:0/3478421470 wait complete. 2024-05-14T15:24:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 Processor -- start 2024-05-14T15:24:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 -- start start 2024-05-14T15:24:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081045d0 0x7f9c08076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 0x7f9c08077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08109850 0x7f9c08077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08114480 con 0x7f9c08108f00 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08114300 con 0x7f9c081045d0 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.081+0000 7f9c0e792700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08114600 con 0x7f9c08109850 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 0x7f9c08077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 0x7f9c08077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35020/0 (socket says 172.21.15.5:35020) 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 -- 172.21.15.5:0/3274114833 learned_addr learned my addr 172.21.15.5:0/3274114833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08109850 0x7f9c08077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c07fff700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081045d0 0x7f9c08076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 -- 172.21.15.5:0/3274114833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08109850 msgr2=0x7f9c08077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08109850 0x7f9c08077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 -- 172.21.15.5:0/3274114833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081045d0 msgr2=0x7f9c08076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081045d0 0x7f9c08076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 -- 172.21.15.5:0/3274114833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c081a9150 con 0x7f9c08108f00 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08109850 0x7f9c08077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9bfffff700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 0x7f9c08077110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000e970 tx=0x7f9bf000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c05ffb700 1 -- 172.21.15.5:0/3274114833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000cbd0 con 0x7f9c08108f00 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c05ffb700 1 -- 172.21.15.5:0/3274114833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf000cd70 con 0x7f9c08108f00 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c081a9440 con 0x7f9c08108f00 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.082+0000 7f9c05ffb700 1 -- 172.21.15.5:0/3274114833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0010430 con 0x7f9c08108f00 2024-05-14T15:24:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.083+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081a99d0 con 0x7f9c08108f00 2024-05-14T15:24:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.085+0000 7f9c05ffb700 1 -- 172.21.15.5:0/3274114833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9bf0004d70 con 0x7f9c08108f00 2024-05-14T15:24:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.086+0000 7f9c05ffb700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9bf40b2900 0x7f9bf40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.086+0000 7f9c07fff700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9bf40b2900 0x7f9bf40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.086+0000 7f9c05ffb700 1 -- 172.21.15.5:0/3274114833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f9bf00e2810 con 0x7f9c08108f00 2024-05-14T15:24:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.086+0000 7f9c1d1a6b80 1 Processor -- start 2024-05-14T15:24:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.086+0000 7f9c1d1a6b80 1 -- start start 2024-05-14T15:24:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c07fff700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9bf40b2900 0x7f9bf40b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005fd0 tx=0x7f9bf8005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7988810 con 0x55f9a7a6b420 2024-05-14T15:24:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7927770 con 0x55f9a7a66c10 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9a79847d0 con 0x55f9a78e8df0 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35032/0 (socket says 172.21.15.5:35032) 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/3613051047 learned_addr learned my addr 172.21.15.5:0/3613051047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c07fff700 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9bfffff700 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/3613051047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 msgr2=0x55f9a7a666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/3613051047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 msgr2=0x55f9a7a6aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/3613051047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a78d5880 con 0x55f9a7a6b420 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.087+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6b830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009dc0 tx=0x7f9c00009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d749b550f31a49bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.088+0000 7f9bfe7fc700 1 -- 172.21.15.5:0/3613051047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00022070 con 0x55f9a7a6b420 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.088+0000 7f9bfe7fc700 1 -- 172.21.15.5:0/3613051047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000075c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.088+0000 7f9bfe7fc700 1 -- 172.21.15.5:0/3613051047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0001c6a0 con 0x55f9a7a6b420 2024-05-14T15:24:38.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.088+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/3613051047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 msgr2=0x55f9a7a6b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.088+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6b830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009dc0 tx=0x7f9c00009df0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/3613051047 shutdown_connections 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6b830 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/3613051047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/3613051047 >> 172.21.15.5:0/3613051047 conn(0x55f9a79305b0 msgr2=0x55f9a7a61460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/3613051047 shutdown_connections 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/3613051047 wait complete. 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 -- start start 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.089+0000 7f9c1d1a6b80 1 Processor -- start 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a78e8df0 0x55f9a7a700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a7a66c10 0x55f9a7a70620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a70b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a73370 con 0x55f9a7a6b420 2024-05-14T15:24:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9a78eebe0 con 0x55f9a78e8df0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a734f0 con 0x55f9a7a66c10 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c1d1a6b80 10 client.0 _ll_register_callbacks cb 0x55f9a79208d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a70b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a70b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35044/0 (socket says 172.21.15.5:35044) 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/924909540 learned_addr learned my addr 172.21.15.5:0/924909540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9bfffff700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a7a66c10 0x55f9a7a70620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c07fff700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a78e8df0 0x55f9a7a700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/924909540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a7a66c10 msgr2=0x55f9a7a70620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a7a66c10 0x55f9a7a70620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a78e8df0 msgr2=0x55f9a7a700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a78e8df0 0x55f9a7a700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/924909540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a7924640 con 0x55f9a7a6b420 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a70b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c00006260 tx=0x7f9c0000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d288e490e88cb40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.090+0000 7f9be77fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9be77fe700 1 -- 172.21.15.5:0/924909540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0002d070 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9be77fe700 1 -- 172.21.15.5:0/924909540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f9c00027650 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c000075c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9be77fe700 1 -- 172.21.15.5:0/924909540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c00039050 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 10 client.15198 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9a7a7df20 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9be77fe700 1 -- 172.21.15.5:0/924909540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000075c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9be77fe700 1 -- 172.21.15.5:0/924909540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c00011070 con 0x55f9a7a6b420 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 20 client.15198 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7a813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 4 client.15198 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.091+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a79961f0 con 0x55f9a7a7f270 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.092+0000 7f9c07fff700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7a813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.092+0000 7f9be77fe700 10 client.15198 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.092+0000 7f9c07fff700 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7a813c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d99f8cc78d2c37e2 server_cookie=6ab159ca18380813 in_seq=0 out_seq=0 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9be77fe700 1 -- 172.21.15.5:0/924909540 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9a79961f0 con 0x55f9a7a7f270 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9be77fe700 10 client.15198 handle_command_reply: tid=0 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 client.15198 shutdown 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 msgr2=0x55f9a7a70b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a70b60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c00006260 tx=0x7f9c0000b730 comp rx=0 tx=0).stop 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 shutdown_connections 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7a813c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9be77fe700 0 client.15198 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a70b60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a78e8df0 0x55f9a7a700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/924909540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a7a66c10 0x55f9a7a70620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.093+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 >> 172.21.15.5:0/924909540 conn(0x55f9a79305b0 msgr2=0x55f9a7a65400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.094+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 shutdown_connections 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.094+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/924909540 wait complete. 2024-05-14T15:24:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.094+0000 7f9c1d1a6b80 20 client.15198 trim_cache size 0 max 16384 2024-05-14T15:24:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 Processor -- start 2024-05-14T15:24:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 -- start start 2024-05-14T15:24:38.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a7c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a7f700 con 0x55f9a7a6b420 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a7fdd0 con 0x55f9a7a66c10 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a804a0 con 0x55f9a78e8df0 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35052/0 (socket says 172.21.15.5:35052) 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1420134008 learned_addr learned my addr 172.21.15.5:0/1420134008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c07fff700 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a7c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9bfffff700 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1420134008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 msgr2=0x55f9a7a7c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a7c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1420134008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 msgr2=0x55f9a7a6a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1420134008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a7a7fbf0 con 0x55f9a7a6b420 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.116+0000 7f9bfffff700 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.117+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6ac70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b420 tx=0x7f9c00009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcec0d966732bffb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.117+0000 7f9bfdffb700 1 -- 172.21.15.5:0/1420134008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00036070 con 0x55f9a7a6b420 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.117+0000 7f9bfdffb700 1 -- 172.21.15.5:0/1420134008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000075c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.117+0000 7f9bfdffb700 1 -- 172.21.15.5:0/1420134008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0002d070 con 0x55f9a7a6b420 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1420134008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 msgr2=0x55f9a7a6ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6ac70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b420 tx=0x7f9c00009e10 comp rx=0 tx=0).stop 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1420134008 shutdown_connections 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a6ac70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a7c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1420134008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a6a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1420134008 >> 172.21.15.5:0/1420134008 conn(0x55f9a79305b0 msgr2=0x55f9a7a6e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1420134008 shutdown_connections 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1420134008 wait complete. 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 -- start start 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.118+0000 7f9c1d1a6b80 1 Processor -- start 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a86dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a87310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a87850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a81240 con 0x55f9a7a6b420 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a80b70 con 0x55f9a7a66c10 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9a7a81910 con 0x55f9a78e8df0 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c1d1a6b80 10 client.0 _ll_register_callbacks cb 0x55f9a79208d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a87850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a87850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35068/0 (socket says 172.21.15.5:35068) 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1926158160 learned_addr learned my addr 172.21.15.5:0/1926158160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c07fff700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a86dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9bfffff700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a87310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1926158160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 msgr2=0x55f9a7a86dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a86dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 msgr2=0x55f9a7a87310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a87310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 -- 172.21.15.5:0/1926158160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a7aea3c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9bfffff700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a87310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9c0cd2f700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a87850 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000d010 tx=0x7f9c00005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112513feedf397f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.119+0000 7f9be77fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9be77fe700 1 -- 172.21.15.5:0/1926158160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00036070 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9be77fe700 1 -- 172.21.15.5:0/1926158160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f9c0001c650 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c000075c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9be77fe700 1 -- 172.21.15.5:0/1926158160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c0004b050 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 10 client.15204 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9a7aeb4f0 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9be77fe700 1 -- 172.21.15.5:0/1926158160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c000075c0 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9be77fe700 1 -- 172.21.15.5:0/1926158160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c00046030 con 0x55f9a7a6b420 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T15:24:38.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 20 client.15204 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7aeea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 4 client.15204 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.120+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "status"}) -- 0x55f9a7990290 con 0x55f9a7a7f270 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.121+0000 7f9c07fff700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7aeea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.121+0000 7f9c07fff700 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7aeea10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c8d8cde500144d1 server_cookie=a7821537fc7a907 in_seq=0 out_seq=0 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.121+0000 7f9be77fe700 10 client.15204 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.121+0000 7f9be77fe700 1 -- 172.21.15.5:0/1926158160 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f9a7990290 con 0x55f9a7a7f270 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.121+0000 7f9be77fe700 10 client.15204 handle_command_reply: tid=0 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 client.15204 shutdown 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 msgr2=0x55f9a7a87850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a87850 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000d010 tx=0x7f9c00005e90 comp rx=0 tx=0).stop 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 shutdown_connections 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f9a7a7f270 0x55f9a7aeea10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a7a6b420 0x55f9a7a87850 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9a78e8df0 0x55f9a7a86dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9be77fe700 0 client.15204 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 --2- 172.21.15.5:0/1926158160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9a7a66c10 0x55f9a7a87310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 >> 172.21.15.5:0/1926158160 conn(0x55f9a79305b0 msgr2=0x55f9a7a60b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 shutdown_connections 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 1 -- 172.21.15.5:0/1926158160 wait complete. 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c1d1a6b80 20 client.15204 trim_cache size 0 max 16384 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24491, 2024-05-14T15:24:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 163.37000912400001, 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 251.019483837 2024-05-14T15:24:38.128 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.122+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9bf40b2900 msgr2=0x7f9bf40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9bf40b2900 0x7f9bf40b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005fd0 tx=0x7f9bf8005f60 comp rx=0 tx=0).stop 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 msgr2=0x7f9c08077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 0x7f9c08077110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000e970 tx=0x7f9bf000ed30 comp rx=0 tx=0).stop 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 shutdown_connections 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f9bf40b2900 0x7f9bf40b4e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c08109850 0x7f9c08077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c081045d0 0x7f9c08076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 --2- 172.21.15.5:0/3274114833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c08108f00 0x7f9c08077110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 >> 172.21.15.5:0/3274114833 conn(0x7f9c080ffea0 msgr2=0x7f9c08108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 shutdown_connections 2024-05-14T15:24:38.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:38.123+0000 7f9c0e792700 1 -- 172.21.15.5:0/3274114833 wait complete. 2024-05-14T15:24:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:37 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:37 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:37 smithi144 ceph-mon[35650]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:37 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:37 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:37 smithi193 ceph-mon[36001]: from='mgr.14150 172.21.15.5:0/3234976250' entity='mgr.x' 2024-05-14T15:24:38.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T15:24:39.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:38 smithi005 ceph-mon[31559]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:39.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:38 smithi144 ceph-mon[35650]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:39.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:38 smithi193 ceph-mon[36001]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T15:24:40.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:40 smithi005 ceph-mon[31559]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:40.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.835+0000 7fc714bdb700 1 -- 172.21.15.5:0/2541404968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710107e20 msgr2=0x7fc710108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.835+0000 7fc714bdb700 1 --2- 172.21.15.5:0/2541404968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710107e20 0x7fc710108230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc7040099c0 tx=0x7fc704009cd0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 -- 172.21.15.5:0/2541404968 shutdown_connections 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 --2- 172.21.15.5:0/2541404968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710108770 0x7fc7101100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 --2- 172.21.15.5:0/2541404968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc710103540 0x7fc710103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 --2- 172.21.15.5:0/2541404968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710107e20 0x7fc710108230 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 -- 172.21.15.5:0/2541404968 >> 172.21.15.5:0/2541404968 conn(0x7fc710078e50 msgr2=0x7fc710079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 -- 172.21.15.5:0/2541404968 shutdown_connections 2024-05-14T15:24:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.836+0000 7fc714bdb700 1 -- 172.21.15.5:0/2541404968 wait complete. 2024-05-14T15:24:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 Processor -- start 2024-05-14T15:24:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 -- start start 2024-05-14T15:24:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710103540 0x7fc71019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc710107e20 0x7fc71019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 0x7fc71019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc710113380 con 0x7fc710108770 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc710113200 con 0x7fc710107e20 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc714bdb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc710113500 con 0x7fc710103540 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 0x7fc71019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 0x7fc71019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35088/0 (socket says 172.21.15.5:35088) 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 -- 172.21.15.5:0/300789835 learned_addr learned my addr 172.21.15.5:0/300789835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc710107e20 0x7fc71019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70e59c700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710103540 0x7fc71019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 -- 172.21.15.5:0/300789835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710103540 msgr2=0x7fc71019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710103540 0x7fc71019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 -- 172.21.15.5:0/300789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc710107e20 msgr2=0x7fc71019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.837+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc710107e20 0x7fc71019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc70ed9d700 1 -- 172.21.15.5:0/300789835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7101a6fd0 con 0x7fc710108770 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 0x7fc71019c720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d980 tx=0x7fc70800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc6ff7fe700 1 -- 172.21.15.5:0/300789835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708017070 con 0x7fc710108770 2024-05-14T15:24:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7101a72c0 con 0x7fc710108770 2024-05-14T15:24:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc6ff7fe700 1 -- 172.21.15.5:0/300789835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708005020 con 0x7fc710108770 2024-05-14T15:24:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc6ff7fe700 1 -- 172.21.15.5:0/300789835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70800fa10 con 0x7fc710108770 2024-05-14T15:24:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.838+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a77d0 con 0x7fc710108770 2024-05-14T15:24:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.841+0000 7fc6ff7fe700 1 -- 172.21.15.5:0/300789835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc708004a30 con 0x7fc710108770 2024-05-14T15:24:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.842+0000 7fc6ff7fe700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc6f80b2880 0x7fc6f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.842+0000 7fc7235efb80 1 Processor -- start 2024-05-14T15:24:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.842+0000 7fc70e59c700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc6f80b2880 0x7fc6f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.842+0000 7fc6ff7fe700 1 -- 172.21.15.5:0/300789835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fc7080e23c0 con 0x7fc710108770 2024-05-14T15:24:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.842+0000 7fc7235efb80 1 -- start start 2024-05-14T15:24:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085835b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708582fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x5570858305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70e59c700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc6f80b2880 0x7fc6f80b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc700009550 tx=0x7fc7000091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc7235efb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55708574d8f0 con 0x557085836060 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc7235efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570856b2d50 con 0x5570856addf0 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc7235efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5570857498b0 con 0x5570858301d0 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708582fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708582fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35092/0 (socket says 172.21.15.5:35092) 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 -- 172.21.15.5:0/1115292667 learned_addr learned my addr 172.21.15.5:0/1115292667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x5570858305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70e59c700 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085835b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 -- 172.21.15.5:0/1115292667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 msgr2=0x5570858305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x5570858305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 -- 172.21.15.5:0/1115292667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 msgr2=0x557085835b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085835b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 -- 172.21.15.5:0/1115292667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708569a880 con 0x557085836060 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70e59c700 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085835b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.843+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708582fc90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc7040099c0 tx=0x7fc704009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f132f3c0cb860fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.844+0000 7fc6fcff9700 1 -- 172.21.15.5:0/1115292667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704003ae0 con 0x557085836060 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.844+0000 7fc6fcff9700 1 -- 172.21.15.5:0/1115292667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc704005580 con 0x557085836060 2024-05-14T15:24:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.844+0000 7fc6fcff9700 1 -- 172.21.15.5:0/1115292667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400bc70 con 0x557085836060 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 -- 172.21.15.5:0/1115292667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 msgr2=0x55708582fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708582fc90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc7040099c0 tx=0x7fc704009630 comp rx=0 tx=0).stop 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 -- 172.21.15.5:0/1115292667 shutdown_connections 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x5570858305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085835b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 --2- 172.21.15.5:0/1115292667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708582fc90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 -- 172.21.15.5:0/1115292667 >> 172.21.15.5:0/1115292667 conn(0x5570856f55e0 msgr2=0x5570858269b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 -- 172.21.15.5:0/1115292667 shutdown_connections 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 -- 172.21.15.5:0/1115292667 wait complete. 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 -- start start 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.845+0000 7fc7235efb80 1 Processor -- start 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085839890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x557085839dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708583a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55708583cae0 con 0x557085836060 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557085706e80 con 0x5570856addf0 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55708583cc60 con 0x5570858301d0 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc7235efb80 10 client.0 _ll_register_callbacks cb 0x5570856e5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708583a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708583a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35106/0 (socket says 172.21.15.5:35106) 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 -- 172.21.15.5:0/3556690489 learned_addr learned my addr 172.21.15.5:0/3556690489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x557085839dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70e59c700 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085839890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 -- 172.21.15.5:0/3556690489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 msgr2=0x557085839dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x557085839dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 -- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 msgr2=0x557085839890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085839890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 -- 172.21.15.5:0/3556690489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557085704710 con 0x557085836060 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708583a310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7080dfab0 tx=0x7fc708005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150fcbe12be5aaf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.846+0000 7fc6f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/3556690489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080a95d0 con 0x557085836060 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/3556690489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fc7080e20b0 con 0x557085836060 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc704005580 con 0x557085836060 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/3556690489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7080ad980 con 0x557085836060 2024-05-14T15:24:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 10 client.15216 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5570858476b0 con 0x557085836060 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/3556690489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc704005580 con 0x557085836060 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/3556690489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7080b1fa0 con 0x557085836060 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 10 client.15216 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 20 client.15216 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557085849cc0 0x55708584b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 4 client.15216 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.847+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55708575b2d0 con 0x557085849cc0 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.848+0000 7fc70e59c700 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557085849cc0 0x55708584b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.848+0000 7fc70e59c700 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557085849cc0 0x55708584b130 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af7b715ee4c81db server_cookie=20ac1616010c492c in_seq=0 out_seq=0 2024-05-14T15:24:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.848+0000 7fc6f5ffb700 10 client.15216 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/3556690489 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55708575b2d0 con 0x557085849cc0 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc6f5ffb700 10 client.15216 handle_command_reply: tid=0 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 client.15216 shutdown 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 msgr2=0x55708583a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708583a310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7080dfab0 tx=0x7fc708005700 comp rx=0 tx=0).stop 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 shutdown_connections 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557085849cc0 0x55708584b130 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557085836060 0x55708583a310 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x557085839890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc6f5ffb700 0 client.15216 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:40.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3556690489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570858301d0 0x557085839dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 >> 172.21.15.5:0/3556690489 conn(0x5570856f55e0 msgr2=0x55708582a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:40.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 shutdown_connections 2024-05-14T15:24:40.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.849+0000 7fc7235efb80 1 -- 172.21.15.5:0/3556690489 wait complete. 2024-05-14T15:24:40.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.850+0000 7fc7235efb80 20 client.15216 trim_cache size 0 max 16384 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 Processor -- start 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 -- start start 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 0x55708584a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570858301d0 0x557085835160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x55708583bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557085849470 con 0x5570856addf0 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557085849b40 con 0x5570858301d0 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc7235efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55708584a210 con 0x557085836060 2024-05-14T15:24:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 0x55708584a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 0x55708584a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35114/0 (socket says 172.21.15.5:35114) 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 -- 172.21.15.5:0/3435153564 learned_addr learned my addr 172.21.15.5:0/3435153564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570858301d0 0x557085835160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 -- 172.21.15.5:0/3435153564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 msgr2=0x55708583bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x55708583bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x55708583bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 -- 172.21.15.5:0/3435153564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570858301d0 msgr2=0x557085835160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.871+0000 7fc70e59c700 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570858301d0 0x557085835160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.872+0000 7fc70e59c700 1 -- 172.21.15.5:0/3435153564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557085837e20 con 0x5570856addf0 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.872+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570858301d0 0x557085835160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.872+0000 7fc70e59c700 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 0x55708584a390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc700009690 tx=0x7fc700013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629415d945fe0916 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.872+0000 7fc6f7fff700 1 -- 172.21.15.5:0/3435153564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70001c040 con 0x5570856addf0 2024-05-14T15:24:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.872+0000 7fc6f7fff700 1 -- 172.21.15.5:0/3435153564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70002a7f0 con 0x5570856addf0 2024-05-14T15:24:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.872+0000 7fc6f7fff700 1 -- 172.21.15.5:0/3435153564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700015a30 con 0x5570856addf0 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 -- 172.21.15.5:0/3435153564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 msgr2=0x55708584a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 0x55708584a390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc700009690 tx=0x7fc700013000 comp rx=0 tx=0).stop 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 -- 172.21.15.5:0/3435153564 shutdown_connections 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x55708583bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570856addf0 0x55708584a390 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 --2- 172.21.15.5:0/3435153564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570858301d0 0x557085835160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 -- 172.21.15.5:0/3435153564 >> 172.21.15.5:0/3435153564 conn(0x5570856f55e0 msgr2=0x557085834a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 -- 172.21.15.5:0/3435153564 shutdown_connections 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 -- 172.21.15.5:0/3435153564 wait complete. 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 -- start start 2024-05-14T15:24:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.873+0000 7fc7235efb80 1 Processor -- start 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x5570858549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 0x5570858391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x557085839730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55708584afb0 con 0x5570858301d0 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55708584a8e0 con 0x5570856addf0 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55708584b680 con 0x557085836060 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc7235efb80 10 client.0 _ll_register_callbacks cb 0x5570856e5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 0x5570858391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 0x5570858391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35116/0 (socket says 172.21.15.5:35116) 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 -- 172.21.15.5:0/2060689828 learned_addr learned my addr 172.21.15.5:0/2060689828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x557085839730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70e59c700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x5570858549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 -- 172.21.15.5:0/2060689828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 msgr2=0x557085839730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x557085839730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 -- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 msgr2=0x5570858549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x5570858549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 -- 172.21.15.5:0/2060689828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557085839d70 con 0x5570858301d0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70e59c700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x5570858549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70ed9d700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x557085839730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc70dd9b700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 0x5570858391f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc704009d10 tx=0x7fc704003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72eae108690e2414 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc6f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.874+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/2060689828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704031070 con 0x5570858301d0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/2060689828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fc7040054e0 con 0x5570858301d0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70002a7f0 con 0x5570858301d0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/2060689828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc70403e050 con 0x5570858301d0 2024-05-14T15:24:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 10 client.15222 fetch_fsmap learned FSMap version 25 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557085853290 con 0x5570858301d0 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/2060689828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70002a7f0 con 0x5570858301d0 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/2060689828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc70400f6d0 con 0x5570858301d0 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 10 client.15222 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 20 client.15222 populate_metadata read hostname 'smithi005' 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5570858b6da0 0x5570858b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 4 client.15222 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.875+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "status"}) -- 0x557085755370 con 0x5570858b6da0 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc70e59c700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5570858b6da0 0x5570858b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc6f5ffb700 10 client.15222 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc70e59c700 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5570858b6da0 0x5570858b8210 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889b0645d107b841 server_cookie=d3d7202fd1ab47a9 in_seq=0 out_seq=0 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc6f5ffb700 1 -- 172.21.15.5:0/2060689828 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557085755370 con 0x5570858b6da0 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc6f5ffb700 10 client.15222 handle_command_reply: tid=0 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc7235efb80 1 client.15222 shutdown 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 msgr2=0x5570858391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.876+0000 7fc7235efb80 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 0x5570858391f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc704009d10 tx=0x7fc704003c30 comp rx=0 tx=0).stop 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 shutdown_connections 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5570858b6da0 0x5570858b8210 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc6f5ffb700 0 client.15222 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557085836060 0x557085839730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570856addf0 0x5570858549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 --2- 172.21.15.5:0/2060689828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570858301d0 0x5570858391f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 >> 172.21.15.5:0/2060689828 conn(0x5570856f55e0 msgr2=0x557085826300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 shutdown_connections 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 1 -- 172.21.15.5:0/2060689828 wait complete. 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc7235efb80 20 client.15222 trim_cache size 0 max 16384 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "4a7148fe-1204-11ef-bc98-c7b262605968", 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24515, 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-14T15:24:40.884 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-14T15:24:40.885 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 222.95800280700001, 2024-05-14T15:24:40.885 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-05-14T15:24:40.885 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-14T15:24:40.885 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-05-14T15:24:40.885 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 240.50900603599999 2024-05-14T15:24:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc6f80b2880 msgr2=0x7fc6f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc714bdb700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc6f80b2880 0x7fc6f80b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc700009550 tx=0x7fc7000091b0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 msgr2=0x7fc71019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.877+0000 7fc714bdb700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 0x7fc71019c720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d980 tx=0x7fc70800dc90 comp rx=0 tx=0).stop 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 shutdown_connections 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc6f80b2880 0x7fc6f80b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc710108770 0x7fc71019c720 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710103540 0x7fc71019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 --2- 172.21.15.5:0/300789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc710107e20 0x7fc71019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 >> 172.21.15.5:0/300789835 conn(0x7fc710078e50 msgr2=0x7fc710100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 shutdown_connections 2024-05-14T15:24:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:40.878+0000 7fc714bdb700 1 -- 172.21.15.5:0/300789835 wait complete. 2024-05-14T15:24:41.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:40 smithi144 ceph-mon[35650]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:41.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:40 smithi193 ceph-mon[36001]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:24:41.603 INFO:teuthology.run_tasks:Running task sequential... 2024-05-14T15:24:41.613 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-14T15:24:41.613 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-14T15:24:41.613 INFO:teuthology.task.print:Enabling mgr modules 2024-05-14T15:24:41.613 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-14T15:24:41.614 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T15:24:41.614 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:24:41.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-14T15:24:41.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 -- 172.21.15.5:0/2954386591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54102b50 msgr2=0x7f2c54103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/2954386591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54102b50 0x7f2c54103850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009900 tx=0x7f2c4402a410 comp rx=0 tx=0).stop 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 -- 172.21.15.5:0/2954386591 shutdown_connections 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/2954386591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c54108a90 0x7f2c5410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/2954386591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c54103d90 0x7f2c541041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/2954386591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54102b50 0x7f2c54103850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.733+0000 7f2c5aa53640 1 -- 172.21.15.5:0/2954386591 >> 172.21.15.5:0/2954386591 conn(0x7f2c540fdd40 msgr2=0x7f2c541001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.734+0000 7f2c5aa53640 1 -- 172.21.15.5:0/2954386591 shutdown_connections 2024-05-14T15:24:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.734+0000 7f2c5aa53640 1 -- 172.21.15.5:0/2954386591 wait complete. 2024-05-14T15:24:41.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.734+0000 7f2c5aa53640 1 Processor -- start 2024-05-14T15:24:41.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 -- start start 2024-05-14T15:24:41.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c54102b50 0x7f2c5406af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 0x7f2c5419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c54108a90 0x7f2c541a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410e650 con 0x7f2c54103d90 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410e4d0 con 0x7f2c54108a90 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c5aa53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410e7d0 con 0x7f2c54102b50 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 0x7f2c5419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 0x7f2c5419f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35124/0 (socket says 172.21.15.5:35124) 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 -- 172.21.15.5:0/1880948642 learned_addr learned my addr 172.21.15.5:0/1880948642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c53fff640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c54102b50 0x7f2c5406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c58ff0640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c54108a90 0x7f2c541a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 -- 172.21.15.5:0/1880948642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c54102b50 msgr2=0x7f2c5406af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c54102b50 0x7f2c5406af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 -- 172.21.15.5:0/1880948642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c54108a90 msgr2=0x7f2c541a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c54108a90 0x7f2c541a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.735+0000 7f2c537fe640 1 -- 172.21.15.5:0/1880948642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c541a1ea0 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.736+0000 7f2c537fe640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 0x7f2c5419f180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e9f0 tx=0x7f2c4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.736+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800cd80 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.736+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c480108e0 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.736+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541a2190 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.736+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48018980 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.736+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a26e0 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.737+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c541078a0 con 0x7f2c54103d90 2024-05-14T15:24:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.739+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2c48010430 con 0x7f2c54103d90 2024-05-14T15:24:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.740+0000 7f2c517fa640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c3c0b27e0 0x7f2c3c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.740+0000 7f2c53fff640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c3c0b27e0 0x7f2c3c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.740+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f2c480e2e30 con 0x7f2c54103d90 2024-05-14T15:24:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.740+0000 7f2c53fff640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c3c0b27e0 0x7f2c3c0b4cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005ab0 tx=0x7f2c44031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:41.744+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c480aa780 con 0x7f2c54103d90 2024-05-14T15:24:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.007+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f2c540638d0 con 0x7f2c54103d90 2024-05-14T15:24:42.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.812+0000 7f2c517fa640 1 -- 172.21.15.5:0/1880948642 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f2c480aa080 con 0x7f2c54103d90 2024-05-14T15:24:42.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.816+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c3c0b27e0 msgr2=0x7f2c3c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:42.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.816+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c3c0b27e0 0x7f2c3c0b4cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005ab0 tx=0x7f2c44031400 comp rx=0 tx=0).stop 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.816+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 msgr2=0x7f2c5419f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.816+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 0x7f2c5419f180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e9f0 tx=0x7f2c4800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 shutdown_connections 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f2c3c0b27e0 0x7f2c3c0b4cf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c54108a90 0x7f2c541a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c54103d90 0x7f2c5419f180 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 --2- 172.21.15.5:0/1880948642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c54102b50 0x7f2c5406af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 >> 172.21.15.5:0/1880948642 conn(0x7f2c540fdd40 msgr2=0x7f2c54104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 shutdown_connections 2024-05-14T15:24:42.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:42.817+0000 7f2c5aa53640 1 -- 172.21.15.5:0/1880948642 wait complete. 2024-05-14T15:24:42.839 INFO:teuthology.run_tasks:Running task kclient... 2024-05-14T15:24:42.852 INFO:tasks.kclient:Mounting kernel clients... 2024-05-14T15:24:42.852 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-14T15:24:42.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:42.852 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-14T15:24:42.872 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-14T15:24:42.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:42.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-14T15:24:42.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:42.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-14T15:24:43.051 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-14T15:24:43.051 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T15:24:43.051 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T15:24:43.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T15:24:43.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:42 smithi144 ceph-mon[35650]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:43.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:42 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1880948642' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T15:24:43.210 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: ignoring --setuser ceph since I am not root 2024-05-14T15:24:43.210 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: ignoring --setgroup ceph since I am not root 2024-05-14T15:24:43.210 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:42 smithi144 ceph-mgr[37041]: -- 172.21.15.144:0/1534882902 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b52bcc5a0 con 0x558b57956000 2024-05-14T15:24:43.210 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:43.019+0000 7fc964fc6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:24:43.211 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:43.208+0000 7fc964fc6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:24:43.215 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:42 smithi193 ceph-mon[36001]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:43.215 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:42 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1880948642' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T15:24:43.215 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: ignoring --setuser ceph since I am not root 2024-05-14T15:24:43.216 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: ignoring --setgroup ceph since I am not root 2024-05-14T15:24:43.216 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:42 smithi193 ceph-mgr[37862]: -- 172.21.15.193:0/2890709641 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56287c41a5a0 con 0x5628811a4000 2024-05-14T15:24:43.216 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:43.025+0000 7f85226b2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:24:43.216 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:43.214+0000 7f85226b2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:24:43.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:42 smithi005 ceph-mon[31559]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T15:24:43.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:42 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1880948642' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T15:24:43.261 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: ignoring --setuser ceph since I am not root 2024-05-14T15:24:43.261 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: ignoring --setgroup ceph since I am not root 2024-05-14T15:24:43.261 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:43.052+0000 7fbca477d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T15:24:43.261 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:43.260+0000 7fbca477d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:24:43.307 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:43.547 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:43.311+0000 7fc964fc6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:24:43.553 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:43.318+0000 7f85226b2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:24:43.565 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:43.378+0000 7fbca477d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T15:24:43.835 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:43.712+0000 7fbca477d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:24:43.835 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:43.806+0000 7fbca477d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:24:44.047 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:43.619+0000 7fc964fc6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:24:44.047 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:43.705+0000 7fc964fc6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:24:44.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:43 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1880948642' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T15:24:44.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:43 smithi144 ceph-mon[35650]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-14T15:24:44.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:43 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1880948642' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T15:24:44.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:43 smithi193 ceph-mon[36001]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-14T15:24:44.053 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:43.631+0000 7f85226b2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T15:24:44.053 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:43.718+0000 7f85226b2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T15:24:44.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:43 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1880948642' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T15:24:44.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:43 smithi005 ceph-mon[31559]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 -- 172.21.15.5:0/3336887378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6110108740 msgr2=0x7f61101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 --2- 172.21.15.5:0/3336887378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6110108740 0x7f61101100a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f610800b0d0 tx=0x7f610800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 -- 172.21.15.5:0/3336887378 shutdown_connections 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 --2- 172.21.15.5:0/3336887378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6110108740 0x7f61101100a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 --2- 172.21.15.5:0/3336887378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61101034c0 0x7f61101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 --2- 172.21.15.5:0/3336887378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110107df0 0x7f6110108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:44.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 -- 172.21.15.5:0/3336887378 >> 172.21.15.5:0/3336887378 conn(0x7f61100fedc0 msgr2=0x7f61101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:44.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 -- 172.21.15.5:0/3336887378 shutdown_connections 2024-05-14T15:24:44.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.981+0000 7f611494f700 1 -- 172.21.15.5:0/3336887378 wait complete. 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 Processor -- start 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 -- start start 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 0x7f611019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6110107df0 0x7f611019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110108740 0x7f611019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6110113370 con 0x7f61101034c0 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61101131f0 con 0x7f6110107df0 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f611494f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f61101134f0 con 0x7f6110108740 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f610e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 0x7f611019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.982+0000 7f610e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 0x7f611019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35164/0 (socket says 172.21.15.5:35164) 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 -- 172.21.15.5:0/772682538 learned_addr learned my addr 172.21.15.5:0/772682538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610dd9b700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6110107df0 0x7f611019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610ed9d700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110108740 0x7f611019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:44.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 -- 172.21.15.5:0/772682538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110108740 msgr2=0x7f611019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110108740 0x7f611019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 -- 172.21.15.5:0/772682538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6110107df0 msgr2=0x7f611019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6110107df0 0x7f611019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 -- 172.21.15.5:0/772682538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611019fc20 con 0x7f61101034c0 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610dd9b700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6110107df0 0x7f611019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f610e59c700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 0x7f611019ad50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f610000e9a0 tx=0x7f610000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000ca10 con 0x7f61101034c0 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610000cbb0 con 0x7f61101034c0 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611019ff10 con 0x7f61101034c0 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100018680 con 0x7f61101034c0 2024-05-14T15:24:44.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.983+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61101a7830 con 0x7f61101034c0 2024-05-14T15:24:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.985+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61100512c0 con 0x7f61101034c0 2024-05-14T15:24:44.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.986+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f6100010430 con 0x7f61101034c0 2024-05-14T15:24:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.987+0000 7f61077fe700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 0x7f60fc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.987+0000 7f610dd9b700 1 -- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 msgr2=0x7f60fc0b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.988+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f61000e2ab0 con 0x7f61101034c0 2024-05-14T15:24:44.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.988+0000 7f610dd9b700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 0x7f60fc0b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:24:44.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:44.991+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61000aa3c0 con 0x7f61101034c0 2024-05-14T15:24:45.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.189+0000 7f610dd9b700 1 -- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 msgr2=0x7f60fc0b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:45.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.189+0000 7f610dd9b700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 0x7f60fc0b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:24:45.235 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:44.883+0000 7fc964fc6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:24:45.235 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:45.059+0000 7fc964fc6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:24:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.254+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f611019c0e0 con 0x7f61101034c0 2024-05-14T15:24:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.254+0000 7f61077fe700 1 -- 172.21.15.5:0/772682538 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f611019c0e0 con 0x7f61101034c0 2024-05-14T15:24:45.255 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-14T15:24:45.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 msgr2=0x7f60fc0b4e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:24:45.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 0x7f60fc0b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 msgr2=0x7f611019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 0x7f611019ad50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f610000e9a0 tx=0x7f610000ecb0 comp rx=0 tx=0).stop 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 shutdown_connections 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f60fc0b2950 0x7f60fc0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6110108740 0x7f611019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61101034c0 0x7f611019ad50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 --2- 172.21.15.5:0/772682538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6110107df0 0x7f611019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 >> 172.21.15.5:0/772682538 conn(0x7f61100fedc0 msgr2=0x7f611006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 shutdown_connections 2024-05-14T15:24:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:45.257+0000 7f611494f700 1 -- 172.21.15.5:0/772682538 wait complete. 2024-05-14T15:24:45.262 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:44 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:44.910+0000 7f85226b2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:24:45.262 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:45.086+0000 7f85226b2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:24:45.262 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:45.261+0000 7f85226b2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:24:45.512 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:45 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/772682538' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T15:24:45.512 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:45.132+0000 7fbca477d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T15:24:45.512 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:45.328+0000 7fbca477d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T15:24:45.512 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:45.511+0000 7fbca477d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:24:45.527 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:45.233+0000 7fc964fc6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T15:24:45.527 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:45 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/772682538' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T15:24:45.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:45 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/772682538' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T15:24:45.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:45.526+0000 7fc964fc6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:24:45.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:45.620+0000 7fc964fc6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:24:45.860 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:45.556+0000 7f85226b2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:24:45.860 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:45.650+0000 7f85226b2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:24:45.860 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:45.859+0000 7f85226b2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:24:45.914 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T15:24:45.914 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T15:24:45.914 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-14T15:24:45.914 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T15:24:45.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-14T15:24:45.914 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T15:24:45.915 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T15:24:45.915 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T15:24:45.915 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-14T15:24:45.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:45.915 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-14T15:24:45.932 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-14T15:24:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42016sec preferred_lft 42016sec 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-14T15:24:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-14T15:24:45.934 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T15:24:45.934 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:24:45.934 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:24:45.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-05-14T15:24:45.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-05-14T15:24:45.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-05-14T15:24:45.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T15:24:45.935 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:24:46.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:24:46.075 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:45.824+0000 7fbca477d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T15:24:46.075 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:45.928+0000 7fbca477d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T15:24:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:46.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:46.085 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T15:24:46.110 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-14T15:24:46.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:46.112 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-14T15:24:46.169 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-14T15:24:46.170 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T15:24:46.170 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-14T15:24:46.170 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-14T15:24:46.170 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T15:24:46.170 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:24:46.170 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:24:46.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-14T15:24:46.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-14T15:24:46.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-14T15:24:46.171 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:24:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:24:46.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:46.297 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:45.827+0000 7fc964fc6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:24:46.297 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:46.022+0000 7fc964fc6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:24:46.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:46.303 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-14T15:24:46.304 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:46.054+0000 7f85226b2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:24:46.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:46.321 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-14T15:24:46.367 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:46.151+0000 7fbca477d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T15:24:46.367 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:46.366+0000 7fbca477d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T15:24:46.377 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:24:46.378 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:24:46.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T15:24:46.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-14T15:24:46.378 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:24:46.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:24:46.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:46.499 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-14T15:24:46.499 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:24:46.499 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:24:46.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-14T15:24:46.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T15:24:46.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-14T15:24:46.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-14T15:24:46.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-14T15:24:46.500 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:24:46.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:24:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:46.621 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T15:24:46.621 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-14T15:24:46.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-14T15:24:46.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T15:24:46.621 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-14T15:24:46.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:24:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:46.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T15:24:46.711 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-14T15:24:46.737 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-14T15:24:46.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T15:24:46.738 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-14T15:24:46.793 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-14T15:24:46.793 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-14T15:24:46.793 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/a7c2309d-a418-4b06-b5dd-d00a7f41b7c3 2024-05-14T15:24:46.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:46.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/a7c2309d-a418-4b06-b5dd-d00a7f41b7c3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-05-14T15:24:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-05-14T15:24:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-05-14T15:24:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-14T15:24:47.010 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-05-14T15:24:47.010 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-05-14T15:24:47.010 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-14T15:24:47.010 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.144:6789,172.21.15.193:6789" 2024-05-14T15:24:47.010 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.144:6789,172.21.15.193:6789:/volumes/qa/sv_1/a7c2309d-a418-4b06-b5dd-d00a7f41b7c3 2024-05-14T15:24:47.011 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=4a7148fe-1204-11ef-bc98-c7b262605968" will pass to kernel 2024-05-14T15:24:47.013 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-14T15:24:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-14T15:24:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-14T15:24:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-14T15:24:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-14T15:24:47.014 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-14T15:24:47.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:24:47.014 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-14T15:24:47.046 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T15:24:47.047 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:46.625+0000 7fc964fc6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:24:47.047 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:46.848+0000 7fc964fc6200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:24:47.053 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:46.662+0000 7f85226b2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:24:47.053 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:46.888+0000 7f85226b2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:24:47.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T15:24:47.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:47.147 INFO:teuthology.orchestra.run.smithi005.stdout:client24991 2024-05-14T15:24:47.147 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:24:47.147 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T15:24:47.164 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T15:24:47.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T15:24:47.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:47.263 INFO:teuthology.orchestra.run.smithi005.stdout:client24991 2024-05-14T15:24:47.263 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T15:24:47.278 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:47.033+0000 7fbca477d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T15:24:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T15:24:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:47.322 INFO:teuthology.orchestra.run.smithi005.stdout:client24991 2024-05-14T15:24:47.322 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4a7148fe-1204-11ef-bc98-c7b262605968.client24991/status') 2024-05-14T15:24:47.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T15:24:47.424 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.24991 (3)172.21.15.5:0/3902307168 2024-05-14T15:24:47.424 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-05-14T15:24:47.424 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-05-14T15:24:47.424 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-05-14T15:24:47.424 INFO:teuthology.orchestra.run.smithi005.stderr:70 bytes copied, 2.5751e-05 s, 2.7 MB/s 2024-05-14T15:24:47.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T15:24:47.432 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T15:24:47.442 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T15:24:47.443 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:24:47.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-14T15:24:47.547 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:47.277+0000 7fc964fc6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:24:47.547 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:47.375+0000 7fc964fc6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.559+0000 7f859fe53640 1 -- 172.21.15.5:0/48397738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598102b50 msgr2=0x7f8598103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 --2- 172.21.15.5:0/48397738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598102b50 0x7f8598103850 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8594009900 tx=0x7f859402a410 comp rx=0 tx=0).stop 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 -- 172.21.15.5:0/48397738 shutdown_connections 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 --2- 172.21.15.5:0/48397738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8598103d90 0x7f85981041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 --2- 172.21.15.5:0/48397738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598108a90 0x7f859810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 --2- 172.21.15.5:0/48397738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598102b50 0x7f8598103850 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 -- 172.21.15.5:0/48397738 >> 172.21.15.5:0/48397738 conn(0x7f85980fdd40 msgr2=0x7f85981001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 -- 172.21.15.5:0/48397738 shutdown_connections 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 -- 172.21.15.5:0/48397738 wait complete. 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 Processor -- start 2024-05-14T15:24:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.560+0000 7f859fe53640 1 -- start start 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859fe53640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598102b50 0x7f859806af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859fe53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8598103d90 0x7f859819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859fe53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 0x7f85981a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859fe53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f859810e650 con 0x7f8598108a90 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859fe53640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f859810e4d0 con 0x7f8598102b50 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859fe53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f859810e7d0 con 0x7f8598103d90 2024-05-14T15:24:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 0x7f85981a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 0x7f85981a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51196/0 (socket says 172.21.15.5:51196) 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 -- 172.21.15.5:0/1271592964 learned_addr learned my addr 172.21.15.5:0/1271592964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859dbef640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598102b50 0x7f859806af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859d3ee640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8598103d90 0x7f859819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 -- 172.21.15.5:0/1271592964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8598103d90 msgr2=0x7f859819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8598103d90 0x7f859819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 -- 172.21.15.5:0/1271592964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598102b50 msgr2=0x7f859806af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598102b50 0x7f859806af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 -- 172.21.15.5:0/1271592964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85981a1ec0 con 0x7f8598108a90 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f859e3f0640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 0x7f85981a16a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f859000eb10 tx=0x7f859000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.561+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590004b20 con 0x7f8598108a90 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.562+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8590004cc0 con 0x7f8598108a90 2024-05-14T15:24:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.562+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590010670 con 0x7f8598108a90 2024-05-14T15:24:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.562+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981a21b0 con 0x7f8598108a90 2024-05-14T15:24:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.562+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981a2700 con 0x7f8598108a90 2024-05-14T15:24:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.563+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85981078a0 con 0x7f8598108a90 2024-05-14T15:24:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.564+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f8590005000 con 0x7f8598108a90 2024-05-14T15:24:47.566 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:47.278+0000 7fbca477d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T15:24:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.565+0000 7f858effd640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 0x7f85840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.565+0000 7f859dbef640 1 -- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 msgr2=0x7f85840b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.566+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7f85900e2d60 con 0x7f8598108a90 2024-05-14T15:24:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.566+0000 7f859dbef640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 0x7f85840b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:24:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.569+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8590054df0 con 0x7f8598108a90 2024-05-14T15:24:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.767+0000 7f859dbef640 1 -- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 msgr2=0x7f85840b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:47.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.767+0000 7f859dbef640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 0x7f85840b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:24:47.792 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:47.317+0000 7f85226b2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:24:47.792 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:47.415+0000 7f85226b2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:24:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.804+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f859802dc30 con 0x7f8598108a90 2024-05-14T15:24:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.857+0000 7f858effd640 1 -- 172.21.15.5:0/1271592964 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f85900aa6b0 con 0x7f8598108a90 2024-05-14T15:24:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 msgr2=0x7f85840b4dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 0x7f85840b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 msgr2=0x7f85981a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 0x7f85981a16a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f859000eb10 tx=0x7f859000c2d0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 shutdown_connections 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7f85840b28b0 0x7f85840b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8598103d90 0x7f859819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8598108a90 0x7f85981a16a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 --2- 172.21.15.5:0/1271592964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8598102b50 0x7f859806af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 >> 172.21.15.5:0/1271592964 conn(0x7f85980fdd40 msgr2=0x7f8598104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 shutdown_connections 2024-05-14T15:24:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.859+0000 7f859fe53640 1 -- 172.21.15.5:0/1271592964 wait complete. 2024-05-14T15:24:47.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-14T15:24:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1366264252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508108ac0 msgr2=0x7fc50810aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 --2- 172.21.15.5:0/1366264252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508108ac0 0x7fc50810aee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc500009ca0 tx=0x7fc50002a8c0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1366264252 shutdown_connections 2024-05-14T15:24:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 --2- 172.21.15.5:0/1366264252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc508103dc0 0x7fc5081041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 --2- 172.21.15.5:0/1366264252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508108ac0 0x7fc50810aee0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 --2- 172.21.15.5:0/1366264252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc508102b80 0x7fc508103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1366264252 >> 172.21.15.5:0/1366264252 conn(0x7fc5080fdd90 msgr2=0x7fc508100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1366264252 shutdown_connections 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.994+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1366264252 wait complete. 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 Processor -- start 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 -- start start 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc508102b80 0x7fc50806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508103dc0 0x7fc50819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc508108ac0 0x7fc5081a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810e6b0 con 0x7fc508103dc0 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810e530 con 0x7fc508102b80 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50e2d8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810e830 con 0x7fc508108ac0 2024-05-14T15:24:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508103dc0 0x7fc50819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc4fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508103dc0 0x7fc50819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51202/0 (socket says 172.21.15.5:51202) 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc4fffff640 1 -- 172.21.15.5:0/1674957477 learned_addr learned my addr 172.21.15.5:0/1674957477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc508102b80 0x7fc50806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.995+0000 7fc50c875640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc508108ac0 0x7fc5081a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc4fffff640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc508108ac0 msgr2=0x7fc5081a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc4fffff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc508108ac0 0x7fc5081a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc4fffff640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc508102b80 msgr2=0x7fc50806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc4fffff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc508102b80 0x7fc50806bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc4fffff640 1 -- 172.21.15.5:0/1674957477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081a1e50 con 0x7fc508103dc0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc508102b80 0x7fc50806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc4fffff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508103dc0 0x7fc50819f0e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9f0 tx=0x7fc4f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000cd80 con 0x7fc508103dc0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081a2140 con 0x7fc508103dc0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f00054d0 con 0x7fc508103dc0 2024-05-14T15:24:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0010690 con 0x7fc508103dc0 2024-05-14T15:24:48.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.996+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a2690 con 0x7fc508103dc0 2024-05-14T15:24:48.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:47.997+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5081078d0 con 0x7fc508103dc0 2024-05-14T15:24:48.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.000+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7fc4f0004e30 con 0x7fc508103dc0 2024-05-14T15:24:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.001+0000 7fc505ffb640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 0x7fc4f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.001+0000 7fc507fff640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 msgr2=0x7fc4f40b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.001+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9216+0+0 (secure 0 0 0) 0x7fc4f00e2c50 con 0x7fc508103dc0 2024-05-14T15:24:48.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.001+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 0x7fc4f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T15:24:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.004+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4f0054df0 con 0x7fc508103dc0 2024-05-14T15:24:48.047 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:47.749+0000 7fc964fc6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:24:48.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:47 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1271592964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-14T15:24:48.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:47 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1271592964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-14T15:24:48.053 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:47.791+0000 7f85226b2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:24:48.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:47 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1271592964' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-14T15:24:48.065 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:47.750+0000 7fbca477d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T15:24:48.065 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:47.855+0000 7fbca477d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T15:24:48.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.202+0000 7fc507fff640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 msgr2=0x7fc4f40b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:48.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.202+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 0x7fc4f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T15:24:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.227+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 --> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fc508063110 con 0x7fc4f40b2830 2024-05-14T15:24:48.453 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:48.113+0000 7fc964fc6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:24:48.453 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:48.208+0000 7fc964fc6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:24:48.500 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:48.157+0000 7f85226b2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:24:48.500 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:48.252+0000 7f85226b2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:24:48.565 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:48.263+0000 7fbca477d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T15:24:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.603+0000 7fc507fff640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 msgr2=0x7fc4f40b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:48.603+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 0x7fc4f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T15:24:48.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:48.451+0000 7fc964fc6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:24:48.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:48.586+0000 7fc964fc6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:24:48.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:48.682+0000 7fc964fc6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:24:48.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:48.499+0000 7f85226b2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:24:48.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:48.634+0000 7f85226b2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:24:48.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:48.732+0000 7f85226b2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:24:49.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:48 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1271592964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-14T15:24:49.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:48 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:24:49.016 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:48.648+0000 7fbca477d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T15:24:49.016 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:48.750+0000 7fbca477d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T15:24:49.016 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:49.014+0000 7fbca477d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T15:24:49.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:48 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1271592964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-14T15:24:49.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:48 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:24:49.165 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:49.068+0000 7f85226b2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:24:49.165 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:49.164+0000 7f85226b2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:24:49.267 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:49.160+0000 7fbca477d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T15:24:49.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:48 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1271592964' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-14T15:24:49.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:48 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:24:49.297 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:49.014+0000 7fc964fc6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:24:49.297 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:49.109+0000 7fc964fc6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:24:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:49.404+0000 7fc507fff640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 msgr2=0x7fc4f40b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/887023568 2024-05-14T15:24:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:49.404+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 0x7fc4f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T15:24:49.565 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:49.267+0000 7fbca477d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T15:24:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:49.996+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 20) ==== 149668+0+0 (secure 0 0 0) 0x7fc4f00a6130 con 0x7fc508103dc0 2024-05-14T15:24:49.998 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:49.712+0000 7fc964fc6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:24:49.998 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:24:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y[37037]: 2024-05-14T15:24:49.930+0000 7fc964fc6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:24:49.998 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:49.611+0000 7fbca477d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T15:24:49.998 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:49.713+0000 7fbca477d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T15:24:50.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:49 smithi193 ceph-mon[36001]: Standby manager daemon y restarted 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:49 smithi193 ceph-mon[36001]: Standby manager daemon y started 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:49 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:49 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:49 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:49 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:49.768+0000 7f85226b2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:24:50.054 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:24:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z[37858]: 2024-05-14T15:24:49.986+0000 7f85226b2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:24:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:49 smithi144 ceph-mon[35650]: Standby manager daemon y restarted 2024-05-14T15:24:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:49 smithi144 ceph-mon[35650]: Standby manager daemon y started 2024-05-14T15:24:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:49 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T15:24:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:49 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:24:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:49 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T15:24:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:49 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:24:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:49 smithi005 ceph-mon[31559]: Standby manager daemon y restarted 2024-05-14T15:24:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:49 smithi005 ceph-mon[31559]: Standby manager daemon y started 2024-05-14T15:24:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:49 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T15:24:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:49 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:24:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:49 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T15:24:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:49 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.144:0/255299517' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:24:50.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:50.624+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7fc4f00e0400 con 0x7fc508103dc0 2024-05-14T15:24:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:50.624+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 msgr2=0x7fc4f40b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:24:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:50.624+0000 7fc505ffb640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/887023568,v1:172.21.15.5:6801/887023568] conn(0x7fc4f40b2830 0x7fc4f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:50.801 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.369+0000 7fbca477d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T15:24:50.801 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.609+0000 7fbca477d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: Standby manager daemon z restarted 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: Standby manager daemon z started 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:24:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: Active manager daemon x restarted 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: Activating manager daemon x 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: mgrmap e21: x(active, starting, since 0.0140361s), standbys: y, z 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:24:51.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:24:51.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: Manager daemon x is now available 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T15:24:51.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:51 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T15:24:51.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-14T15:24:51.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: Standby manager daemon z restarted 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: Standby manager daemon z started 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: Active manager daemon x restarted 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: Activating manager daemon x 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: mgrmap e21: x(active, starting, since 0.0140361s), standbys: y, z 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T15:24:51.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T15:24:51.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc05d8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc05d8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc05d8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc05d8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc05d8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc02d89700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc02d89700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc02d89700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc02d89700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:50.881+0000 7fbc02d89700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: Manager daemon x is now available 2024-05-14T15:24:51.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:51.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T15:24:51.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:51 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T15:24:51.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: Standby manager daemon z restarted 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: Standby manager daemon z started 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.? 172.21.15.193:0/796408441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: Active manager daemon x restarted 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: Activating manager daemon x 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: mgrmap e21: x(active, starting, since 0.0140361s), standbys: y, z 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T15:24:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T15:24:51.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T15:24:51.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T15:24:51.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T15:24:51.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T15:24:51.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: Manager daemon x is now available 2024-05-14T15:24:51.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:51.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T15:24:51.301 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:51 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T15:24:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.632+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mon.0 v2:172.21.15.5:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fc4f00ba0f0 con 0x7fc508103dc0 2024-05-14T15:24:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.633+0000 7fc505ffb640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc4f4039fc0 0x7fc4f403a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.633+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fc4f401ea20 con 0x7fc4f4039fc0 2024-05-14T15:24:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.635+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc4f4039fc0 0x7fc4f403a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:51.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.638+0000 7fc507fff640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc4f4039fc0 0x7fc4f403a3d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc508103780 tx=0x7fc4f8005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:51.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.657+0000 7fc505ffb640 1 -- 172.21.15.5:0/1674957477 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fc4f401ea20 con 0x7fc4f4039fc0 2024-05-14T15:24:51.658 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-05-14T15:24:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc4f4039fc0 msgr2=0x7fc4f403a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50e2d8640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc4f4039fc0 0x7fc4f403a3d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc508103780 tx=0x7fc4f8005f90 comp rx=0 tx=0).stop 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508103dc0 msgr2=0x7fc50819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50e2d8640 1 --2- 172.21.15.5:0/1674957477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc508103dc0 0x7fc50819f0e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9f0 tx=0x7fc4f000edf0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50c875640 1 -- 172.21.15.5:0/1674957477 reap_dead start 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 shutdown_connections 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.659+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 >> 172.21.15.5:0/1674957477 conn(0x7fc5080fdd90 msgr2=0x7fc508104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.660+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 shutdown_connections 2024-05-14T15:24:51.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.660+0000 7fc50e2d8640 1 -- 172.21.15.5:0/1674957477 wait complete. 2024-05-14T15:24:51.672 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-14T15:24:51.683 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-14T15:24:51.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T15:24:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 -- 172.21.15.5:0/2973636242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4108a50 msgr2=0x7f1cf410ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 --2- 172.21.15.5:0/2973636242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4108a50 0x7f1cf410ae70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009ca0 tx=0x7f1cec02a8c0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 -- 172.21.15.5:0/2973636242 shutdown_connections 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 --2- 172.21.15.5:0/2973636242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cf4103d50 0x7f1cf4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 --2- 172.21.15.5:0/2973636242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4108a50 0x7f1cf410ae70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 --2- 172.21.15.5:0/2973636242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4102b10 0x7f1cf4103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 -- 172.21.15.5:0/2973636242 >> 172.21.15.5:0/2973636242 conn(0x7f1cf40fdd20 msgr2=0x7f1cf41001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 -- 172.21.15.5:0/2973636242 shutdown_connections 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 -- 172.21.15.5:0/2973636242 wait complete. 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 Processor -- start 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.821+0000 7f1cfb214640 1 -- start start 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cfb214640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 0x7f1cf419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cfb214640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cf4103d50 0x7f1cf419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cfb214640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108a50 0x7f1cf41a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cfb214640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410e640 con 0x7f1cf4102b10 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cfb214640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410e4c0 con 0x7f1cf4108a50 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cfb214640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410e7c0 con 0x7f1cf4103d50 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 0x7f1cf419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 0x7f1cf419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51348/0 (socket says 172.21.15.5:51348) 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 -- 172.21.15.5:0/981199854 learned_addr learned my addr 172.21.15.5:0/981199854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cf4103d50 0x7f1cf419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf97b1640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108a50 0x7f1cf41a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 -- 172.21.15.5:0/981199854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cf4103d50 msgr2=0x7f1cf419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cf4103d50 0x7f1cf419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 -- 172.21.15.5:0/981199854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108a50 msgr2=0x7f1cf41a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108a50 0x7f1cf41a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 -- 172.21.15.5:0/981199854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf41a1e50 con 0x7f1cf4102b10 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf8fb0640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 0x7f1cf419a310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9f0 tx=0x7f1ce800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800ccc0 con 0x7f1cf4102b10 2024-05-14T15:24:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce80035f0 con 0x7f1cf4102b10 2024-05-14T15:24:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.822+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800ce60 con 0x7f1cf4102b10 2024-05-14T15:24:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.823+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf41a2140 con 0x7f1cf4102b10 2024-05-14T15:24:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.823+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a2690 con 0x7f1cf4102b10 2024-05-14T15:24:51.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.824+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf41a2950 con 0x7f1cf4102b10 2024-05-14T15:24:51.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.825+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f1ce8003aa0 con 0x7f1cf4102b10 2024-05-14T15:24:51.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.826+0000 7f1cf1ffb640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1ce40b25e0 0x7f1ce40b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:51.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.826+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1ce40b25e0 0x7f1ce40b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:51.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.826+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f1ce80e2750 con 0x7f1cf4102b10 2024-05-14T15:24:51.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.827+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1ce40b25e0 0x7f1ce40b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cf419b3b0 tx=0x7f1ce0005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:51.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:51.830+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ce80aa280 con 0x7f1cf4102b10 2024-05-14T15:24:51.943 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:51.645+0000 7fbbffd83700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.943 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:51.645+0000 7fbbffd83700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.943 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:51.645+0000 7fbbffd83700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.943 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:51.645+0000 7fbbffd83700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:51.943 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:24:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:24:51.645+0000 7fbbffd83700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.023+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cf41a2b20 con 0x7f1cf4102b10 2024-05-14T15:24:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.025+0000 7f1cf1ffb640 1 -- 172.21.15.5:0/981199854 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25130 (secure 0 0 0) 0x7f1ce80a9b80 con 0x7f1cf4102b10 2024-05-14T15:24:52.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:52.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:24:50.611181+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2413445605":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/365942426":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/4280832550":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/3326284548":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1ce40b25e0 msgr2=0x7f1ce40b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1ce40b25e0 0x7f1ce40b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cf419b3b0 tx=0x7f1ce0005f90 comp rx=0 tx=0).stop 2024-05-14T15:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 msgr2=0x7f1cf419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 0x7f1cf419a310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9f0 tx=0x7f1ce800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 shutdown_connections 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1ce40b25e0 0x7f1ce40b4af0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cf4103d50 0x7f1cf419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1cf4108a50 0x7f1cf41a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 --2- 172.21.15.5:0/981199854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4102b10 0x7f1cf419a310 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.027+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 >> 172.21.15.5:0/981199854 conn(0x7f1cf40fdd20 msgr2=0x7f1cf4109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.028+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 shutdown_connections 2024-05-14T15:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.028+0000 7f1cfb214640 1 -- 172.21.15.5:0/981199854 wait complete. 2024-05-14T15:24:52.041 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T15:16:05.463462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T15:20:51.199481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T15:20:54.665379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-14T15:21:04.523784+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-14T15:24:52.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 -- 172.21.15.5:0/3740852501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc104350 msgr2=0x7fe5cc106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 --2- 172.21.15.5:0/3740852501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc104350 0x7fe5cc106770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009900 tx=0x7fe5bc02a410 comp rx=0 tx=0).stop 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 -- 172.21.15.5:0/3740852501 shutdown_connections 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 --2- 172.21.15.5:0/3740852501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc1098b0 0x7fe5cc10bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 --2- 172.21.15.5:0/3740852501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc106d40 0x7fe5cc109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 --2- 172.21.15.5:0/3740852501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc104350 0x7fe5cc106770 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 -- 172.21.15.5:0/3740852501 >> 172.21.15.5:0/3740852501 conn(0x7fe5cc0fdd30 msgr2=0x7fe5cc1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 -- 172.21.15.5:0/3740852501 shutdown_connections 2024-05-14T15:24:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.165+0000 7fe5d158d640 1 -- 172.21.15.5:0/3740852501 wait complete. 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 Processor -- start 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 -- start start 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc104350 0x7fe5cc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 0x7fe5cc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1098b0 0x7fe5cc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10f710 con 0x7fe5cc106d40 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10f590 con 0x7fe5cc1098b0 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5d158d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10f890 con 0x7fe5cc104350 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 0x7fe5cc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 0x7fe5cc19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51372/0 (socket says 172.21.15.5:51372) 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 -- 172.21.15.5:0/2603857654 learned_addr learned my addr 172.21.15.5:0/2603857654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5cb7fe640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1098b0 0x7fe5cc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5caffd640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc104350 0x7fe5cc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 -- 172.21.15.5:0/2603857654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc104350 msgr2=0x7fe5cc19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc104350 0x7fe5cc19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 -- 172.21.15.5:0/2603857654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1098b0 msgr2=0x7fe5cc1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1098b0 0x7fe5cc1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.166+0000 7fe5c3fff640 1 -- 172.21.15.5:0/2603857654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc1a1e50 con 0x7fe5cc106d40 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.167+0000 7fe5c3fff640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 0x7fe5cc19a850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400d950 tx=0x7fe5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.167+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4017070 con 0x7fe5cc106d40 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.167+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a2140 con 0x7fe5cc106d40 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.167+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5b4009c30 con 0x7fe5cc106d40 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.167+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400f620 con 0x7fe5cc106d40 2024-05-14T15:24:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.167+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a26e0 con 0x7fe5cc106d40 2024-05-14T15:24:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.168+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc0ff850 con 0x7fe5cc106d40 2024-05-14T15:24:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.169+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fe5b4010460 con 0x7fe5cc106d40 2024-05-14T15:24:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.170+0000 7fe5c8ff9640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5b80b25e0 0x7fe5b80b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.170+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fe5b40e1f00 con 0x7fe5cc106d40 2024-05-14T15:24:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.170+0000 7fe5caffd640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5b80b25e0 0x7fe5b80b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.171+0000 7fe5caffd640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5b80b25e0 0x7fe5b80b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc02a930 tx=0x7fe5bc005fd0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:52.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.174+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5b40a9a30 con 0x7fe5cc106d40 2024-05-14T15:24:52.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.367+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5cc0638d0 con 0x7fe5cc106d40 2024-05-14T15:24:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.367+0000 7fe5c8ff9640 1 -- 172.21.15.5:0/2603857654 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fe5b4009de0 con 0x7fe5cc106d40 2024-05-14T15:24:52.369 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-14T15:24:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5b80b25e0 msgr2=0x7fe5b80b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5b80b25e0 0x7fe5b80b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc02a930 tx=0x7fe5bc005fd0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 msgr2=0x7fe5cc19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 0x7fe5cc19a850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400d950 tx=0x7fe5b400de10 comp rx=0 tx=0).stop 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 shutdown_connections 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5b80b25e0 0x7fe5b80b4af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1098b0 0x7fe5cc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5cc106d40 0x7fe5cc19a850 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 --2- 172.21.15.5:0/2603857654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5cc104350 0x7fe5cc19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.369+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 >> 172.21.15.5:0/2603857654 conn(0x7fe5cc0fdd30 msgr2=0x7fe5cc106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.370+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 shutdown_connections 2024-05-14T15:24:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.370+0000 7fe5d158d640 1 -- 172.21.15.5:0/2603857654 wait complete. 2024-05-14T15:24:52.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 -- 172.21.15.5:0/3657179905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598108a50 msgr2=0x7f659810ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 --2- 172.21.15.5:0/3657179905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598108a50 0x7f659810ae70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6590009ca0 tx=0x7f659002a8c0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 -- 172.21.15.5:0/3657179905 shutdown_connections 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 --2- 172.21.15.5:0/3657179905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598108a50 0x7f659810ae70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 --2- 172.21.15.5:0/3657179905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6598103d50 0x7f6598104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 --2- 172.21.15.5:0/3657179905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6598102b10 0x7f6598103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 -- 172.21.15.5:0/3657179905 >> 172.21.15.5:0/3657179905 conn(0x7f65980fdd00 msgr2=0x7f65981001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 -- 172.21.15.5:0/3657179905 shutdown_connections 2024-05-14T15:24:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.507+0000 7f65a051f640 1 -- 172.21.15.5:0/3657179905 wait complete. 2024-05-14T15:24:52.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 Processor -- start 2024-05-14T15:24:52.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 -- start start 2024-05-14T15:24:52.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6598102b10 0x7f659806aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 0x7f659819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6598108a50 0x7f65981a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f659810e640 con 0x7f6598103d50 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f659810e4c0 con 0x7f6598102b10 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f65a051f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f659810e7c0 con 0x7f6598108a50 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 0x7f659819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 0x7f659819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51400/0 (socket says 172.21.15.5:51400) 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 -- 172.21.15.5:0/2905633119 learned_addr learned my addr 172.21.15.5:0/2905633119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659e2bb640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6598102b10 0x7f659806aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659eabc640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6598108a50 0x7f65981a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 -- 172.21.15.5:0/2905633119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6598108a50 msgr2=0x7f65981a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6598108a50 0x7f65981a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 -- 172.21.15.5:0/2905633119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6598102b10 msgr2=0x7f659806aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6598102b10 0x7f659806aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.508+0000 7f659daba640 1 -- 172.21.15.5:0/2905633119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981a2160 con 0x7f6598103d50 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.509+0000 7f659daba640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 0x7f659819f390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f658800e9f0 tx=0x7f658800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.509+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658800cd80 con 0x7f6598103d50 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.509+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65981a2450 con 0x7f6598103d50 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.509+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65880108e0 con 0x7f6598103d50 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.509+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588018980 con 0x7f6598103d50 2024-05-14T15:24:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.509+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981a29a0 con 0x7f6598103d50 2024-05-14T15:24:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.510+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6598107860 con 0x7f6598103d50 2024-05-14T15:24:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.512+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f6588010430 con 0x7f6598103d50 2024-05-14T15:24:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.512+0000 7f658f7fe640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65840b25e0 0x7f65840b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.512+0000 7f659e2bb640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65840b25e0 0x7f65840b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.512+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f65880e25f0 con 0x7f6598103d50 2024-05-14T15:24:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.513+0000 7f659e2bb640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65840b25e0 0x7f65840b4af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6598103710 tx=0x7f6594005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.519+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65880a9ad0 con 0x7f6598103d50 2024-05-14T15:24:52.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.636+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 23) ==== 149822+0+0 (secure 0 0 0) 0x7f65880a5d60 con 0x7f6598103d50 2024-05-14T15:24:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.766+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f659806c470 con 0x7f6598103d50 2024-05-14T15:24:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.767+0000 7f658f7fe640 1 -- 172.21.15.5:0/2905633119 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f659806c470 con 0x7f6598103d50 2024-05-14T15:24:52.768 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65840b25e0 msgr2=0x7f65840b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65840b25e0 0x7f65840b4af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6598103710 tx=0x7f6594005f90 comp rx=0 tx=0).stop 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 msgr2=0x7f659819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 0x7f659819f390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f658800e9f0 tx=0x7f658800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 shutdown_connections 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65840b25e0 0x7f65840b4af0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6598108a50 0x7f65981a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6598103d50 0x7f659819f390 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 --2- 172.21.15.5:0/2905633119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6598102b10 0x7f659806aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 >> 172.21.15.5:0/2905633119 conn(0x7f65980fdd00 msgr2=0x7f6598104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 shutdown_connections 2024-05-14T15:24:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.769+0000 7f65a051f640 1 -- 172.21.15.5:0/2905633119 wait complete. 2024-05-14T15:24:52.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T15:24:52.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 -- 172.21.15.5:0/2622915190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 msgr2=0x7f3534104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 --2- 172.21.15.5:0/2622915190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f3534104930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3528009900 tx=0x7f352802a410 comp rx=0 tx=0).stop 2024-05-14T15:24:52.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 -- 172.21.15.5:0/2622915190 shutdown_connections 2024-05-14T15:24:52.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 --2- 172.21.15.5:0/2622915190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3534109b70 0x7f353410bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 --2- 172.21.15.5:0/2622915190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 0x7f3534105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 --2- 172.21.15.5:0/2622915190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f3534104930 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 -- 172.21.15.5:0/2622915190 >> 172.21.15.5:0/2622915190 conn(0x7f35340fee20 msgr2=0x7f35341012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:52.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 -- 172.21.15.5:0/2622915190 shutdown_connections 2024-05-14T15:24:52.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.909+0000 7f353b229640 1 -- 172.21.15.5:0/2622915190 wait complete. 2024-05-14T15:24:52.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 Processor -- start 2024-05-14T15:24:52.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 -- start start 2024-05-14T15:24:52.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f353419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 0x7f353419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3534109b70 0x7f35341a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f353410f780 con 0x7f3534103c30 2024-05-14T15:24:52.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f353410f600 con 0x7f3534104e70 2024-05-14T15:24:52.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f353b229640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f353410f900 con 0x7f3534109b70 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f3538fc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f353419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f3538fc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f353419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51444/0 (socket says 172.21.15.5:51444) 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f3538fc5640 1 -- 172.21.15.5:0/4216328236 learned_addr learned my addr 172.21.15.5:0/4216328236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f3533fff640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 0x7f353419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.910+0000 7f35397c6640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3534109b70 0x7f35341a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3533fff640 1 -- 172.21.15.5:0/4216328236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3534109b70 msgr2=0x7f35341a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3533fff640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3534109b70 0x7f35341a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3533fff640 1 -- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 msgr2=0x7f353419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3533fff640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f353419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3533fff640 1 -- 172.21.15.5:0/4216328236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35341a1e70 con 0x7f3534104e70 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3538fc5640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f353419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:24:52.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.911+0000 7f3533fff640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 0x7f353419a870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f352000e9f0 tx=0x7f352000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:52.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.914+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352000ccc0 con 0x7f3534104e70 2024-05-14T15:24:52.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.914+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35200108e0 con 0x7f3534104e70 2024-05-14T15:24:52.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.914+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35341a2160 con 0x7f3534104e70 2024-05-14T15:24:52.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.914+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520018880 con 0x7f3534104e70 2024-05-14T15:24:52.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.914+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35341a26b0 con 0x7f3534104e70 2024-05-14T15:24:52.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.915+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3534108980 con 0x7f3534104e70 2024-05-14T15:24:52.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.917+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 23) ==== 149822+0+0 (secure 0 0 0) 0x7f3520010430 con 0x7f3534104e70 2024-05-14T15:24:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.918+0000 7f3531ffb640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35240b29f0 0x7f35240b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.918+0000 7f3538fc5640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35240b29f0 0x7f35240b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.918+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f35200e2eb0 con 0x7f3534104e70 2024-05-14T15:24:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.919+0000 7f3538fc5640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35240b29f0 0x7f35240b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3528000c00 tx=0x7f35280314a0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:52.925+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f35200a9d10 con 0x7f3534104e70 2024-05-14T15:24:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: [14/May/2024:15:24:51] ENGINE Bus STARTING 2024-05-14T15:24:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: [14/May/2024:15:24:51] ENGINE Serving on https://172.21.15.5:7150 2024-05-14T15:24:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: mgrmap e22: x(active, since 1.02107s), standbys: y, z 2024-05-14T15:24:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: [14/May/2024:15:24:51] ENGINE Serving on http://172.21.15.5:8765 2024-05-14T15:24:53.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: [14/May/2024:15:24:51] ENGINE Bus STARTED 2024-05-14T15:24:53.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/981199854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:24:53.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:52 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2603857654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:24:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: [14/May/2024:15:24:51] ENGINE Bus STARTING 2024-05-14T15:24:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: [14/May/2024:15:24:51] ENGINE Serving on https://172.21.15.5:7150 2024-05-14T15:24:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: mgrmap e22: x(active, since 1.02107s), standbys: y, z 2024-05-14T15:24:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: [14/May/2024:15:24:51] ENGINE Serving on http://172.21.15.5:8765 2024-05-14T15:24:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: [14/May/2024:15:24:51] ENGINE Bus STARTED 2024-05-14T15:24:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/981199854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:24:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:52 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2603857654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: [14/May/2024:15:24:51] ENGINE Bus STARTING 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: [14/May/2024:15:24:51] ENGINE Serving on https://172.21.15.5:7150 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: mgrmap e22: x(active, since 1.02107s), standbys: y, z 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: [14/May/2024:15:24:51] ENGINE Serving on http://172.21.15.5:8765 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: [14/May/2024:15:24:51] ENGINE Bus STARTED 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/981199854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:24:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:52 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2603857654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:24:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.123+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35340638d0 con 0x7f3534104e70 2024-05-14T15:24:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.124+0000 7f3531ffb640 1 -- 172.21.15.5:0/4216328236 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f35200ae010 con 0x7f3534104e70 2024-05-14T15:24:53.125 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35240b29f0 msgr2=0x7f35240b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35240b29f0 0x7f35240b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3528000c00 tx=0x7f35280314a0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 msgr2=0x7f353419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 0x7f353419a870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f352000e9f0 tx=0x7f352000eeb0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 shutdown_connections 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35240b29f0 0x7f35240b4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3534109b70 0x7f35341a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3534104e70 0x7f353419a870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 --2- 172.21.15.5:0/4216328236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534103c30 0x7f353419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 >> 172.21.15.5:0/4216328236 conn(0x7f35340fee20 msgr2=0x7f3534105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 shutdown_connections 2024-05-14T15:24:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.126+0000 7f353b229640 1 -- 172.21.15.5:0/4216328236 wait complete. 2024-05-14T15:24:53.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-14T15:24:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 -- 172.21.15.5:0/2191852750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 msgr2=0x7f1e5c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 --2- 172.21.15.5:0/2191852750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c102fe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009930 tx=0x7f1e4c02a420 comp rx=0 tx=0).stop 2024-05-14T15:24:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 -- 172.21.15.5:0/2191852750 shutdown_connections 2024-05-14T15:24:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 --2- 172.21.15.5:0/2191852750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e5c103670 0x7f1e5c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 --2- 172.21.15.5:0/2191852750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c102fe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 --2- 172.21.15.5:0/2191852750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e5c102110 0x7f1e5c102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 -- 172.21.15.5:0/2191852750 >> 172.21.15.5:0/2191852750 conn(0x7f1e5c0fdd00 msgr2=0x7f1e5c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 -- 172.21.15.5:0/2191852750 shutdown_connections 2024-05-14T15:24:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.256+0000 7f1e64138640 1 -- 172.21.15.5:0/2191852750 wait complete. 2024-05-14T15:24:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 Processor -- start 2024-05-14T15:24:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 -- start start 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e5c102110 0x7f1e5c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e5c103670 0x7f1e5c1a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c111940 con 0x7f1e5c102b10 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c1117c0 con 0x7f1e5c103670 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e64138640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c111ac0 con 0x7f1e5c102110 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51462/0 (socket says 172.21.15.5:51462) 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 -- 172.21.15.5:0/168754338 learned_addr learned my addr 172.21.15.5:0/168754338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e626d5640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e5c103670 0x7f1e5c1a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e61ed4640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e5c102110 0x7f1e5c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 -- 172.21.15.5:0/168754338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e5c102110 msgr2=0x7f1e5c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e5c102110 0x7f1e5c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 -- 172.21.15.5:0/168754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e5c103670 msgr2=0x7f1e5c1a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.257+0000 7f1e616d3640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e5c103670 0x7f1e5c1a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e616d3640 1 -- 172.21.15.5:0/168754338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1a4400 con 0x7f1e5c102b10 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e616d3640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c19ce00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009230 tx=0x7f1e4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c03f030 con 0x7f1e5c102b10 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c031910 con 0x7f1e5c102b10 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1a4690 con 0x7f1e5c102b10 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c035070 con 0x7f1e5c102b10 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.258+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c1a4b80 con 0x7f1e5c102b10 2024-05-14T15:24:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.259+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c10d570 con 0x7f1e5c102b10 2024-05-14T15:24:53.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.261+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 149822+0+0 (secure 0 0 0) 0x7f1e4c042030 con 0x7f1e5c102b10 2024-05-14T15:24:53.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.261+0000 7f1e52ffd640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1e480b29f0 0x7f1e480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:53.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.262+0000 7f1e61ed4640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1e480b29f0 0x7f1e480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:53.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.262+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f1e4c103630 con 0x7f1e5c102b10 2024-05-14T15:24:53.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.262+0000 7f1e61ed4640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1e480b29f0 0x7f1e480b4f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06a9a0 tx=0x7f1e58005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:53.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.265+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e4c0ca490 con 0x7f1e5c102b10 2024-05-14T15:24:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.491+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1e5c02ddb0 con 0x7f1e5c102b10 2024-05-14T15:24:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.492+0000 7f1e52ffd640 1 -- 172.21.15.5:0/168754338 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f1e4c0ce790 con 0x7f1e5c102b10 2024-05-14T15:24:53.494 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1e480b29f0 msgr2=0x7f1e480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1e480b29f0 0x7f1e480b4f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c06a9a0 tx=0x7f1e58005f90 comp rx=0 tx=0).stop 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 msgr2=0x7f1e5c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c19ce00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009230 tx=0x7f1e4c004270 comp rx=0 tx=0).stop 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 shutdown_connections 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1e480b29f0 0x7f1e480b4f00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e5c103670 0x7f1e5c1a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102b10 0x7f1e5c19ce00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 --2- 172.21.15.5:0/168754338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e5c102110 0x7f1e5c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 >> 172.21.15.5:0/168754338 conn(0x7f1e5c0fdd00 msgr2=0x7f1e5c10a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 shutdown_connections 2024-05-14T15:24:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:53.494+0000 7f1e64138640 1 -- 172.21.15.5:0/168754338 wait complete. 2024-05-14T15:24:53.508 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-14T15:24:53.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:24:53.765 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:53.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:53.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-14T15:24:53.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:53.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2905633119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/4216328236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T15:24:53.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:53 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/168754338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T15:24:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-14T15:24:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2905633119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/4216328236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T15:24:54.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:53 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/168754338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T15:24:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-14T15:24:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2905633119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T15:24:54.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:54.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/4216328236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T15:24:54.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:53 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/168754338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T15:24:55.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:55.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:54 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:55.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:55.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:54 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:55.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:54 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 -- 172.21.15.5:0/4033410591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe320107df0 msgr2=0x7fe320108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 --2- 172.21.15.5:0/4033410591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe320107df0 0x7fe320108200 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe3080099c0 tx=0x7fe308009cd0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 -- 172.21.15.5:0/4033410591 shutdown_connections 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 --2- 172.21.15.5:0/4033410591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320108740 0x7fe3201100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 --2- 172.21.15.5:0/4033410591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3201034c0 0x7fe3201038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 --2- 172.21.15.5:0/4033410591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe320107df0 0x7fe320108200 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 -- 172.21.15.5:0/4033410591 >> 172.21.15.5:0/4033410591 conn(0x7fe3200fedf0 msgr2=0x7fe320101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:55.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 -- 172.21.15.5:0/4033410591 shutdown_connections 2024-05-14T15:24:55.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.676+0000 7fe32552b700 1 -- 172.21.15.5:0/4033410591 wait complete. 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 Processor -- start 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 -- start start 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 0x7fe320100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320107df0 0x7fe3200ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe320108740 0x7fe3200ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe320113350 con 0x7fe3201034c0 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201131d0 con 0x7fe320107df0 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe32552b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201134d0 con 0x7fe320108740 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe31effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 0x7fe320100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe31effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 0x7fe320100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60634/0 (socket says 172.21.15.5:60634) 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe31effd700 1 -- 172.21.15.5:0/2292931077 learned_addr learned my addr 172.21.15.5:0/2292931077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:55.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.677+0000 7fe317fff700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320107df0 0x7fe3200ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31f7fe700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe320108740 0x7fe3200ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31effd700 1 -- 172.21.15.5:0/2292931077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe320108740 msgr2=0x7fe3200ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31effd700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe320108740 0x7fe3200ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31effd700 1 -- 172.21.15.5:0/2292931077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320107df0 msgr2=0x7fe3200ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31effd700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320107df0 0x7fe3200ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31effd700 1 -- 172.21.15.5:0/2292931077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3201000e0 con 0x7fe3201034c0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31f7fe700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe320108740 0x7fe3200ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe317fff700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320107df0 0x7fe3200ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31effd700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 0x7fe320100e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe31000d980 tx=0x7fe31000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310016070 con 0x7fe3201034c0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3201ad8a0 con 0x7fe3201034c0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3100044d0 con 0x7fe3201034c0 2024-05-14T15:24:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310004d00 con 0x7fe3201034c0 2024-05-14T15:24:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.678+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201addb0 con 0x7fe3201034c0 2024-05-14T15:24:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.679+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe320104970 con 0x7fe3201034c0 2024-05-14T15:24:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.681+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fe310004020 con 0x7fe3201034c0 2024-05-14T15:24:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.682+0000 7fe31cff9700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe30c0b2b10 0x7fe30c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.682+0000 7fe317fff700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe30c0b2b10 0x7fe30c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.682+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fe3100e2b60 con 0x7fe3201034c0 2024-05-14T15:24:55.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.682+0000 7fe317fff700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe30c0b2b10 0x7fe30c0b5020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe308000c00 tx=0x7fe30801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:55.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.687+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe31001f020 con 0x7fe3201034c0 2024-05-14T15:24:55.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.951+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe32002e5f0 con 0x7fe3201034c0 2024-05-14T15:24:55.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.952+0000 7fe31cff9700 1 -- 172.21.15.5:0/2292931077 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11864 (secure 0 0 0) 0x7fe3100a9f30 con 0x7fe3201034c0 2024-05-14T15:24:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:55.954 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:24:47.806060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:24:55.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe30c0b2b10 msgr2=0x7fe30c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:55.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe30c0b2b10 0x7fe30c0b5020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe308000c00 tx=0x7fe30801a040 comp rx=0 tx=0).stop 2024-05-14T15:24:55.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 msgr2=0x7fe320100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:55.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 0x7fe320100e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe31000d980 tx=0x7fe31000dd40 comp rx=0 tx=0).stop 2024-05-14T15:24:55.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 shutdown_connections 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe30c0b2b10 0x7fe30c0b5020 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe320108740 0x7fe3200ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3201034c0 0x7fe320100e70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 --2- 172.21.15.5:0/2292931077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe320107df0 0x7fe3200ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.954+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 >> 172.21.15.5:0/2292931077 conn(0x7fe3200fedf0 msgr2=0x7fe320075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.955+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 shutdown_connections 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:55.955+0000 7fe32552b700 1 -- 172.21.15.5:0/2292931077 wait complete. 2024-05-14T15:24:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-14T15:24:56.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-05-14T15:24:56.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Unable to set osd_memory_target on smithi144 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-14T15:24:56.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T15:24:56.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.200 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:56.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Unable to set osd_memory_target on smithi144 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:24:56.206 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.207 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:56.208 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:56.208 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:56.208 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-05-14T15:24:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Unable to set osd_memory_target on smithi144 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi005:/etc/ceph/ceph.conf 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi144:/etc/ceph/ceph.conf 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi193:/etc/ceph/ceph.conf 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-05-14T15:24:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:24:56.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:24:56.776 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-14T15:24:56.776 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-14T15:24:56.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T15:24:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 -- 172.21.15.5:0/3960426202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f553410e6a0 msgr2=0x7f5534110ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 --2- 172.21.15.5:0/3960426202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f553410e6a0 0x7f5534110ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f552c009ca0 tx=0x7f552c02a8c0 comp rx=0 tx=0).stop 2024-05-14T15:24:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 -- 172.21.15.5:0/3960426202 shutdown_connections 2024-05-14T15:24:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 --2- 172.21.15.5:0/3960426202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55341021a0 0x7f55341025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 --2- 172.21.15.5:0/3960426202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f553410e6a0 0x7f5534110ac0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 --2- 172.21.15.5:0/3960426202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5534109cf0 0x7f5534101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 -- 172.21.15.5:0/3960426202 >> 172.21.15.5:0/3960426202 conn(0x7f5534078a50 msgr2=0x7f5534078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 -- 172.21.15.5:0/3960426202 shutdown_connections 2024-05-14T15:24:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 -- 172.21.15.5:0/3960426202 wait complete. 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.906+0000 7f553ae92640 1 Processor -- start 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 -- start start 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55341021a0 0x7f553419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 0x7f553419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553410e6a0 0x7f55341a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5534113e60 con 0x7f5534109cf0 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5534113ce0 con 0x7f553410e6a0 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553ae92640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5534113fe0 con 0x7f55341021a0 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 0x7f553419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 0x7f553419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60656/0 (socket says 172.21.15.5:60656) 2024-05-14T15:24:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 -- 172.21.15.5:0/2984084420 learned_addr learned my addr 172.21.15.5:0/2984084420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f553942f640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553410e6a0 0x7f55341a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5538c2e640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55341021a0 0x7f553419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 -- 172.21.15.5:0/2984084420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55341021a0 msgr2=0x7f553419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55341021a0 0x7f553419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 -- 172.21.15.5:0/2984084420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553410e6a0 msgr2=0x7f55341a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553410e6a0 0x7f55341a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.907+0000 7f5533fff640 1 -- 172.21.15.5:0/2984084420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341a8ce0 con 0x7f5534109cf0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.908+0000 7f5533fff640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 0x7f553419dc60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f552000d770 tx=0x7f552000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.908+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520017070 con 0x7f5534109cf0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.908+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5520009c30 con 0x7f5534109cf0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.908+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341a8eb0 con 0x7f5534109cf0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.908+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552000f620 con 0x7f5534109cf0 2024-05-14T15:24:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.908+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341a92c0 con 0x7f5534109cf0 2024-05-14T15:24:56.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.909+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553406a240 con 0x7f5534109cf0 2024-05-14T15:24:56.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.911+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f552001a070 con 0x7f5534109cf0 2024-05-14T15:24:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.911+0000 7f5531ffb640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55240b2a80 0x7f55240b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.911+0000 7f5538c2e640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55240b2a80 0x7f55240b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.911+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f55200e27d0 con 0x7f5534109cf0 2024-05-14T15:24:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.912+0000 7f5538c2e640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55240b2a80 0x7f55240b4f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5528005e00 tx=0x7f5528005c50 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:56.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:56.915+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5520009de0 con 0x7f5534109cf0 2024-05-14T15:24:57.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:56 smithi005 ceph-mon[31559]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:56 smithi005 ceph-mon[31559]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:56 smithi005 ceph-mon[31559]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:56 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2292931077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.103+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5534101110 con 0x7f55240b2a80 2024-05-14T15:24:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.114+0000 7f5531ffb640 1 -- 172.21.15.5:0/2984084420 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339652 (secure 0 0 0) 0x7f5534101110 con 0x7f55240b2a80 2024-05-14T15:24:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55240b2a80 msgr2=0x7f55240b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55240b2a80 0x7f55240b4f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5528005e00 tx=0x7f5528005c50 comp rx=0 tx=0).stop 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 msgr2=0x7f553419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 0x7f553419dc60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f552000d770 tx=0x7f552000dc30 comp rx=0 tx=0).stop 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 shutdown_connections 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55240b2a80 0x7f55240b4f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553410e6a0 0x7f55341a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5534109cf0 0x7f553419dc60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 --2- 172.21.15.5:0/2984084420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55341021a0 0x7f553419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 >> 172.21.15.5:0/2984084420 conn(0x7f5534078a50 msgr2=0x7f5534100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:57.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 shutdown_connections 2024-05-14T15:24:57.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.118+0000 7f553ae92640 1 -- 172.21.15.5:0/2984084420 wait complete. 2024-05-14T15:24:57.131 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-14T15:24:56.633270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189711,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1250,"num_read_kb":766,"num_write":420,"num_write_kb":3595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378176,"kb_used_data":12344,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563648,"statfs":{"total":1151940427776,"available":1151553175552,"internally_reserved":0,"allocated":12640256,"data_stored":9872377,"data_compressed":605208,"data_compressed_allocated":819200,"data_compressed_original":4489216,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990686"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004427+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:24:51.004427+0000","last_peered":"2024-05-14T15:24:51.004427+0000","last_clean":"2024-05-14T15:24:51.004427+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:24:51.004427+0000","last_undegraded":"2024-05-14T15:24:51.004427+0000","last_fullsized":"2024-05-14T15:24:51.004427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982134+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:24:50.982134+0000","last_peered":"2024-05-14T15:24:50.982134+0000","last_clean":"2024-05-14T15:24:50.982134+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:24:50.982134+0000","last_undegraded":"2024-05-14T15:24:50.982134+0000","last_fullsized":"2024-05-14T15:24:50.982134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981937+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:24:50.981937+0000","last_peered":"2024-05-14T15:24:50.981937+0000","last_clean":"2024-05-14T15:24:50.981937+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:24:50.981937+0000","last_undegraded":"2024-05-14T15:24:50.981937+0000","last_fullsized":"2024-05-14T15:24:50.981937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981876+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:24:50.981876+0000","last_peered":"2024-05-14T15:24:50.981876+0000","last_clean":"2024-05-14T15:24:50.981876+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:24:50.981876+0000","last_undegraded":"2024-05-14T15:24:50.981876+0000","last_fullsized":"2024-05-14T15:24:50.981876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724901+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:24:50.724901+0000","last_peered":"2024-05-14T15:24:50.724901+0000","last_clean":"2024-05-14T15:24:50.724901+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:24:50.724901+0000","last_undegraded":"2024-05-14T15:24:50.724901+0000","last_fullsized":"2024-05-14T15:24:50.724901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739533+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:24:50.739533+0000","last_peered":"2024-05-14T15:24:50.739533+0000","last_clean":"2024-05-14T15:24:50.739533+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:24:50.739533+0000","last_undegraded":"2024-05-14T15:24:50.739533+0000","last_fullsized":"2024-05-14T15:24:50.739533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887231+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:24:50.887231+0000","last_peered":"2024-05-14T15:24:50.887231+0000","last_clean":"2024-05-14T15:24:50.887231+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:24:50.887231+0000","last_undegraded":"2024-05-14T15:24:50.887231+0000","last_fullsized":"2024-05-14T15:24:50.887231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725049+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:24:50.725049+0000","last_peered":"2024-05-14T15:24:50.725049+0000","last_clean":"2024-05-14T15:24:50.725049+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:24:50.725049+0000","last_undegraded":"2024-05-14T15:24:50.725049+0000","last_fullsized":"2024-05-14T15:24:50.725049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618484+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:24:50.618484+0000","last_peered":"2024-05-14T15:24:50.618484+0000","last_clean":"2024-05-14T15:24:50.618484+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:24:50.618484+0000","last_undegraded":"2024-05-14T15:24:50.618484+0000","last_fullsized":"2024-05-14T15:24:50.618484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739931+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:24:50.739931+0000","last_peered":"2024-05-14T15:24:50.739931+0000","last_clean":"2024-05-14T15:24:50.739931+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:24:50.739931+0000","last_undegraded":"2024-05-14T15:24:50.739931+0000","last_fullsized":"2024-05-14T15:24:50.739931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887724+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:24:50.887724+0000","last_peered":"2024-05-14T15:24:50.887724+0000","last_clean":"2024-05-14T15:24:50.887724+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:24:50.887724+0000","last_undegraded":"2024-05-14T15:24:50.887724+0000","last_fullsized":"2024-05-14T15:24:50.887724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981832+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:24:50.981832+0000","last_peered":"2024-05-14T15:24:50.981832+0000","last_clean":"2024-05-14T15:24:50.981832+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:24:50.981832+0000","last_undegraded":"2024-05-14T15:24:50.981832+0000","last_fullsized":"2024-05-14T15:24:50.981832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982434+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:24:50.982434+0000","last_peered":"2024-05-14T15:24:50.982434+0000","last_clean":"2024-05-14T15:24:50.982434+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:24:50.982434+0000","last_undegraded":"2024-05-14T15:24:50.982434+0000","last_fullsized":"2024-05-14T15:24:50.982434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981531+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:24:50.981531+0000","last_peered":"2024-05-14T15:24:50.981531+0000","last_clean":"2024-05-14T15:24:50.981531+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:24:50.981531+0000","last_undegraded":"2024-05-14T15:24:50.981531+0000","last_fullsized":"2024-05-14T15:24:50.981531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981623+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:24:50.981623+0000","last_peered":"2024-05-14T15:24:50.981623+0000","last_clean":"2024-05-14T15:24:50.981623+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:24:50.981623+0000","last_undegraded":"2024-05-14T15:24:50.981623+0000","last_fullsized":"2024-05-14T15:24:50.981623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617800+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:24:50.617800+0000","last_peered":"2024-05-14T15:24:50.617800+0000","last_clean":"2024-05-14T15:24:50.617800+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:24:50.617800+0000","last_undegraded":"2024-05-14T15:24:50.617800+0000","last_fullsized":"2024-05-14T15:24:50.617800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004331+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:24:51.004331+0000","last_peered":"2024-05-14T15:24:51.004331+0000","last_clean":"2024-05-14T15:24:51.004331+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:24:51.004331+0000","last_undegraded":"2024-05-14T15:24:51.004331+0000","last_fullsized":"2024-05-14T15:24:51.004331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616451+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:24:50.616451+0000","last_peered":"2024-05-14T15:24:50.616451+0000","last_clean":"2024-05-14T15:24:50.616451+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:24:50.616451+0000","last_undegraded":"2024-05-14T15:24:50.616451+0000","last_fullsized":"2024-05-14T15:24:50.616451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621311+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:24:50.621311+0000","last_peered":"2024-05-14T15:24:50.621311+0000","last_clean":"2024-05-14T15:24:50.621311+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:24:50.621311+0000","last_undegraded":"2024-05-14T15:24:50.621311+0000","last_fullsized":"2024-05-14T15:24:50.621311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619740+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:24:50.619740+0000","last_peered":"2024-05-14T15:24:50.619740+0000","last_clean":"2024-05-14T15:24:50.619740+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:24:50.619740+0000","last_undegraded":"2024-05-14T15:24:50.619740+0000","last_fullsized":"2024-05-14T15:24:50.619740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616807+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:24:50.616807+0000","last_peered":"2024-05-14T15:24:50.616807+0000","last_clean":"2024-05-14T15:24:50.616807+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:24:50.616807+0000","last_undegraded":"2024-05-14T15:24:50.616807+0000","last_fullsized":"2024-05-14T15:24:50.616807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725463+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:24:50.725463+0000","last_peered":"2024-05-14T15:24:50.725463+0000","last_clean":"2024-05-14T15:24:50.725463+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:24:50.725463+0000","last_undegraded":"2024-05-14T15:24:50.725463+0000","last_fullsized":"2024-05-14T15:24:50.725463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981463+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:24:50.981463+0000","last_peered":"2024-05-14T15:24:50.981463+0000","last_clean":"2024-05-14T15:24:50.981463+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:24:50.981463+0000","last_undegraded":"2024-05-14T15:24:50.981463+0000","last_fullsized":"2024-05-14T15:24:50.981463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725622+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:24:50.725622+0000","last_peered":"2024-05-14T15:24:50.725622+0000","last_clean":"2024-05-14T15:24:50.725622+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:24:50.725622+0000","last_undegraded":"2024-05-14T15:24:50.725622+0000","last_fullsized":"2024-05-14T15:24:50.725622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982136+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:24:50.982136+0000","last_peered":"2024-05-14T15:24:50.982136+0000","last_clean":"2024-05-14T15:24:50.982136+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:24:50.982136+0000","last_undegraded":"2024-05-14T15:24:50.982136+0000","last_fullsized":"2024-05-14T15:24:50.982136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981579+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:24:50.981579+0000","last_peered":"2024-05-14T15:24:50.981579+0000","last_clean":"2024-05-14T15:24:50.981579+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:24:50.981579+0000","last_undegraded":"2024-05-14T15:24:50.981579+0000","last_fullsized":"2024-05-14T15:24:50.981579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725311+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:24:50.725311+0000","last_peered":"2024-05-14T15:24:50.725311+0000","last_clean":"2024-05-14T15:24:50.725311+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:24:50.725311+0000","last_undegraded":"2024-05-14T15:24:50.725311+0000","last_fullsized":"2024-05-14T15:24:50.725311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618077+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:24:50.618077+0000","last_peered":"2024-05-14T15:24:50.618077+0000","last_clean":"2024-05-14T15:24:50.618077+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:24:50.618077+0000","last_undegraded":"2024-05-14T15:24:50.618077+0000","last_fullsized":"2024-05-14T15:24:50.618077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981599+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:24:50.981599+0000","last_peered":"2024-05-14T15:24:50.981599+0000","last_clean":"2024-05-14T15:24:50.981599+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:24:50.981599+0000","last_undegraded":"2024-05-14T15:24:50.981599+0000","last_fullsized":"2024-05-14T15:24:50.981599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725116+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:24:50.725116+0000","last_peered":"2024-05-14T15:24:50.725116+0000","last_clean":"2024-05-14T15:24:50.725116+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:24:50.725116+0000","last_undegraded":"2024-05-14T15:24:50.725116+0000","last_fullsized":"2024-05-14T15:24:50.725116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736594+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:24:50.736594+0000","last_peered":"2024-05-14T15:24:50.736594+0000","last_clean":"2024-05-14T15:24:50.736594+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:24:50.736594+0000","last_undegraded":"2024-05-14T15:24:50.736594+0000","last_fullsized":"2024-05-14T15:24:50.736594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621169+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:24:50.621169+0000","last_peered":"2024-05-14T15:24:50.621169+0000","last_clean":"2024-05-14T15:24:50.621169+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:24:50.621169+0000","last_undegraded":"2024-05-14T15:24:50.621169+0000","last_fullsized":"2024-05-14T15:24:50.621169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.036543+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:24:51.036543+0000","last_peered":"2024-05-14T15:24:51.036543+0000","last_clean":"2024-05-14T15:24:51.036543+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:24:51.036543+0000","last_undegraded":"2024-05-14T15:24:51.036543+0000","last_fullsized":"2024-05-14T15:24:51.036543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724843+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:24:50.724843+0000","last_peered":"2024-05-14T15:24:50.724843+0000","last_clean":"2024-05-14T15:24:50.724843+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:24:50.724843+0000","last_undegraded":"2024-05-14T15:24:50.724843+0000","last_fullsized":"2024-05-14T15:24:50.724843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.620804+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:24:50.620804+0000","last_peered":"2024-05-14T15:24:50.620804+0000","last_clean":"2024-05-14T15:24:50.620804+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:24:50.620804+0000","last_undegraded":"2024-05-14T15:24:50.620804+0000","last_fullsized":"2024-05-14T15:24:50.620804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739485+0000","last_change":"2024-05-14T15:22:43.233044+0000","last_active":"2024-05-14T15:24:50.739485+0000","last_peered":"2024-05-14T15:24:50.739485+0000","last_clean":"2024-05-14T15:24:50.739485+0000","last_became_active":"2024-05-14T15:22:43.229808+0000","last_became_peered":"2024-05-14T15:22:43.229808+0000","last_unstale":"2024-05-14T15:24:50.739485+0000","last_undegraded":"2024-05-14T15:24:50.739485+0000","last_fullsized":"2024-05-14T15:24:50.739485+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:07.226460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736522+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:24:50.736522+0000","last_peered":"2024-05-14T15:24:50.736522+0000","last_clean":"2024-05-14T15:24:50.736522+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:24:50.736522+0000","last_undegraded":"2024-05-14T15:24:50.736522+0000","last_fullsized":"2024-05-14T15:24:50.736522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003934+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:24:51.003934+0000","last_peered":"2024-05-14T15:24:51.003934+0000","last_clean":"2024-05-14T15:24:51.003934+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:24:51.003934+0000","last_undegraded":"2024-05-14T15:24:51.003934+0000","last_fullsized":"2024-05-14T15:24:51.003934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981845+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:24:50.981845+0000","last_peered":"2024-05-14T15:24:50.981845+0000","last_clean":"2024-05-14T15:24:50.981845+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:24:50.981845+0000","last_undegraded":"2024-05-14T15:24:50.981845+0000","last_fullsized":"2024-05-14T15:24:50.981845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724897+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:24:50.724897+0000","last_peered":"2024-05-14T15:24:50.724897+0000","last_clean":"2024-05-14T15:24:50.724897+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:24:50.724897+0000","last_undegraded":"2024-05-14T15:24:50.724897+0000","last_fullsized":"2024-05-14T15:24:50.724897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982168+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:24:50.982168+0000","last_peered":"2024-05-14T15:24:50.982168+0000","last_clean":"2024-05-14T15:24:50.982168+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:24:50.982168+0000","last_undegraded":"2024-05-14T15:24:50.982168+0000","last_fullsized":"2024-05-14T15:24:50.982168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003726+0000","last_change":"2024-05-14T15:22:42.900544+0000","last_active":"2024-05-14T15:24:51.003726+0000","last_peered":"2024-05-14T15:24:51.003726+0000","last_clean":"2024-05-14T15:24:51.003726+0000","last_became_active":"2024-05-14T15:22:42.900015+0000","last_became_peered":"2024-05-14T15:22:42.900015+0000","last_unstale":"2024-05-14T15:24:51.003726+0000","last_undegraded":"2024-05-14T15:24:51.003726+0000","last_fullsized":"2024-05-14T15:24:51.003726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:15.904051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616117+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:24:50.616117+0000","last_peered":"2024-05-14T15:24:50.616117+0000","last_clean":"2024-05-14T15:24:50.616117+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:24:50.616117+0000","last_undegraded":"2024-05-14T15:24:50.616117+0000","last_fullsized":"2024-05-14T15:24:50.616117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724836+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:24:50.724836+0000","last_peered":"2024-05-14T15:24:50.724836+0000","last_clean":"2024-05-14T15:24:50.724836+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:24:50.724836+0000","last_undegraded":"2024-05-14T15:24:50.724836+0000","last_fullsized":"2024-05-14T15:24:50.724836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.886998+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:24:50.886998+0000","last_peered":"2024-05-14T15:24:50.886998+0000","last_clean":"2024-05-14T15:24:50.886998+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:24:50.886998+0000","last_undegraded":"2024-05-14T15:24:50.886998+0000","last_fullsized":"2024-05-14T15:24:50.886998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618501+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:24:50.618501+0000","last_peered":"2024-05-14T15:24:50.618501+0000","last_clean":"2024-05-14T15:24:50.618501+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:24:50.618501+0000","last_undegraded":"2024-05-14T15:24:50.618501+0000","last_fullsized":"2024-05-14T15:24:50.618501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.615998+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:24:50.615998+0000","last_peered":"2024-05-14T15:24:50.615998+0000","last_clean":"2024-05-14T15:24:50.615998+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:24:50.615998+0000","last_undegraded":"2024-05-14T15:24:50.615998+0000","last_fullsized":"2024-05-14T15:24:50.615998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887655+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:24:50.887655+0000","last_peered":"2024-05-14T15:24:50.887655+0000","last_clean":"2024-05-14T15:24:50.887655+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:24:50.887655+0000","last_undegraded":"2024-05-14T15:24:50.887655+0000","last_fullsized":"2024-05-14T15:24:50.887655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982157+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:24:50.982157+0000","last_peered":"2024-05-14T15:24:50.982157+0000","last_clean":"2024-05-14T15:24:50.982157+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:24:50.982157+0000","last_undegraded":"2024-05-14T15:24:50.982157+0000","last_fullsized":"2024-05-14T15:24:50.982157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981722+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:24:50.981722+0000","last_peered":"2024-05-14T15:24:50.981722+0000","last_clean":"2024-05-14T15:24:50.981722+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:24:50.981722+0000","last_undegraded":"2024-05-14T15:24:50.981722+0000","last_fullsized":"2024-05-14T15:24:50.981722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736561+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:24:50.736561+0000","last_peered":"2024-05-14T15:24:50.736561+0000","last_clean":"2024-05-14T15:24:50.736561+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:24:50.736561+0000","last_undegraded":"2024-05-14T15:24:50.736561+0000","last_fullsized":"2024-05-14T15:24:50.736561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618076+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:24:50.618076+0000","last_peered":"2024-05-14T15:24:50.618076+0000","last_clean":"2024-05-14T15:24:50.618076+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:24:50.618076+0000","last_undegraded":"2024-05-14T15:24:50.618076+0000","last_fullsized":"2024-05-14T15:24:50.618076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616173+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:24:50.616173+0000","last_peered":"2024-05-14T15:24:50.616173+0000","last_clean":"2024-05-14T15:24:50.616173+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:24:50.616173+0000","last_undegraded":"2024-05-14T15:24:50.616173+0000","last_fullsized":"2024-05-14T15:24:50.616173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":447,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.036303+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:24:51.036303+0000","last_peered":"2024-05-14T15:24:51.036303+0000","last_clean":"2024-05-14T15:24:51.036303+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:24:51.036303+0000","last_undegraded":"2024-05-14T15:24:51.036303+0000","last_fullsized":"2024-05-14T15:24:51.036303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69369,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":249,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725097+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:24:50.725097+0000","last_peered":"2024-05-14T15:24:50.725097+0000","last_clean":"2024-05-14T15:24:50.725097+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:24:50.725097+0000","last_undegraded":"2024-05-14T15:24:50.725097+0000","last_fullsized":"2024-05-14T15:24:50.725097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736912+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:24:50.736912+0000","last_peered":"2024-05-14T15:24:50.736912+0000","last_clean":"2024-05-14T15:24:50.736912+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:24:50.736912+0000","last_undegraded":"2024-05-14T15:24:50.736912+0000","last_fullsized":"2024-05-14T15:24:50.736912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.620086+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:24:50.620086+0000","last_peered":"2024-05-14T15:24:50.620086+0000","last_clean":"2024-05-14T15:24:50.620086+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:24:50.620086+0000","last_undegraded":"2024-05-14T15:24:50.620086+0000","last_fullsized":"2024-05-14T15:24:50.620086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982061+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:24:50.982061+0000","last_peered":"2024-05-14T15:24:50.982061+0000","last_clean":"2024-05-14T15:24:50.982061+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:24:50.982061+0000","last_undegraded":"2024-05-14T15:24:50.982061+0000","last_fullsized":"2024-05-14T15:24:50.982061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003977+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:24:51.003977+0000","last_peered":"2024-05-14T15:24:51.003977+0000","last_clean":"2024-05-14T15:24:51.003977+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:24:51.003977+0000","last_undegraded":"2024-05-14T15:24:51.003977+0000","last_fullsized":"2024-05-14T15:24:51.003977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736331+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:24:50.736331+0000","last_peered":"2024-05-14T15:24:50.736331+0000","last_clean":"2024-05-14T15:24:50.736331+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:24:50.736331+0000","last_undegraded":"2024-05-14T15:24:50.736331+0000","last_fullsized":"2024-05-14T15:24:50.736331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887498+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:24:50.887498+0000","last_peered":"2024-05-14T15:24:50.887498+0000","last_clean":"2024-05-14T15:24:50.887498+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:24:50.887498+0000","last_undegraded":"2024-05-14T15:24:50.887498+0000","last_fullsized":"2024-05-14T15:24:50.887498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616369+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:24:50.616369+0000","last_peered":"2024-05-14T15:24:50.616369+0000","last_clean":"2024-05-14T15:24:50.616369+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:24:50.616369+0000","last_undegraded":"2024-05-14T15:24:50.616369+0000","last_fullsized":"2024-05-14T15:24:50.616369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982014+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:24:50.982014+0000","last_peered":"2024-05-14T15:24:50.982014+0000","last_clean":"2024-05-14T15:24:50.982014+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:24:50.982014+0000","last_undegraded":"2024-05-14T15:24:50.982014+0000","last_fullsized":"2024-05-14T15:24:50.982014+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887383+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:24:50.887383+0000","last_peered":"2024-05-14T15:24:50.887383+0000","last_clean":"2024-05-14T15:24:50.887383+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:24:50.887383+0000","last_undegraded":"2024-05-14T15:24:50.887383+0000","last_fullsized":"2024-05-14T15:24:50.887383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982928+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:24:50.982928+0000","last_peered":"2024-05-14T15:24:50.982928+0000","last_clean":"2024-05-14T15:24:50.982928+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:24:50.982928+0000","last_undegraded":"2024-05-14T15:24:50.982928+0000","last_fullsized":"2024-05-14T15:24:50.982928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724711+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:24:50.724711+0000","last_peered":"2024-05-14T15:24:50.724711+0000","last_clean":"2024-05-14T15:24:50.724711+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:24:50.724711+0000","last_undegraded":"2024-05-14T15:24:50.724711+0000","last_fullsized":"2024-05-14T15:24:50.724711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736231+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:24:50.736231+0000","last_peered":"2024-05-14T15:24:50.736231+0000","last_clean":"2024-05-14T15:24:50.736231+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:24:50.736231+0000","last_undegraded":"2024-05-14T15:24:50.736231+0000","last_fullsized":"2024-05-14T15:24:50.736231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736232+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:24:50.736232+0000","last_peered":"2024-05-14T15:24:50.736232+0000","last_clean":"2024-05-14T15:24:50.736232+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:24:50.736232+0000","last_undegraded":"2024-05-14T15:24:50.736232+0000","last_fullsized":"2024-05-14T15:24:50.736232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725227+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:24:50.725227+0000","last_peered":"2024-05-14T15:24:50.725227+0000","last_clean":"2024-05-14T15:24:50.725227+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:24:50.725227+0000","last_undegraded":"2024-05-14T15:24:50.725227+0000","last_fullsized":"2024-05-14T15:24:50.725227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004205+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:24:51.004205+0000","last_peered":"2024-05-14T15:24:51.004205+0000","last_clean":"2024-05-14T15:24:51.004205+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:24:51.004205+0000","last_undegraded":"2024-05-14T15:24:51.004205+0000","last_fullsized":"2024-05-14T15:24:51.004205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724662+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:24:50.724662+0000","last_peered":"2024-05-14T15:24:50.724662+0000","last_clean":"2024-05-14T15:24:50.724662+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:24:50.724662+0000","last_undegraded":"2024-05-14T15:24:50.724662+0000","last_fullsized":"2024-05-14T15:24:50.724662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616333+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:24:50.616333+0000","last_peered":"2024-05-14T15:24:50.616333+0000","last_clean":"2024-05-14T15:24:50.616333+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:24:50.616333+0000","last_undegraded":"2024-05-14T15:24:50.616333+0000","last_fullsized":"2024-05-14T15:24:50.616333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739775+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:24:50.739775+0000","last_peered":"2024-05-14T15:24:50.739775+0000","last_clean":"2024-05-14T15:24:50.739775+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:24:50.739775+0000","last_undegraded":"2024-05-14T15:24:50.739775+0000","last_fullsized":"2024-05-14T15:24:50.739775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982918+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:24:50.982918+0000","last_peered":"2024-05-14T15:24:50.982918+0000","last_clean":"2024-05-14T15:24:50.982918+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:24:50.982918+0000","last_undegraded":"2024-05-14T15:24:50.982918+0000","last_fullsized":"2024-05-14T15:24:50.982918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887350+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:24:50.887350+0000","last_peered":"2024-05-14T15:24:50.887350+0000","last_clean":"2024-05-14T15:24:50.887350+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:24:50.887350+0000","last_undegraded":"2024-05-14T15:24:50.887350+0000","last_fullsized":"2024-05-14T15:24:50.887350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739500+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:24:50.739500+0000","last_peered":"2024-05-14T15:24:50.739500+0000","last_clean":"2024-05-14T15:24:50.739500+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:24:50.739500+0000","last_undegraded":"2024-05-14T15:24:50.739500+0000","last_fullsized":"2024-05-14T15:24:50.739500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983481+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:24:50.983481+0000","last_peered":"2024-05-14T15:24:50.983481+0000","last_clean":"2024-05-14T15:24:50.983481+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:24:50.983481+0000","last_undegraded":"2024-05-14T15:24:50.983481+0000","last_fullsized":"2024-05-14T15:24:50.983481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736244+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:24:50.736244+0000","last_peered":"2024-05-14T15:24:50.736244+0000","last_clean":"2024-05-14T15:24:50.736244+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:24:50.736244+0000","last_undegraded":"2024-05-14T15:24:50.736244+0000","last_fullsized":"2024-05-14T15:24:50.736244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981800+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:24:50.981800+0000","last_peered":"2024-05-14T15:24:50.981800+0000","last_clean":"2024-05-14T15:24:50.981800+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:24:50.981800+0000","last_undegraded":"2024-05-14T15:24:50.981800+0000","last_fullsized":"2024-05-14T15:24:50.981800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887629+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:24:50.887629+0000","last_peered":"2024-05-14T15:24:50.887629+0000","last_clean":"2024-05-14T15:24:50.887629+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:24:50.887629+0000","last_undegraded":"2024-05-14T15:24:50.887629+0000","last_fullsized":"2024-05-14T15:24:50.887629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725131+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:24:50.725131+0000","last_peered":"2024-05-14T15:24:50.725131+0000","last_clean":"2024-05-14T15:24:50.725131+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:24:50.725131+0000","last_undegraded":"2024-05-14T15:24:50.725131+0000","last_fullsized":"2024-05-14T15:24:50.725131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981587+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:24:50.981587+0000","last_peered":"2024-05-14T15:24:50.981587+0000","last_clean":"2024-05-14T15:24:50.981587+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:24:50.981587+0000","last_undegraded":"2024-05-14T15:24:50.981587+0000","last_fullsized":"2024-05-14T15:24:50.981587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887139+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:24:50.887139+0000","last_peered":"2024-05-14T15:24:50.887139+0000","last_clean":"2024-05-14T15:24:50.887139+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:24:50.887139+0000","last_undegraded":"2024-05-14T15:24:50.887139+0000","last_fullsized":"2024-05-14T15:24:50.887139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.735953+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:24:50.735953+0000","last_peered":"2024-05-14T15:24:50.735953+0000","last_clean":"2024-05-14T15:24:50.735953+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:24:50.735953+0000","last_undegraded":"2024-05-14T15:24:50.735953+0000","last_fullsized":"2024-05-14T15:24:50.735953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981579+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:24:50.981579+0000","last_peered":"2024-05-14T15:24:50.981579+0000","last_clean":"2024-05-14T15:24:50.981579+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:24:50.981579+0000","last_undegraded":"2024-05-14T15:24:50.981579+0000","last_fullsized":"2024-05-14T15:24:50.981579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887352+0000","last_change":"2024-05-14T15:22:42.903272+0000","last_active":"2024-05-14T15:24:50.887352+0000","last_peered":"2024-05-14T15:24:50.887352+0000","last_clean":"2024-05-14T15:24:50.887352+0000","last_became_active":"2024-05-14T15:22:42.900088+0000","last_became_peered":"2024-05-14T15:22:42.900088+0000","last_unstale":"2024-05-14T15:24:50.887352+0000","last_undegraded":"2024-05-14T15:24:50.887352+0000","last_fullsized":"2024-05-14T15:24:50.887352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:23.975904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003825+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:24:51.003825+0000","last_peered":"2024-05-14T15:24:51.003825+0000","last_clean":"2024-05-14T15:24:51.003825+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:24:51.003825+0000","last_undegraded":"2024-05-14T15:24:51.003825+0000","last_fullsized":"2024-05-14T15:24:51.003825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725154+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:24:50.725154+0000","last_peered":"2024-05-14T15:24:50.725154+0000","last_clean":"2024-05-14T15:24:50.725154+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:24:50.725154+0000","last_undegraded":"2024-05-14T15:24:50.725154+0000","last_fullsized":"2024-05-14T15:24:50.725154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983138+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:24:50.983138+0000","last_peered":"2024-05-14T15:24:50.983138+0000","last_clean":"2024-05-14T15:24:50.983138+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:24:50.983138+0000","last_undegraded":"2024-05-14T15:24:50.983138+0000","last_fullsized":"2024-05-14T15:24:50.983138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724761+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:24:50.724761+0000","last_peered":"2024-05-14T15:24:50.724761+0000","last_clean":"2024-05-14T15:24:50.724761+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:24:50.724761+0000","last_undegraded":"2024-05-14T15:24:50.724761+0000","last_fullsized":"2024-05-14T15:24:50.724761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.855611+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:24:50.855611+0000","last_peered":"2024-05-14T15:24:50.855611+0000","last_clean":"2024-05-14T15:24:50.855611+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:24:50.855611+0000","last_undegraded":"2024-05-14T15:24:50.855611+0000","last_fullsized":"2024-05-14T15:24:50.855611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725675+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:24:50.725675+0000","last_peered":"2024-05-14T15:24:50.725675+0000","last_clean":"2024-05-14T15:24:50.725675+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:24:50.725675+0000","last_undegraded":"2024-05-14T15:24:50.725675+0000","last_fullsized":"2024-05-14T15:24:50.725675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619735+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:24:50.619735+0000","last_peered":"2024-05-14T15:24:50.619735+0000","last_clean":"2024-05-14T15:24:50.619735+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:24:50.619735+0000","last_undegraded":"2024-05-14T15:24:50.619735+0000","last_fullsized":"2024-05-14T15:24:50.619735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981860+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:24:50.981860+0000","last_peered":"2024-05-14T15:24:50.981860+0000","last_clean":"2024-05-14T15:24:50.981860+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:24:50.981860+0000","last_undegraded":"2024-05-14T15:24:50.981860+0000","last_fullsized":"2024-05-14T15:24:50.981860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982282+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:24:50.982282+0000","last_peered":"2024-05-14T15:24:50.982282+0000","last_clean":"2024-05-14T15:24:50.982282+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:24:50.982282+0000","last_undegraded":"2024-05-14T15:24:50.982282+0000","last_fullsized":"2024-05-14T15:24:50.982282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981334+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:24:50.981334+0000","last_peered":"2024-05-14T15:24:50.981334+0000","last_clean":"2024-05-14T15:24:50.981334+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:24:50.981334+0000","last_undegraded":"2024-05-14T15:24:50.981334+0000","last_fullsized":"2024-05-14T15:24:50.981334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617877+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:24:50.617877+0000","last_peered":"2024-05-14T15:24:50.617877+0000","last_clean":"2024-05-14T15:24:50.617877+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:24:50.617877+0000","last_undegraded":"2024-05-14T15:24:50.617877+0000","last_fullsized":"2024-05-14T15:24:50.617877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724725+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:24:50.724725+0000","last_peered":"2024-05-14T15:24:50.724725+0000","last_clean":"2024-05-14T15:24:50.724725+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:24:50.724725+0000","last_undegraded":"2024-05-14T15:24:50.724725+0000","last_fullsized":"2024-05-14T15:24:50.724725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616775+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:24:50.616775+0000","last_peered":"2024-05-14T15:24:50.616775+0000","last_clean":"2024-05-14T15:24:50.616775+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:24:50.616775+0000","last_undegraded":"2024-05-14T15:24:50.616775+0000","last_fullsized":"2024-05-14T15:24:50.616775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724872+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:24:50.724872+0000","last_peered":"2024-05-14T15:24:50.724872+0000","last_clean":"2024-05-14T15:24:50.724872+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:24:50.724872+0000","last_undegraded":"2024-05-14T15:24:50.724872+0000","last_fullsized":"2024-05-14T15:24:50.724872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003974+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:24:51.003974+0000","last_peered":"2024-05-14T15:24:51.003974+0000","last_clean":"2024-05-14T15:24:51.003974+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:24:51.003974+0000","last_undegraded":"2024-05-14T15:24:51.003974+0000","last_fullsized":"2024-05-14T15:24:51.003974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981584+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:24:50.981584+0000","last_peered":"2024-05-14T15:24:50.981584+0000","last_clean":"2024-05-14T15:24:50.981584+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:24:50.981584+0000","last_undegraded":"2024-05-14T15:24:50.981584+0000","last_fullsized":"2024-05-14T15:24:50.981584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619978+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:24:50.619978+0000","last_peered":"2024-05-14T15:24:50.619978+0000","last_clean":"2024-05-14T15:24:50.619978+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:24:50.619978+0000","last_undegraded":"2024-05-14T15:24:50.619978+0000","last_fullsized":"2024-05-14T15:24:50.619978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725647+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:24:50.725647+0000","last_peered":"2024-05-14T15:24:50.725647+0000","last_clean":"2024-05-14T15:24:50.725647+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:24:50.725647+0000","last_undegraded":"2024-05-14T15:24:50.725647+0000","last_fullsized":"2024-05-14T15:24:50.725647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981327+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:24:50.981327+0000","last_peered":"2024-05-14T15:24:50.981327+0000","last_clean":"2024-05-14T15:24:50.981327+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:24:50.981327+0000","last_undegraded":"2024-05-14T15:24:50.981327+0000","last_fullsized":"2024-05-14T15:24:50.981327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724801+0000","last_change":"2024-05-14T15:22:42.901814+0000","last_active":"2024-05-14T15:24:50.724801+0000","last_peered":"2024-05-14T15:24:50.724801+0000","last_clean":"2024-05-14T15:24:50.724801+0000","last_became_active":"2024-05-14T15:22:42.896795+0000","last_became_peered":"2024-05-14T15:22:42.896795+0000","last_unstale":"2024-05-14T15:24:50.724801+0000","last_undegraded":"2024-05-14T15:24:50.724801+0000","last_fullsized":"2024-05-14T15:24:50.724801+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:15.981532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887356+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:24:50.887356+0000","last_peered":"2024-05-14T15:24:50.887356+0000","last_clean":"2024-05-14T15:24:50.887356+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:24:50.887356+0000","last_undegraded":"2024-05-14T15:24:50.887356+0000","last_fullsized":"2024-05-14T15:24:50.887356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617801+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:24:50.617801+0000","last_peered":"2024-05-14T15:24:50.617801+0000","last_clean":"2024-05-14T15:24:50.617801+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:24:50.617801+0000","last_undegraded":"2024-05-14T15:24:50.617801+0000","last_fullsized":"2024-05-14T15:24:50.617801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621026+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:24:50.621026+0000","last_peered":"2024-05-14T15:24:50.621026+0000","last_clean":"2024-05-14T15:24:50.621026+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:24:50.621026+0000","last_undegraded":"2024-05-14T15:24:50.621026+0000","last_fullsized":"2024-05-14T15:24:50.621026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887834+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:24:50.887834+0000","last_peered":"2024-05-14T15:24:50.887834+0000","last_clean":"2024-05-14T15:24:50.887834+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:24:50.887834+0000","last_undegraded":"2024-05-14T15:24:50.887834+0000","last_fullsized":"2024-05-14T15:24:50.887834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981549+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:24:50.981549+0000","last_peered":"2024-05-14T15:24:50.981549+0000","last_clean":"2024-05-14T15:24:50.981549+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:24:50.981549+0000","last_undegraded":"2024-05-14T15:24:50.981549+0000","last_fullsized":"2024-05-14T15:24:50.981549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":93,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.850897+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:24:50.850897+0000","last_peered":"2024-05-14T15:24:50.850897+0000","last_clean":"2024-05-14T15:24:50.850897+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:24:50.850897+0000","last_undegraded":"2024-05-14T15:24:50.850897+0000","last_fullsized":"2024-05-14T15:24:50.850897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887586+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:24:50.887586+0000","last_peered":"2024-05-14T15:24:50.887586+0000","last_clean":"2024-05-14T15:24:50.887586+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:24:50.887586+0000","last_undegraded":"2024-05-14T15:24:50.887586+0000","last_fullsized":"2024-05-14T15:24:50.887586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736013+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:24:50.736013+0000","last_peered":"2024-05-14T15:24:50.736013+0000","last_clean":"2024-05-14T15:24:50.736013+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:24:50.736013+0000","last_undegraded":"2024-05-14T15:24:50.736013+0000","last_fullsized":"2024-05-14T15:24:50.736013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983098+0000","last_change":"2024-05-14T15:22:43.235879+0000","last_active":"2024-05-14T15:24:50.983098+0000","last_peered":"2024-05-14T15:24:50.983098+0000","last_clean":"2024-05-14T15:24:50.983098+0000","last_became_active":"2024-05-14T15:22:43.230116+0000","last_became_peered":"2024-05-14T15:22:43.230116+0000","last_unstale":"2024-05-14T15:24:50.983098+0000","last_undegraded":"2024-05-14T15:24:50.983098+0000","last_fullsized":"2024-05-14T15:24:50.983098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:25:50.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887809+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:24:50.887809+0000","last_peered":"2024-05-14T15:24:50.887809+0000","last_clean":"2024-05-14T15:24:50.887809+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:24:50.887809+0000","last_undegraded":"2024-05-14T15:24:50.887809+0000","last_fullsized":"2024-05-14T15:24:50.887809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004023+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:24:51.004023+0000","last_peered":"2024-05-14T15:24:51.004023+0000","last_clean":"2024-05-14T15:24:51.004023+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:24:51.004023+0000","last_undegraded":"2024-05-14T15:24:51.004023+0000","last_fullsized":"2024-05-14T15:24:51.004023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619978+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:24:50.619978+0000","last_peered":"2024-05-14T15:24:50.619978+0000","last_clean":"2024-05-14T15:24:50.619978+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:24:50.619978+0000","last_undegraded":"2024-05-14T15:24:50.619978+0000","last_fullsized":"2024-05-14T15:24:50.619978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619801+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:24:50.619801+0000","last_peered":"2024-05-14T15:24:50.619801+0000","last_clean":"2024-05-14T15:24:50.619801+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:24:50.619801+0000","last_undegraded":"2024-05-14T15:24:50.619801+0000","last_fullsized":"2024-05-14T15:24:50.619801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982632+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:24:50.982632+0000","last_peered":"2024-05-14T15:24:50.982632+0000","last_clean":"2024-05-14T15:24:50.982632+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:24:50.982632+0000","last_undegraded":"2024-05-14T15:24:50.982632+0000","last_fullsized":"2024-05-14T15:24:50.982632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983315+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:24:50.983315+0000","last_peered":"2024-05-14T15:24:50.983315+0000","last_clean":"2024-05-14T15:24:50.983315+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:24:50.983315+0000","last_undegraded":"2024-05-14T15:24:50.983315+0000","last_fullsized":"2024-05-14T15:24:50.983315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004200+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:24:51.004200+0000","last_peered":"2024-05-14T15:24:51.004200+0000","last_clean":"2024-05-14T15:24:51.004200+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:24:51.004200+0000","last_undegraded":"2024-05-14T15:24:51.004200+0000","last_fullsized":"2024-05-14T15:24:51.004200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621083+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:24:50.621083+0000","last_peered":"2024-05-14T15:24:50.621083+0000","last_clean":"2024-05-14T15:24:50.621083+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:24:50.621083+0000","last_undegraded":"2024-05-14T15:24:50.621083+0000","last_fullsized":"2024-05-14T15:24:50.621083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725796+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:24:50.725796+0000","last_peered":"2024-05-14T15:24:50.725796+0000","last_clean":"2024-05-14T15:24:50.725796+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:24:50.725796+0000","last_undegraded":"2024-05-14T15:24:50.725796+0000","last_fullsized":"2024-05-14T15:24:50.725796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736083+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:24:50.736083+0000","last_peered":"2024-05-14T15:24:50.736083+0000","last_clean":"2024-05-14T15:24:50.736083+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:24:50.736083+0000","last_undegraded":"2024-05-14T15:24:50.736083+0000","last_fullsized":"2024-05-14T15:24:50.736083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724963+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:24:50.724963+0000","last_peered":"2024-05-14T15:24:50.724963+0000","last_clean":"2024-05-14T15:24:50.724963+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:24:50.724963+0000","last_undegraded":"2024-05-14T15:24:50.724963+0000","last_fullsized":"2024-05-14T15:24:50.724963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617965+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:24:50.617965+0000","last_peered":"2024-05-14T15:24:50.617965+0000","last_clean":"2024-05-14T15:24:50.617965+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:24:50.617965+0000","last_undegraded":"2024-05-14T15:24:50.617965+0000","last_fullsized":"2024-05-14T15:24:50.617965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982527+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:24:50.982527+0000","last_peered":"2024-05-14T15:24:50.982527+0000","last_clean":"2024-05-14T15:24:50.982527+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:24:50.982527+0000","last_undegraded":"2024-05-14T15:24:50.982527+0000","last_fullsized":"2024-05-14T15:24:50.982527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004115+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:24:51.004115+0000","last_peered":"2024-05-14T15:24:51.004115+0000","last_clean":"2024-05-14T15:24:51.004115+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:24:51.004115+0000","last_undegraded":"2024-05-14T15:24:51.004115+0000","last_fullsized":"2024-05-14T15:24:51.004115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108306,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599109,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1169,"num_read_kb":697,"num_write":295,"num_write_kb":1443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1797327,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678079,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":528169,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874307,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1171456,"data_stored":1316406,"data_compressed":77588,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":925717,"data_compressed":54476,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234063,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":921541,"data_compressed":54476,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430291,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":1125157,"data_compressed":71432,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593815,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":513649,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":592078,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":524969,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":595524,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1048591,"data_compressed":60632,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953570,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":528231,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790044,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1252345,"data_compressed":77588,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134390,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202547,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:24:57.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T15:24:57.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:56 smithi144 ceph-mon[35650]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.210 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:56 smithi144 ceph-mon[35650]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.211 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:56 smithi144 ceph-mon[35650]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.211 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:56 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2292931077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:57.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:56 smithi193 ceph-mon[36001]: Updating smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:56 smithi193 ceph-mon[36001]: Updating smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:56 smithi193 ceph-mon[36001]: Updating smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T15:24:57.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:56 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2292931077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1700652623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 msgr2=0x7fa7181088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1700652623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181088b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa708009900 tx=0x7fa70802a410 comp rx=0 tx=0).stop 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1700652623 shutdown_connections 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1700652623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa718108f40 0x7fa7181106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1700652623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181088b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1700652623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa718102420 0x7fa718102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1700652623 >> 172.21.15.5:0/1700652623 conn(0x7fa718078a50 msgr2=0x7fa718078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1700652623 shutdown_connections 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1700652623 wait complete. 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.285+0000 7fa71fe0c640 1 Processor -- start 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 -- start start 2024-05-14T15:24:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa718102420 0x7fa7181a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa718108f40 0x7fa71819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa718113e50 con 0x7fa7181084a0 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa718113cd0 con 0x7fa718108f40 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71fe0c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa718113fd0 con 0x7fa718102420 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60666/0 (socket says 172.21.15.5:60666) 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 -- 172.21.15.5:0/1786888582 learned_addr learned my addr 172.21.15.5:0/1786888582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71dba8640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa718102420 0x7fa7181a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71e3a9640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa718108f40 0x7fa71819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 -- 172.21.15.5:0/1786888582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa718102420 msgr2=0x7fa7181a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa718102420 0x7fa7181a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 -- 172.21.15.5:0/1786888582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa718108f40 msgr2=0x7fa71819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa718108f40 0x7fa71819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 -- 172.21.15.5:0/1786888582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71819ade0 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.286+0000 7fa71d3a7640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181a0a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa708006010 tx=0x7fa708004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.287+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70803f030 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.287+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa708003aa0 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.287+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71819b010 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.287+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708035070 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.287+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a72d0 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.288+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718105080 con 0x7fa7181084a0 2024-05-14T15:24:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.289+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fa708042030 con 0x7fa7181084a0 2024-05-14T15:24:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.291+0000 7fa70effd640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa7040b2b00 0x7fa7040b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.291+0000 7fa71dba8640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa7040b2b00 0x7fa7040b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.291+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa7081033a0 con 0x7fa7181084a0 2024-05-14T15:24:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.291+0000 7fa71dba8640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa7040b2b00 0x7fa7040b5010 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa71806af80 tx=0x7fa71400b390 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.295+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7080ca1f0 con 0x7fa7181084a0 2024-05-14T15:24:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.482+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa718101110 con 0x7fa7040b2b00 2024-05-14T15:24:57.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.491+0000 7fa70effd640 1 -- 172.21.15.5:0/1786888582 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339652 (secure 0 0 0) 0x7fa718101110 con 0x7fa7040b2b00 2024-05-14T15:24:57.497 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa7040b2b00 msgr2=0x7fa7040b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa7040b2b00 0x7fa7040b5010 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa71806af80 tx=0x7fa71400b390 comp rx=0 tx=0).stop 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 msgr2=0x7fa7181a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181a0a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa708006010 tx=0x7fa708004060 comp rx=0 tx=0).stop 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 shutdown_connections 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa7040b2b00 0x7fa7040b5010 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa718108f40 0x7fa71819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7181084a0 0x7fa7181a0a70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 --2- 172.21.15.5:0/1786888582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa718102420 0x7fa7181a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 >> 172.21.15.5:0/1786888582 conn(0x7fa718078a50 msgr2=0x7fa718100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 shutdown_connections 2024-05-14T15:24:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.495+0000 7fa71fe0c640 1 -- 172.21.15.5:0/1786888582 wait complete. 2024-05-14T15:24:57.508 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-14T15:24:56.633270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189711,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1250,"num_read_kb":766,"num_write":420,"num_write_kb":3595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378176,"kb_used_data":12344,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563648,"statfs":{"total":1151940427776,"available":1151553175552,"internally_reserved":0,"allocated":12640256,"data_stored":9872377,"data_compressed":605208,"data_compressed_allocated":819200,"data_compressed_original":4489216,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990686"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004427+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:24:51.004427+0000","last_peered":"2024-05-14T15:24:51.004427+0000","last_clean":"2024-05-14T15:24:51.004427+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:24:51.004427+0000","last_undegraded":"2024-05-14T15:24:51.004427+0000","last_fullsized":"2024-05-14T15:24:51.004427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982134+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:24:50.982134+0000","last_peered":"2024-05-14T15:24:50.982134+0000","last_clean":"2024-05-14T15:24:50.982134+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:24:50.982134+0000","last_undegraded":"2024-05-14T15:24:50.982134+0000","last_fullsized":"2024-05-14T15:24:50.982134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981937+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:24:50.981937+0000","last_peered":"2024-05-14T15:24:50.981937+0000","last_clean":"2024-05-14T15:24:50.981937+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:24:50.981937+0000","last_undegraded":"2024-05-14T15:24:50.981937+0000","last_fullsized":"2024-05-14T15:24:50.981937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981876+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:24:50.981876+0000","last_peered":"2024-05-14T15:24:50.981876+0000","last_clean":"2024-05-14T15:24:50.981876+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:24:50.981876+0000","last_undegraded":"2024-05-14T15:24:50.981876+0000","last_fullsized":"2024-05-14T15:24:50.981876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724901+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:24:50.724901+0000","last_peered":"2024-05-14T15:24:50.724901+0000","last_clean":"2024-05-14T15:24:50.724901+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:24:50.724901+0000","last_undegraded":"2024-05-14T15:24:50.724901+0000","last_fullsized":"2024-05-14T15:24:50.724901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739533+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:24:50.739533+0000","last_peered":"2024-05-14T15:24:50.739533+0000","last_clean":"2024-05-14T15:24:50.739533+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:24:50.739533+0000","last_undegraded":"2024-05-14T15:24:50.739533+0000","last_fullsized":"2024-05-14T15:24:50.739533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887231+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:24:50.887231+0000","last_peered":"2024-05-14T15:24:50.887231+0000","last_clean":"2024-05-14T15:24:50.887231+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:24:50.887231+0000","last_undegraded":"2024-05-14T15:24:50.887231+0000","last_fullsized":"2024-05-14T15:24:50.887231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725049+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:24:50.725049+0000","last_peered":"2024-05-14T15:24:50.725049+0000","last_clean":"2024-05-14T15:24:50.725049+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:24:50.725049+0000","last_undegraded":"2024-05-14T15:24:50.725049+0000","last_fullsized":"2024-05-14T15:24:50.725049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618484+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:24:50.618484+0000","last_peered":"2024-05-14T15:24:50.618484+0000","last_clean":"2024-05-14T15:24:50.618484+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:24:50.618484+0000","last_undegraded":"2024-05-14T15:24:50.618484+0000","last_fullsized":"2024-05-14T15:24:50.618484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739931+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:24:50.739931+0000","last_peered":"2024-05-14T15:24:50.739931+0000","last_clean":"2024-05-14T15:24:50.739931+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:24:50.739931+0000","last_undegraded":"2024-05-14T15:24:50.739931+0000","last_fullsized":"2024-05-14T15:24:50.739931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887724+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:24:50.887724+0000","last_peered":"2024-05-14T15:24:50.887724+0000","last_clean":"2024-05-14T15:24:50.887724+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:24:50.887724+0000","last_undegraded":"2024-05-14T15:24:50.887724+0000","last_fullsized":"2024-05-14T15:24:50.887724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981832+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:24:50.981832+0000","last_peered":"2024-05-14T15:24:50.981832+0000","last_clean":"2024-05-14T15:24:50.981832+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:24:50.981832+0000","last_undegraded":"2024-05-14T15:24:50.981832+0000","last_fullsized":"2024-05-14T15:24:50.981832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982434+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:24:50.982434+0000","last_peered":"2024-05-14T15:24:50.982434+0000","last_clean":"2024-05-14T15:24:50.982434+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:24:50.982434+0000","last_undegraded":"2024-05-14T15:24:50.982434+0000","last_fullsized":"2024-05-14T15:24:50.982434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981531+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:24:50.981531+0000","last_peered":"2024-05-14T15:24:50.981531+0000","last_clean":"2024-05-14T15:24:50.981531+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:24:50.981531+0000","last_undegraded":"2024-05-14T15:24:50.981531+0000","last_fullsized":"2024-05-14T15:24:50.981531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981623+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:24:50.981623+0000","last_peered":"2024-05-14T15:24:50.981623+0000","last_clean":"2024-05-14T15:24:50.981623+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:24:50.981623+0000","last_undegraded":"2024-05-14T15:24:50.981623+0000","last_fullsized":"2024-05-14T15:24:50.981623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617800+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:24:50.617800+0000","last_peered":"2024-05-14T15:24:50.617800+0000","last_clean":"2024-05-14T15:24:50.617800+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:24:50.617800+0000","last_undegraded":"2024-05-14T15:24:50.617800+0000","last_fullsized":"2024-05-14T15:24:50.617800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004331+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:24:51.004331+0000","last_peered":"2024-05-14T15:24:51.004331+0000","last_clean":"2024-05-14T15:24:51.004331+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:24:51.004331+0000","last_undegraded":"2024-05-14T15:24:51.004331+0000","last_fullsized":"2024-05-14T15:24:51.004331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616451+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:24:50.616451+0000","last_peered":"2024-05-14T15:24:50.616451+0000","last_clean":"2024-05-14T15:24:50.616451+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:24:50.616451+0000","last_undegraded":"2024-05-14T15:24:50.616451+0000","last_fullsized":"2024-05-14T15:24:50.616451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621311+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:24:50.621311+0000","last_peered":"2024-05-14T15:24:50.621311+0000","last_clean":"2024-05-14T15:24:50.621311+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:24:50.621311+0000","last_undegraded":"2024-05-14T15:24:50.621311+0000","last_fullsized":"2024-05-14T15:24:50.621311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619740+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:24:50.619740+0000","last_peered":"2024-05-14T15:24:50.619740+0000","last_clean":"2024-05-14T15:24:50.619740+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:24:50.619740+0000","last_undegraded":"2024-05-14T15:24:50.619740+0000","last_fullsized":"2024-05-14T15:24:50.619740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616807+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:24:50.616807+0000","last_peered":"2024-05-14T15:24:50.616807+0000","last_clean":"2024-05-14T15:24:50.616807+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:24:50.616807+0000","last_undegraded":"2024-05-14T15:24:50.616807+0000","last_fullsized":"2024-05-14T15:24:50.616807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725463+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:24:50.725463+0000","last_peered":"2024-05-14T15:24:50.725463+0000","last_clean":"2024-05-14T15:24:50.725463+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:24:50.725463+0000","last_undegraded":"2024-05-14T15:24:50.725463+0000","last_fullsized":"2024-05-14T15:24:50.725463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981463+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:24:50.981463+0000","last_peered":"2024-05-14T15:24:50.981463+0000","last_clean":"2024-05-14T15:24:50.981463+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:24:50.981463+0000","last_undegraded":"2024-05-14T15:24:50.981463+0000","last_fullsized":"2024-05-14T15:24:50.981463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725622+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:24:50.725622+0000","last_peered":"2024-05-14T15:24:50.725622+0000","last_clean":"2024-05-14T15:24:50.725622+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:24:50.725622+0000","last_undegraded":"2024-05-14T15:24:50.725622+0000","last_fullsized":"2024-05-14T15:24:50.725622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982136+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:24:50.982136+0000","last_peered":"2024-05-14T15:24:50.982136+0000","last_clean":"2024-05-14T15:24:50.982136+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:24:50.982136+0000","last_undegraded":"2024-05-14T15:24:50.982136+0000","last_fullsized":"2024-05-14T15:24:50.982136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981579+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:24:50.981579+0000","last_peered":"2024-05-14T15:24:50.981579+0000","last_clean":"2024-05-14T15:24:50.981579+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:24:50.981579+0000","last_undegraded":"2024-05-14T15:24:50.981579+0000","last_fullsized":"2024-05-14T15:24:50.981579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725311+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:24:50.725311+0000","last_peered":"2024-05-14T15:24:50.725311+0000","last_clean":"2024-05-14T15:24:50.725311+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:24:50.725311+0000","last_undegraded":"2024-05-14T15:24:50.725311+0000","last_fullsized":"2024-05-14T15:24:50.725311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618077+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:24:50.618077+0000","last_peered":"2024-05-14T15:24:50.618077+0000","last_clean":"2024-05-14T15:24:50.618077+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:24:50.618077+0000","last_undegraded":"2024-05-14T15:24:50.618077+0000","last_fullsized":"2024-05-14T15:24:50.618077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981599+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:24:50.981599+0000","last_peered":"2024-05-14T15:24:50.981599+0000","last_clean":"2024-05-14T15:24:50.981599+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:24:50.981599+0000","last_undegraded":"2024-05-14T15:24:50.981599+0000","last_fullsized":"2024-05-14T15:24:50.981599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725116+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:24:50.725116+0000","last_peered":"2024-05-14T15:24:50.725116+0000","last_clean":"2024-05-14T15:24:50.725116+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:24:50.725116+0000","last_undegraded":"2024-05-14T15:24:50.725116+0000","last_fullsized":"2024-05-14T15:24:50.725116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736594+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:24:50.736594+0000","last_peered":"2024-05-14T15:24:50.736594+0000","last_clean":"2024-05-14T15:24:50.736594+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:24:50.736594+0000","last_undegraded":"2024-05-14T15:24:50.736594+0000","last_fullsized":"2024-05-14T15:24:50.736594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621169+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:24:50.621169+0000","last_peered":"2024-05-14T15:24:50.621169+0000","last_clean":"2024-05-14T15:24:50.621169+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:24:50.621169+0000","last_undegraded":"2024-05-14T15:24:50.621169+0000","last_fullsized":"2024-05-14T15:24:50.621169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.036543+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:24:51.036543+0000","last_peered":"2024-05-14T15:24:51.036543+0000","last_clean":"2024-05-14T15:24:51.036543+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:24:51.036543+0000","last_undegraded":"2024-05-14T15:24:51.036543+0000","last_fullsized":"2024-05-14T15:24:51.036543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724843+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:24:50.724843+0000","last_peered":"2024-05-14T15:24:50.724843+0000","last_clean":"2024-05-14T15:24:50.724843+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:24:50.724843+0000","last_undegraded":"2024-05-14T15:24:50.724843+0000","last_fullsized":"2024-05-14T15:24:50.724843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.620804+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:24:50.620804+0000","last_peered":"2024-05-14T15:24:50.620804+0000","last_clean":"2024-05-14T15:24:50.620804+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:24:50.620804+0000","last_undegraded":"2024-05-14T15:24:50.620804+0000","last_fullsized":"2024-05-14T15:24:50.620804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739485+0000","last_change":"2024-05-14T15:22:43.233044+0000","last_active":"2024-05-14T15:24:50.739485+0000","last_peered":"2024-05-14T15:24:50.739485+0000","last_clean":"2024-05-14T15:24:50.739485+0000","last_became_active":"2024-05-14T15:22:43.229808+0000","last_became_peered":"2024-05-14T15:22:43.229808+0000","last_unstale":"2024-05-14T15:24:50.739485+0000","last_undegraded":"2024-05-14T15:24:50.739485+0000","last_fullsized":"2024-05-14T15:24:50.739485+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:07.226460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736522+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:24:50.736522+0000","last_peered":"2024-05-14T15:24:50.736522+0000","last_clean":"2024-05-14T15:24:50.736522+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:24:50.736522+0000","last_undegraded":"2024-05-14T15:24:50.736522+0000","last_fullsized":"2024-05-14T15:24:50.736522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003934+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:24:51.003934+0000","last_peered":"2024-05-14T15:24:51.003934+0000","last_clean":"2024-05-14T15:24:51.003934+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:24:51.003934+0000","last_undegraded":"2024-05-14T15:24:51.003934+0000","last_fullsized":"2024-05-14T15:24:51.003934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981845+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:24:50.981845+0000","last_peered":"2024-05-14T15:24:50.981845+0000","last_clean":"2024-05-14T15:24:50.981845+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:24:50.981845+0000","last_undegraded":"2024-05-14T15:24:50.981845+0000","last_fullsized":"2024-05-14T15:24:50.981845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724897+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:24:50.724897+0000","last_peered":"2024-05-14T15:24:50.724897+0000","last_clean":"2024-05-14T15:24:50.724897+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:24:50.724897+0000","last_undegraded":"2024-05-14T15:24:50.724897+0000","last_fullsized":"2024-05-14T15:24:50.724897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982168+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:24:50.982168+0000","last_peered":"2024-05-14T15:24:50.982168+0000","last_clean":"2024-05-14T15:24:50.982168+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:24:50.982168+0000","last_undegraded":"2024-05-14T15:24:50.982168+0000","last_fullsized":"2024-05-14T15:24:50.982168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003726+0000","last_change":"2024-05-14T15:22:42.900544+0000","last_active":"2024-05-14T15:24:51.003726+0000","last_peered":"2024-05-14T15:24:51.003726+0000","last_clean":"2024-05-14T15:24:51.003726+0000","last_became_active":"2024-05-14T15:22:42.900015+0000","last_became_peered":"2024-05-14T15:22:42.900015+0000","last_unstale":"2024-05-14T15:24:51.003726+0000","last_undegraded":"2024-05-14T15:24:51.003726+0000","last_fullsized":"2024-05-14T15:24:51.003726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:15.904051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616117+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:24:50.616117+0000","last_peered":"2024-05-14T15:24:50.616117+0000","last_clean":"2024-05-14T15:24:50.616117+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:24:50.616117+0000","last_undegraded":"2024-05-14T15:24:50.616117+0000","last_fullsized":"2024-05-14T15:24:50.616117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724836+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:24:50.724836+0000","last_peered":"2024-05-14T15:24:50.724836+0000","last_clean":"2024-05-14T15:24:50.724836+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:24:50.724836+0000","last_undegraded":"2024-05-14T15:24:50.724836+0000","last_fullsized":"2024-05-14T15:24:50.724836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.886998+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:24:50.886998+0000","last_peered":"2024-05-14T15:24:50.886998+0000","last_clean":"2024-05-14T15:24:50.886998+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:24:50.886998+0000","last_undegraded":"2024-05-14T15:24:50.886998+0000","last_fullsized":"2024-05-14T15:24:50.886998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618501+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:24:50.618501+0000","last_peered":"2024-05-14T15:24:50.618501+0000","last_clean":"2024-05-14T15:24:50.618501+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:24:50.618501+0000","last_undegraded":"2024-05-14T15:24:50.618501+0000","last_fullsized":"2024-05-14T15:24:50.618501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.615998+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:24:50.615998+0000","last_peered":"2024-05-14T15:24:50.615998+0000","last_clean":"2024-05-14T15:24:50.615998+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:24:50.615998+0000","last_undegraded":"2024-05-14T15:24:50.615998+0000","last_fullsized":"2024-05-14T15:24:50.615998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887655+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:24:50.887655+0000","last_peered":"2024-05-14T15:24:50.887655+0000","last_clean":"2024-05-14T15:24:50.887655+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:24:50.887655+0000","last_undegraded":"2024-05-14T15:24:50.887655+0000","last_fullsized":"2024-05-14T15:24:50.887655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982157+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:24:50.982157+0000","last_peered":"2024-05-14T15:24:50.982157+0000","last_clean":"2024-05-14T15:24:50.982157+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:24:50.982157+0000","last_undegraded":"2024-05-14T15:24:50.982157+0000","last_fullsized":"2024-05-14T15:24:50.982157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981722+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:24:50.981722+0000","last_peered":"2024-05-14T15:24:50.981722+0000","last_clean":"2024-05-14T15:24:50.981722+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:24:50.981722+0000","last_undegraded":"2024-05-14T15:24:50.981722+0000","last_fullsized":"2024-05-14T15:24:50.981722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736561+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:24:50.736561+0000","last_peered":"2024-05-14T15:24:50.736561+0000","last_clean":"2024-05-14T15:24:50.736561+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:24:50.736561+0000","last_undegraded":"2024-05-14T15:24:50.736561+0000","last_fullsized":"2024-05-14T15:24:50.736561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618076+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:24:50.618076+0000","last_peered":"2024-05-14T15:24:50.618076+0000","last_clean":"2024-05-14T15:24:50.618076+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:24:50.618076+0000","last_undegraded":"2024-05-14T15:24:50.618076+0000","last_fullsized":"2024-05-14T15:24:50.618076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616173+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:24:50.616173+0000","last_peered":"2024-05-14T15:24:50.616173+0000","last_clean":"2024-05-14T15:24:50.616173+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:24:50.616173+0000","last_undegraded":"2024-05-14T15:24:50.616173+0000","last_fullsized":"2024-05-14T15:24:50.616173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":447,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.036303+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:24:51.036303+0000","last_peered":"2024-05-14T15:24:51.036303+0000","last_clean":"2024-05-14T15:24:51.036303+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:24:51.036303+0000","last_undegraded":"2024-05-14T15:24:51.036303+0000","last_fullsized":"2024-05-14T15:24:51.036303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69369,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":249,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725097+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:24:50.725097+0000","last_peered":"2024-05-14T15:24:50.725097+0000","last_clean":"2024-05-14T15:24:50.725097+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:24:50.725097+0000","last_undegraded":"2024-05-14T15:24:50.725097+0000","last_fullsized":"2024-05-14T15:24:50.725097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736912+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:24:50.736912+0000","last_peered":"2024-05-14T15:24:50.736912+0000","last_clean":"2024-05-14T15:24:50.736912+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:24:50.736912+0000","last_undegraded":"2024-05-14T15:24:50.736912+0000","last_fullsized":"2024-05-14T15:24:50.736912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.620086+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:24:50.620086+0000","last_peered":"2024-05-14T15:24:50.620086+0000","last_clean":"2024-05-14T15:24:50.620086+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:24:50.620086+0000","last_undegraded":"2024-05-14T15:24:50.620086+0000","last_fullsized":"2024-05-14T15:24:50.620086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982061+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:24:50.982061+0000","last_peered":"2024-05-14T15:24:50.982061+0000","last_clean":"2024-05-14T15:24:50.982061+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:24:50.982061+0000","last_undegraded":"2024-05-14T15:24:50.982061+0000","last_fullsized":"2024-05-14T15:24:50.982061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003977+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:24:51.003977+0000","last_peered":"2024-05-14T15:24:51.003977+0000","last_clean":"2024-05-14T15:24:51.003977+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:24:51.003977+0000","last_undegraded":"2024-05-14T15:24:51.003977+0000","last_fullsized":"2024-05-14T15:24:51.003977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736331+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:24:50.736331+0000","last_peered":"2024-05-14T15:24:50.736331+0000","last_clean":"2024-05-14T15:24:50.736331+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:24:50.736331+0000","last_undegraded":"2024-05-14T15:24:50.736331+0000","last_fullsized":"2024-05-14T15:24:50.736331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887498+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:24:50.887498+0000","last_peered":"2024-05-14T15:24:50.887498+0000","last_clean":"2024-05-14T15:24:50.887498+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:24:50.887498+0000","last_undegraded":"2024-05-14T15:24:50.887498+0000","last_fullsized":"2024-05-14T15:24:50.887498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616369+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:24:50.616369+0000","last_peered":"2024-05-14T15:24:50.616369+0000","last_clean":"2024-05-14T15:24:50.616369+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:24:50.616369+0000","last_undegraded":"2024-05-14T15:24:50.616369+0000","last_fullsized":"2024-05-14T15:24:50.616369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982014+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:24:50.982014+0000","last_peered":"2024-05-14T15:24:50.982014+0000","last_clean":"2024-05-14T15:24:50.982014+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:24:50.982014+0000","last_undegraded":"2024-05-14T15:24:50.982014+0000","last_fullsized":"2024-05-14T15:24:50.982014+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887383+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:24:50.887383+0000","last_peered":"2024-05-14T15:24:50.887383+0000","last_clean":"2024-05-14T15:24:50.887383+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:24:50.887383+0000","last_undegraded":"2024-05-14T15:24:50.887383+0000","last_fullsized":"2024-05-14T15:24:50.887383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982928+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:24:50.982928+0000","last_peered":"2024-05-14T15:24:50.982928+0000","last_clean":"2024-05-14T15:24:50.982928+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:24:50.982928+0000","last_undegraded":"2024-05-14T15:24:50.982928+0000","last_fullsized":"2024-05-14T15:24:50.982928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724711+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:24:50.724711+0000","last_peered":"2024-05-14T15:24:50.724711+0000","last_clean":"2024-05-14T15:24:50.724711+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:24:50.724711+0000","last_undegraded":"2024-05-14T15:24:50.724711+0000","last_fullsized":"2024-05-14T15:24:50.724711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736231+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:24:50.736231+0000","last_peered":"2024-05-14T15:24:50.736231+0000","last_clean":"2024-05-14T15:24:50.736231+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:24:50.736231+0000","last_undegraded":"2024-05-14T15:24:50.736231+0000","last_fullsized":"2024-05-14T15:24:50.736231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736232+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:24:50.736232+0000","last_peered":"2024-05-14T15:24:50.736232+0000","last_clean":"2024-05-14T15:24:50.736232+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:24:50.736232+0000","last_undegraded":"2024-05-14T15:24:50.736232+0000","last_fullsized":"2024-05-14T15:24:50.736232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725227+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:24:50.725227+0000","last_peered":"2024-05-14T15:24:50.725227+0000","last_clean":"2024-05-14T15:24:50.725227+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:24:50.725227+0000","last_undegraded":"2024-05-14T15:24:50.725227+0000","last_fullsized":"2024-05-14T15:24:50.725227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004205+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:24:51.004205+0000","last_peered":"2024-05-14T15:24:51.004205+0000","last_clean":"2024-05-14T15:24:51.004205+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:24:51.004205+0000","last_undegraded":"2024-05-14T15:24:51.004205+0000","last_fullsized":"2024-05-14T15:24:51.004205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724662+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:24:50.724662+0000","last_peered":"2024-05-14T15:24:50.724662+0000","last_clean":"2024-05-14T15:24:50.724662+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:24:50.724662+0000","last_undegraded":"2024-05-14T15:24:50.724662+0000","last_fullsized":"2024-05-14T15:24:50.724662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616333+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:24:50.616333+0000","last_peered":"2024-05-14T15:24:50.616333+0000","last_clean":"2024-05-14T15:24:50.616333+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:24:50.616333+0000","last_undegraded":"2024-05-14T15:24:50.616333+0000","last_fullsized":"2024-05-14T15:24:50.616333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739775+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:24:50.739775+0000","last_peered":"2024-05-14T15:24:50.739775+0000","last_clean":"2024-05-14T15:24:50.739775+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:24:50.739775+0000","last_undegraded":"2024-05-14T15:24:50.739775+0000","last_fullsized":"2024-05-14T15:24:50.739775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982918+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:24:50.982918+0000","last_peered":"2024-05-14T15:24:50.982918+0000","last_clean":"2024-05-14T15:24:50.982918+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:24:50.982918+0000","last_undegraded":"2024-05-14T15:24:50.982918+0000","last_fullsized":"2024-05-14T15:24:50.982918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887350+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:24:50.887350+0000","last_peered":"2024-05-14T15:24:50.887350+0000","last_clean":"2024-05-14T15:24:50.887350+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:24:50.887350+0000","last_undegraded":"2024-05-14T15:24:50.887350+0000","last_fullsized":"2024-05-14T15:24:50.887350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739500+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:24:50.739500+0000","last_peered":"2024-05-14T15:24:50.739500+0000","last_clean":"2024-05-14T15:24:50.739500+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:24:50.739500+0000","last_undegraded":"2024-05-14T15:24:50.739500+0000","last_fullsized":"2024-05-14T15:24:50.739500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983481+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:24:50.983481+0000","last_peered":"2024-05-14T15:24:50.983481+0000","last_clean":"2024-05-14T15:24:50.983481+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:24:50.983481+0000","last_undegraded":"2024-05-14T15:24:50.983481+0000","last_fullsized":"2024-05-14T15:24:50.983481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736244+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:24:50.736244+0000","last_peered":"2024-05-14T15:24:50.736244+0000","last_clean":"2024-05-14T15:24:50.736244+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:24:50.736244+0000","last_undegraded":"2024-05-14T15:24:50.736244+0000","last_fullsized":"2024-05-14T15:24:50.736244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981800+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:24:50.981800+0000","last_peered":"2024-05-14T15:24:50.981800+0000","last_clean":"2024-05-14T15:24:50.981800+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:24:50.981800+0000","last_undegraded":"2024-05-14T15:24:50.981800+0000","last_fullsized":"2024-05-14T15:24:50.981800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887629+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:24:50.887629+0000","last_peered":"2024-05-14T15:24:50.887629+0000","last_clean":"2024-05-14T15:24:50.887629+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:24:50.887629+0000","last_undegraded":"2024-05-14T15:24:50.887629+0000","last_fullsized":"2024-05-14T15:24:50.887629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725131+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:24:50.725131+0000","last_peered":"2024-05-14T15:24:50.725131+0000","last_clean":"2024-05-14T15:24:50.725131+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:24:50.725131+0000","last_undegraded":"2024-05-14T15:24:50.725131+0000","last_fullsized":"2024-05-14T15:24:50.725131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981587+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:24:50.981587+0000","last_peered":"2024-05-14T15:24:50.981587+0000","last_clean":"2024-05-14T15:24:50.981587+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:24:50.981587+0000","last_undegraded":"2024-05-14T15:24:50.981587+0000","last_fullsized":"2024-05-14T15:24:50.981587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887139+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:24:50.887139+0000","last_peered":"2024-05-14T15:24:50.887139+0000","last_clean":"2024-05-14T15:24:50.887139+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:24:50.887139+0000","last_undegraded":"2024-05-14T15:24:50.887139+0000","last_fullsized":"2024-05-14T15:24:50.887139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.735953+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:24:50.735953+0000","last_peered":"2024-05-14T15:24:50.735953+0000","last_clean":"2024-05-14T15:24:50.735953+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:24:50.735953+0000","last_undegraded":"2024-05-14T15:24:50.735953+0000","last_fullsized":"2024-05-14T15:24:50.735953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981579+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:24:50.981579+0000","last_peered":"2024-05-14T15:24:50.981579+0000","last_clean":"2024-05-14T15:24:50.981579+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:24:50.981579+0000","last_undegraded":"2024-05-14T15:24:50.981579+0000","last_fullsized":"2024-05-14T15:24:50.981579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887352+0000","last_change":"2024-05-14T15:22:42.903272+0000","last_active":"2024-05-14T15:24:50.887352+0000","last_peered":"2024-05-14T15:24:50.887352+0000","last_clean":"2024-05-14T15:24:50.887352+0000","last_became_active":"2024-05-14T15:22:42.900088+0000","last_became_peered":"2024-05-14T15:22:42.900088+0000","last_unstale":"2024-05-14T15:24:50.887352+0000","last_undegraded":"2024-05-14T15:24:50.887352+0000","last_fullsized":"2024-05-14T15:24:50.887352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:23.975904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003825+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:24:51.003825+0000","last_peered":"2024-05-14T15:24:51.003825+0000","last_clean":"2024-05-14T15:24:51.003825+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:24:51.003825+0000","last_undegraded":"2024-05-14T15:24:51.003825+0000","last_fullsized":"2024-05-14T15:24:51.003825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725154+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:24:50.725154+0000","last_peered":"2024-05-14T15:24:50.725154+0000","last_clean":"2024-05-14T15:24:50.725154+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:24:50.725154+0000","last_undegraded":"2024-05-14T15:24:50.725154+0000","last_fullsized":"2024-05-14T15:24:50.725154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983138+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:24:50.983138+0000","last_peered":"2024-05-14T15:24:50.983138+0000","last_clean":"2024-05-14T15:24:50.983138+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:24:50.983138+0000","last_undegraded":"2024-05-14T15:24:50.983138+0000","last_fullsized":"2024-05-14T15:24:50.983138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724761+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:24:50.724761+0000","last_peered":"2024-05-14T15:24:50.724761+0000","last_clean":"2024-05-14T15:24:50.724761+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:24:50.724761+0000","last_undegraded":"2024-05-14T15:24:50.724761+0000","last_fullsized":"2024-05-14T15:24:50.724761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.855611+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:24:50.855611+0000","last_peered":"2024-05-14T15:24:50.855611+0000","last_clean":"2024-05-14T15:24:50.855611+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:24:50.855611+0000","last_undegraded":"2024-05-14T15:24:50.855611+0000","last_fullsized":"2024-05-14T15:24:50.855611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725675+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:24:50.725675+0000","last_peered":"2024-05-14T15:24:50.725675+0000","last_clean":"2024-05-14T15:24:50.725675+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:24:50.725675+0000","last_undegraded":"2024-05-14T15:24:50.725675+0000","last_fullsized":"2024-05-14T15:24:50.725675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619735+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:24:50.619735+0000","last_peered":"2024-05-14T15:24:50.619735+0000","last_clean":"2024-05-14T15:24:50.619735+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:24:50.619735+0000","last_undegraded":"2024-05-14T15:24:50.619735+0000","last_fullsized":"2024-05-14T15:24:50.619735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981860+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:24:50.981860+0000","last_peered":"2024-05-14T15:24:50.981860+0000","last_clean":"2024-05-14T15:24:50.981860+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:24:50.981860+0000","last_undegraded":"2024-05-14T15:24:50.981860+0000","last_fullsized":"2024-05-14T15:24:50.981860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982282+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:24:50.982282+0000","last_peered":"2024-05-14T15:24:50.982282+0000","last_clean":"2024-05-14T15:24:50.982282+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:24:50.982282+0000","last_undegraded":"2024-05-14T15:24:50.982282+0000","last_fullsized":"2024-05-14T15:24:50.982282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981334+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:24:50.981334+0000","last_peered":"2024-05-14T15:24:50.981334+0000","last_clean":"2024-05-14T15:24:50.981334+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:24:50.981334+0000","last_undegraded":"2024-05-14T15:24:50.981334+0000","last_fullsized":"2024-05-14T15:24:50.981334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617877+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:24:50.617877+0000","last_peered":"2024-05-14T15:24:50.617877+0000","last_clean":"2024-05-14T15:24:50.617877+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:24:50.617877+0000","last_undegraded":"2024-05-14T15:24:50.617877+0000","last_fullsized":"2024-05-14T15:24:50.617877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724725+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:24:50.724725+0000","last_peered":"2024-05-14T15:24:50.724725+0000","last_clean":"2024-05-14T15:24:50.724725+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:24:50.724725+0000","last_undegraded":"2024-05-14T15:24:50.724725+0000","last_fullsized":"2024-05-14T15:24:50.724725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616775+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:24:50.616775+0000","last_peered":"2024-05-14T15:24:50.616775+0000","last_clean":"2024-05-14T15:24:50.616775+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:24:50.616775+0000","last_undegraded":"2024-05-14T15:24:50.616775+0000","last_fullsized":"2024-05-14T15:24:50.616775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724872+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:24:50.724872+0000","last_peered":"2024-05-14T15:24:50.724872+0000","last_clean":"2024-05-14T15:24:50.724872+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:24:50.724872+0000","last_undegraded":"2024-05-14T15:24:50.724872+0000","last_fullsized":"2024-05-14T15:24:50.724872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003974+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:24:51.003974+0000","last_peered":"2024-05-14T15:24:51.003974+0000","last_clean":"2024-05-14T15:24:51.003974+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:24:51.003974+0000","last_undegraded":"2024-05-14T15:24:51.003974+0000","last_fullsized":"2024-05-14T15:24:51.003974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981584+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:24:50.981584+0000","last_peered":"2024-05-14T15:24:50.981584+0000","last_clean":"2024-05-14T15:24:50.981584+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:24:50.981584+0000","last_undegraded":"2024-05-14T15:24:50.981584+0000","last_fullsized":"2024-05-14T15:24:50.981584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619978+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:24:50.619978+0000","last_peered":"2024-05-14T15:24:50.619978+0000","last_clean":"2024-05-14T15:24:50.619978+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:24:50.619978+0000","last_undegraded":"2024-05-14T15:24:50.619978+0000","last_fullsized":"2024-05-14T15:24:50.619978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725647+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:24:50.725647+0000","last_peered":"2024-05-14T15:24:50.725647+0000","last_clean":"2024-05-14T15:24:50.725647+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:24:50.725647+0000","last_undegraded":"2024-05-14T15:24:50.725647+0000","last_fullsized":"2024-05-14T15:24:50.725647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981327+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:24:50.981327+0000","last_peered":"2024-05-14T15:24:50.981327+0000","last_clean":"2024-05-14T15:24:50.981327+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:24:50.981327+0000","last_undegraded":"2024-05-14T15:24:50.981327+0000","last_fullsized":"2024-05-14T15:24:50.981327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724801+0000","last_change":"2024-05-14T15:22:42.901814+0000","last_active":"2024-05-14T15:24:50.724801+0000","last_peered":"2024-05-14T15:24:50.724801+0000","last_clean":"2024-05-14T15:24:50.724801+0000","last_became_active":"2024-05-14T15:22:42.896795+0000","last_became_peered":"2024-05-14T15:22:42.896795+0000","last_unstale":"2024-05-14T15:24:50.724801+0000","last_undegraded":"2024-05-14T15:24:50.724801+0000","last_fullsized":"2024-05-14T15:24:50.724801+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:15.981532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887356+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:24:50.887356+0000","last_peered":"2024-05-14T15:24:50.887356+0000","last_clean":"2024-05-14T15:24:50.887356+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:24:50.887356+0000","last_undegraded":"2024-05-14T15:24:50.887356+0000","last_fullsized":"2024-05-14T15:24:50.887356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617801+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:24:50.617801+0000","last_peered":"2024-05-14T15:24:50.617801+0000","last_clean":"2024-05-14T15:24:50.617801+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:24:50.617801+0000","last_undegraded":"2024-05-14T15:24:50.617801+0000","last_fullsized":"2024-05-14T15:24:50.617801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621026+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:24:50.621026+0000","last_peered":"2024-05-14T15:24:50.621026+0000","last_clean":"2024-05-14T15:24:50.621026+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:24:50.621026+0000","last_undegraded":"2024-05-14T15:24:50.621026+0000","last_fullsized":"2024-05-14T15:24:50.621026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887834+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:24:50.887834+0000","last_peered":"2024-05-14T15:24:50.887834+0000","last_clean":"2024-05-14T15:24:50.887834+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:24:50.887834+0000","last_undegraded":"2024-05-14T15:24:50.887834+0000","last_fullsized":"2024-05-14T15:24:50.887834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981549+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:24:50.981549+0000","last_peered":"2024-05-14T15:24:50.981549+0000","last_clean":"2024-05-14T15:24:50.981549+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:24:50.981549+0000","last_undegraded":"2024-05-14T15:24:50.981549+0000","last_fullsized":"2024-05-14T15:24:50.981549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":93,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.850897+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:24:50.850897+0000","last_peered":"2024-05-14T15:24:50.850897+0000","last_clean":"2024-05-14T15:24:50.850897+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:24:50.850897+0000","last_undegraded":"2024-05-14T15:24:50.850897+0000","last_fullsized":"2024-05-14T15:24:50.850897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887586+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:24:50.887586+0000","last_peered":"2024-05-14T15:24:50.887586+0000","last_clean":"2024-05-14T15:24:50.887586+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:24:50.887586+0000","last_undegraded":"2024-05-14T15:24:50.887586+0000","last_fullsized":"2024-05-14T15:24:50.887586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736013+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:24:50.736013+0000","last_peered":"2024-05-14T15:24:50.736013+0000","last_clean":"2024-05-14T15:24:50.736013+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:24:50.736013+0000","last_undegraded":"2024-05-14T15:24:50.736013+0000","last_fullsized":"2024-05-14T15:24:50.736013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983098+0000","last_change":"2024-05-14T15:22:43.235879+0000","last_active":"2024-05-14T15:24:50.983098+0000","last_peered":"2024-05-14T15:24:50.983098+0000","last_clean":"2024-05-14T15:24:50.983098+0000","last_became_active":"2024-05-14T15:22:43.230116+0000","last_became_peered":"2024-05-14T15:22:43.230116+0000","last_unstale":"2024-05-14T15:24:50.983098+0000","last_undegraded":"2024-05-14T15:24:50.983098+0000","last_fullsized":"2024-05-14T15:24:50.983098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:25:50.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887809+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:24:50.887809+0000","last_peered":"2024-05-14T15:24:50.887809+0000","last_clean":"2024-05-14T15:24:50.887809+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:24:50.887809+0000","last_undegraded":"2024-05-14T15:24:50.887809+0000","last_fullsized":"2024-05-14T15:24:50.887809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004023+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:24:51.004023+0000","last_peered":"2024-05-14T15:24:51.004023+0000","last_clean":"2024-05-14T15:24:51.004023+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:24:51.004023+0000","last_undegraded":"2024-05-14T15:24:51.004023+0000","last_fullsized":"2024-05-14T15:24:51.004023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619978+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:24:50.619978+0000","last_peered":"2024-05-14T15:24:50.619978+0000","last_clean":"2024-05-14T15:24:50.619978+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:24:50.619978+0000","last_undegraded":"2024-05-14T15:24:50.619978+0000","last_fullsized":"2024-05-14T15:24:50.619978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619801+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:24:50.619801+0000","last_peered":"2024-05-14T15:24:50.619801+0000","last_clean":"2024-05-14T15:24:50.619801+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:24:50.619801+0000","last_undegraded":"2024-05-14T15:24:50.619801+0000","last_fullsized":"2024-05-14T15:24:50.619801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982632+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:24:50.982632+0000","last_peered":"2024-05-14T15:24:50.982632+0000","last_clean":"2024-05-14T15:24:50.982632+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:24:50.982632+0000","last_undegraded":"2024-05-14T15:24:50.982632+0000","last_fullsized":"2024-05-14T15:24:50.982632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983315+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:24:50.983315+0000","last_peered":"2024-05-14T15:24:50.983315+0000","last_clean":"2024-05-14T15:24:50.983315+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:24:50.983315+0000","last_undegraded":"2024-05-14T15:24:50.983315+0000","last_fullsized":"2024-05-14T15:24:50.983315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004200+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:24:51.004200+0000","last_peered":"2024-05-14T15:24:51.004200+0000","last_clean":"2024-05-14T15:24:51.004200+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:24:51.004200+0000","last_undegraded":"2024-05-14T15:24:51.004200+0000","last_fullsized":"2024-05-14T15:24:51.004200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621083+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:24:50.621083+0000","last_peered":"2024-05-14T15:24:50.621083+0000","last_clean":"2024-05-14T15:24:50.621083+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:24:50.621083+0000","last_undegraded":"2024-05-14T15:24:50.621083+0000","last_fullsized":"2024-05-14T15:24:50.621083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725796+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:24:50.725796+0000","last_peered":"2024-05-14T15:24:50.725796+0000","last_clean":"2024-05-14T15:24:50.725796+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:24:50.725796+0000","last_undegraded":"2024-05-14T15:24:50.725796+0000","last_fullsized":"2024-05-14T15:24:50.725796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736083+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:24:50.736083+0000","last_peered":"2024-05-14T15:24:50.736083+0000","last_clean":"2024-05-14T15:24:50.736083+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:24:50.736083+0000","last_undegraded":"2024-05-14T15:24:50.736083+0000","last_fullsized":"2024-05-14T15:24:50.736083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724963+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:24:50.724963+0000","last_peered":"2024-05-14T15:24:50.724963+0000","last_clean":"2024-05-14T15:24:50.724963+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:24:50.724963+0000","last_undegraded":"2024-05-14T15:24:50.724963+0000","last_fullsized":"2024-05-14T15:24:50.724963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617965+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:24:50.617965+0000","last_peered":"2024-05-14T15:24:50.617965+0000","last_clean":"2024-05-14T15:24:50.617965+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:24:50.617965+0000","last_undegraded":"2024-05-14T15:24:50.617965+0000","last_fullsized":"2024-05-14T15:24:50.617965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982527+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:24:50.982527+0000","last_peered":"2024-05-14T15:24:50.982527+0000","last_clean":"2024-05-14T15:24:50.982527+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:24:50.982527+0000","last_undegraded":"2024-05-14T15:24:50.982527+0000","last_fullsized":"2024-05-14T15:24:50.982527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004115+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:24:51.004115+0000","last_peered":"2024-05-14T15:24:51.004115+0000","last_clean":"2024-05-14T15:24:51.004115+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:24:51.004115+0000","last_undegraded":"2024-05-14T15:24:51.004115+0000","last_fullsized":"2024-05-14T15:24:51.004115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108306,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599109,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1169,"num_read_kb":697,"num_write":295,"num_write_kb":1443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1797327,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678079,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":528169,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874307,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1171456,"data_stored":1316406,"data_compressed":77588,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":925717,"data_compressed":54476,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234063,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":921541,"data_compressed":54476,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430291,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":1125157,"data_compressed":71432,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593815,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":513649,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":592078,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":524969,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":595524,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1048591,"data_compressed":60632,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953570,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":528231,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790044,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1252345,"data_compressed":77588,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134390,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202547,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:24:57.513 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-14T15:24:57.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-14T15:24:57.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.661+0000 7f622aca1640 1 -- 172.21.15.5:0/3910515036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6224109c60 msgr2=0x7f6224103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.661+0000 7f622aca1640 1 --2- 172.21.15.5:0/3910515036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6224109c60 0x7f6224103800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6218009900 tx=0x7f621802a410 comp rx=0 tx=0).stop 2024-05-14T15:24:57.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 -- 172.21.15.5:0/3910515036 shutdown_connections 2024-05-14T15:24:57.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 --2- 172.21.15.5:0/3910515036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f622410a100 0x7f6224103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 --2- 172.21.15.5:0/3910515036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62241043b0 0x7f6224110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 --2- 172.21.15.5:0/3910515036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6224109c60 0x7f6224103800 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 -- 172.21.15.5:0/3910515036 >> 172.21.15.5:0/3910515036 conn(0x7f62240ff340 msgr2=0x7f6224101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 -- 172.21.15.5:0/3910515036 shutdown_connections 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 -- 172.21.15.5:0/3910515036 wait complete. 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 Processor -- start 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.662+0000 7f622aca1640 1 -- start start 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 0x7f62241a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6224109c60 0x7f62241a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f622410a100 0x7f622419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113e50 con 0x7f62241043b0 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113cd0 con 0x7f6224109c60 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6224113fd0 con 0x7f622410a100 2024-05-14T15:24:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 0x7f62241a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 0x7f62241a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60688/0 (socket says 172.21.15.5:60688) 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 -- 172.21.15.5:0/2987626769 learned_addr learned my addr 172.21.15.5:0/2987626769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6223fff640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6224109c60 0x7f62241a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622923e640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f622410a100 0x7f622419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 -- 172.21.15.5:0/2987626769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f622410a100 msgr2=0x7f622419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f622410a100 0x7f622419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 -- 172.21.15.5:0/2987626769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6224109c60 msgr2=0x7f62241a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6224109c60 0x7f62241a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 -- 172.21.15.5:0/2987626769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622419ae00 con 0x7f62241043b0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6228a3d640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 0x7f62241a0550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6218032fd0 tx=0x7f6218031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621803f030 con 0x7f62241043b0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6218004490 con 0x7f62241043b0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622419b030 con 0x7f62241043b0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.663+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218035070 con 0x7f62241043b0 2024-05-14T15:24:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.664+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622419b520 con 0x7f62241043b0 2024-05-14T15:24:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.665+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224104f40 con 0x7f62241043b0 2024-05-14T15:24:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.666+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f6218042030 con 0x7f62241043b0 2024-05-14T15:24:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.667+0000 7f6221ffb640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f62140b2a80 0x7f62140b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.667+0000 7f6223fff640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f62140b2a80 0x7f62140b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:57.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.667+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6218103390 con 0x7f62241043b0 2024-05-14T15:24:57.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.667+0000 7f6223fff640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f62140b2a80 0x7f62140b4f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f622419be70 tx=0x7f6210005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:57.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.671+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62180ca180 con 0x7f62241043b0 2024-05-14T15:24:57.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.858+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62240ffe60 con 0x7f62140b2a80 2024-05-14T15:24:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.867+0000 7f6221ffb640 1 -- 172.21.15.5:0/2987626769 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339652 (secure 0 0 0) 0x7f62240ffe60 con 0x7f62140b2a80 2024-05-14T15:24:57.873 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:24:57.874 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f62140b2a80 msgr2=0x7f62140b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f62140b2a80 0x7f62140b4f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f622419be70 tx=0x7f6210005f90 comp rx=0 tx=0).stop 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 msgr2=0x7f62241a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 0x7f62241a0550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6218032fd0 tx=0x7f6218031a20 comp rx=0 tx=0).stop 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 shutdown_connections 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f62140b2a80 0x7f62140b4f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f622410a100 0x7f622419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6224109c60 0x7f62241a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 --2- 172.21.15.5:0/2987626769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62241043b0 0x7f62241a0550 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 >> 172.21.15.5:0/2987626769 conn(0x7f62240ff340 msgr2=0x7f62240ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 shutdown_connections 2024-05-14T15:24:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:57.871+0000 7f622aca1640 1 -- 172.21.15.5:0/2987626769 wait complete. 2024-05-14T15:24:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-14T15:24:56.633270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189711,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1250,"num_read_kb":766,"num_write":420,"num_write_kb":3595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378176,"kb_used_data":12344,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563648,"statfs":{"total":1151940427776,"available":1151553175552,"internally_reserved":0,"allocated":12640256,"data_stored":9872377,"data_compressed":605208,"data_compressed_allocated":819200,"data_compressed_original":4489216,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990686"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004427+0000","last_change":"2024-05-14T15:20:55.930291+0000","last_active":"2024-05-14T15:24:51.004427+0000","last_peered":"2024-05-14T15:24:51.004427+0000","last_clean":"2024-05-14T15:24:51.004427+0000","last_became_active":"2024-05-14T15:20:55.929019+0000","last_became_peered":"2024-05-14T15:20:55.929019+0000","last_unstale":"2024-05-14T15:24:51.004427+0000","last_undegraded":"2024-05-14T15:24:51.004427+0000","last_fullsized":"2024-05-14T15:24:51.004427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:43:58.790168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982134+0000","last_change":"2024-05-14T15:20:55.922432+0000","last_active":"2024-05-14T15:24:50.982134+0000","last_peered":"2024-05-14T15:24:50.982134+0000","last_clean":"2024-05-14T15:24:50.982134+0000","last_became_active":"2024-05-14T15:20:55.922073+0000","last_became_peered":"2024-05-14T15:20:55.922073+0000","last_unstale":"2024-05-14T15:24:50.982134+0000","last_undegraded":"2024-05-14T15:24:50.982134+0000","last_fullsized":"2024-05-14T15:24:50.982134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:38:02.456142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981937+0000","last_change":"2024-05-14T15:20:55.929479+0000","last_active":"2024-05-14T15:24:50.981937+0000","last_peered":"2024-05-14T15:24:50.981937+0000","last_clean":"2024-05-14T15:24:50.981937+0000","last_became_active":"2024-05-14T15:20:55.929121+0000","last_became_peered":"2024-05-14T15:20:55.929121+0000","last_unstale":"2024-05-14T15:24:50.981937+0000","last_undegraded":"2024-05-14T15:24:50.981937+0000","last_fullsized":"2024-05-14T15:24:50.981937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:13:05.819372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981876+0000","last_change":"2024-05-14T15:20:55.920632+0000","last_active":"2024-05-14T15:24:50.981876+0000","last_peered":"2024-05-14T15:24:50.981876+0000","last_clean":"2024-05-14T15:24:50.981876+0000","last_became_active":"2024-05-14T15:20:55.920221+0000","last_became_peered":"2024-05-14T15:20:55.920221+0000","last_unstale":"2024-05-14T15:24:50.981876+0000","last_undegraded":"2024-05-14T15:24:50.981876+0000","last_fullsized":"2024-05-14T15:24:50.981876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:03:08.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724901+0000","last_change":"2024-05-14T15:20:55.917688+0000","last_active":"2024-05-14T15:24:50.724901+0000","last_peered":"2024-05-14T15:24:50.724901+0000","last_clean":"2024-05-14T15:24:50.724901+0000","last_became_active":"2024-05-14T15:20:55.917357+0000","last_became_peered":"2024-05-14T15:20:55.917357+0000","last_unstale":"2024-05-14T15:24:50.724901+0000","last_undegraded":"2024-05-14T15:24:50.724901+0000","last_fullsized":"2024-05-14T15:24:50.724901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:23:15.608020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739533+0000","last_change":"2024-05-14T15:20:55.937629+0000","last_active":"2024-05-14T15:24:50.739533+0000","last_peered":"2024-05-14T15:24:50.739533+0000","last_clean":"2024-05-14T15:24:50.739533+0000","last_became_active":"2024-05-14T15:20:55.936725+0000","last_became_peered":"2024-05-14T15:20:55.936725+0000","last_unstale":"2024-05-14T15:24:50.739533+0000","last_undegraded":"2024-05-14T15:24:50.739533+0000","last_fullsized":"2024-05-14T15:24:50.739533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:05:28.186119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887231+0000","last_change":"2024-05-14T15:20:55.931942+0000","last_active":"2024-05-14T15:24:50.887231+0000","last_peered":"2024-05-14T15:24:50.887231+0000","last_clean":"2024-05-14T15:24:50.887231+0000","last_became_active":"2024-05-14T15:20:55.931607+0000","last_became_peered":"2024-05-14T15:20:55.931607+0000","last_unstale":"2024-05-14T15:24:50.887231+0000","last_undegraded":"2024-05-14T15:24:50.887231+0000","last_fullsized":"2024-05-14T15:24:50.887231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:13:30.413794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725049+0000","last_change":"2024-05-14T15:20:55.924528+0000","last_active":"2024-05-14T15:24:50.725049+0000","last_peered":"2024-05-14T15:24:50.725049+0000","last_clean":"2024-05-14T15:24:50.725049+0000","last_became_active":"2024-05-14T15:20:55.924160+0000","last_became_peered":"2024-05-14T15:20:55.924160+0000","last_unstale":"2024-05-14T15:24:50.725049+0000","last_undegraded":"2024-05-14T15:24:50.725049+0000","last_fullsized":"2024-05-14T15:24:50.725049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:54.692657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618484+0000","last_change":"2024-05-14T15:20:55.929591+0000","last_active":"2024-05-14T15:24:50.618484+0000","last_peered":"2024-05-14T15:24:50.618484+0000","last_clean":"2024-05-14T15:24:50.618484+0000","last_became_active":"2024-05-14T15:20:55.929248+0000","last_became_peered":"2024-05-14T15:20:55.929248+0000","last_unstale":"2024-05-14T15:24:50.618484+0000","last_undegraded":"2024-05-14T15:24:50.618484+0000","last_fullsized":"2024-05-14T15:24:50.618484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:31.851054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739931+0000","last_change":"2024-05-14T15:21:42.622265+0000","last_active":"2024-05-14T15:24:50.739931+0000","last_peered":"2024-05-14T15:24:50.739931+0000","last_clean":"2024-05-14T15:24:50.739931+0000","last_became_active":"2024-05-14T15:21:42.621876+0000","last_became_peered":"2024-05-14T15:21:42.621876+0000","last_unstale":"2024-05-14T15:24:50.739931+0000","last_undegraded":"2024-05-14T15:24:50.739931+0000","last_fullsized":"2024-05-14T15:24:50.739931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:59.656204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887724+0000","last_change":"2024-05-14T15:20:55.934095+0000","last_active":"2024-05-14T15:24:50.887724+0000","last_peered":"2024-05-14T15:24:50.887724+0000","last_clean":"2024-05-14T15:24:50.887724+0000","last_became_active":"2024-05-14T15:20:55.933742+0000","last_became_peered":"2024-05-14T15:20:55.933742+0000","last_unstale":"2024-05-14T15:24:50.887724+0000","last_undegraded":"2024-05-14T15:24:50.887724+0000","last_fullsized":"2024-05-14T15:24:50.887724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:02:41.305640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981832+0000","last_change":"2024-05-14T15:20:55.930730+0000","last_active":"2024-05-14T15:24:50.981832+0000","last_peered":"2024-05-14T15:24:50.981832+0000","last_clean":"2024-05-14T15:24:50.981832+0000","last_became_active":"2024-05-14T15:20:55.930300+0000","last_became_peered":"2024-05-14T15:20:55.930300+0000","last_unstale":"2024-05-14T15:24:50.981832+0000","last_undegraded":"2024-05-14T15:24:50.981832+0000","last_fullsized":"2024-05-14T15:24:50.981832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:10:34.736162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982434+0000","last_change":"2024-05-14T15:20:55.926570+0000","last_active":"2024-05-14T15:24:50.982434+0000","last_peered":"2024-05-14T15:24:50.982434+0000","last_clean":"2024-05-14T15:24:50.982434+0000","last_became_active":"2024-05-14T15:20:55.926084+0000","last_became_peered":"2024-05-14T15:20:55.926084+0000","last_unstale":"2024-05-14T15:24:50.982434+0000","last_undegraded":"2024-05-14T15:24:50.982434+0000","last_fullsized":"2024-05-14T15:24:50.982434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:24:16.766994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981531+0000","last_change":"2024-05-14T15:20:55.923189+0000","last_active":"2024-05-14T15:24:50.981531+0000","last_peered":"2024-05-14T15:24:50.981531+0000","last_clean":"2024-05-14T15:24:50.981531+0000","last_became_active":"2024-05-14T15:20:55.922860+0000","last_became_peered":"2024-05-14T15:20:55.922860+0000","last_unstale":"2024-05-14T15:24:50.981531+0000","last_undegraded":"2024-05-14T15:24:50.981531+0000","last_fullsized":"2024-05-14T15:24:50.981531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:43:05.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981623+0000","last_change":"2024-05-14T15:20:55.920657+0000","last_active":"2024-05-14T15:24:50.981623+0000","last_peered":"2024-05-14T15:24:50.981623+0000","last_clean":"2024-05-14T15:24:50.981623+0000","last_became_active":"2024-05-14T15:20:55.920311+0000","last_became_peered":"2024-05-14T15:20:55.920311+0000","last_unstale":"2024-05-14T15:24:50.981623+0000","last_undegraded":"2024-05-14T15:24:50.981623+0000","last_fullsized":"2024-05-14T15:24:50.981623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:30.501562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617800+0000","last_change":"2024-05-14T15:20:55.935716+0000","last_active":"2024-05-14T15:24:50.617800+0000","last_peered":"2024-05-14T15:24:50.617800+0000","last_clean":"2024-05-14T15:24:50.617800+0000","last_became_active":"2024-05-14T15:20:55.935102+0000","last_became_peered":"2024-05-14T15:20:55.935102+0000","last_unstale":"2024-05-14T15:24:50.617800+0000","last_undegraded":"2024-05-14T15:24:50.617800+0000","last_fullsized":"2024-05-14T15:24:50.617800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:10:13.644074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004331+0000","last_change":"2024-05-14T15:20:55.930654+0000","last_active":"2024-05-14T15:24:51.004331+0000","last_peered":"2024-05-14T15:24:51.004331+0000","last_clean":"2024-05-14T15:24:51.004331+0000","last_became_active":"2024-05-14T15:20:55.929821+0000","last_became_peered":"2024-05-14T15:20:55.929821+0000","last_unstale":"2024-05-14T15:24:51.004331+0000","last_undegraded":"2024-05-14T15:24:51.004331+0000","last_fullsized":"2024-05-14T15:24:51.004331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:15:01.641734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616451+0000","last_change":"2024-05-14T15:20:55.923683+0000","last_active":"2024-05-14T15:24:50.616451+0000","last_peered":"2024-05-14T15:24:50.616451+0000","last_clean":"2024-05-14T15:24:50.616451+0000","last_became_active":"2024-05-14T15:20:55.923332+0000","last_became_peered":"2024-05-14T15:20:55.923332+0000","last_unstale":"2024-05-14T15:24:50.616451+0000","last_undegraded":"2024-05-14T15:24:50.616451+0000","last_fullsized":"2024-05-14T15:24:50.616451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:49:28.206414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621311+0000","last_change":"2024-05-14T15:20:55.919944+0000","last_active":"2024-05-14T15:24:50.621311+0000","last_peered":"2024-05-14T15:24:50.621311+0000","last_clean":"2024-05-14T15:24:50.621311+0000","last_became_active":"2024-05-14T15:20:55.919536+0000","last_became_peered":"2024-05-14T15:20:55.919536+0000","last_unstale":"2024-05-14T15:24:50.621311+0000","last_undegraded":"2024-05-14T15:24:50.621311+0000","last_fullsized":"2024-05-14T15:24:50.621311+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:55.616178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619740+0000","last_change":"2024-05-14T15:21:06.535997+0000","last_active":"2024-05-14T15:24:50.619740+0000","last_peered":"2024-05-14T15:24:50.619740+0000","last_clean":"2024-05-14T15:24:50.619740+0000","last_became_active":"2024-05-14T15:21:06.535526+0000","last_became_peered":"2024-05-14T15:21:06.535526+0000","last_unstale":"2024-05-14T15:24:50.619740+0000","last_undegraded":"2024-05-14T15:24:50.619740+0000","last_fullsized":"2024-05-14T15:24:50.619740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:50.007826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616807+0000","last_change":"2024-05-14T15:20:52.895584+0000","last_active":"2024-05-14T15:24:50.616807+0000","last_peered":"2024-05-14T15:24:50.616807+0000","last_clean":"2024-05-14T15:24:50.616807+0000","last_became_active":"2024-05-14T15:20:52.895213+0000","last_became_peered":"2024-05-14T15:20:52.895213+0000","last_unstale":"2024-05-14T15:24:50.616807+0000","last_undegraded":"2024-05-14T15:24:50.616807+0000","last_fullsized":"2024-05-14T15:24:50.616807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:41:42.511944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725463+0000","last_change":"2024-05-14T15:20:55.928587+0000","last_active":"2024-05-14T15:24:50.725463+0000","last_peered":"2024-05-14T15:24:50.725463+0000","last_clean":"2024-05-14T15:24:50.725463+0000","last_became_active":"2024-05-14T15:20:55.928209+0000","last_became_peered":"2024-05-14T15:20:55.928209+0000","last_unstale":"2024-05-14T15:24:50.725463+0000","last_undegraded":"2024-05-14T15:24:50.725463+0000","last_fullsized":"2024-05-14T15:24:50.725463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:34.842001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981463+0000","last_change":"2024-05-14T15:21:42.717433+0000","last_active":"2024-05-14T15:24:50.981463+0000","last_peered":"2024-05-14T15:24:50.981463+0000","last_clean":"2024-05-14T15:24:50.981463+0000","last_became_active":"2024-05-14T15:21:42.716994+0000","last_became_peered":"2024-05-14T15:21:42.716994+0000","last_unstale":"2024-05-14T15:24:50.981463+0000","last_undegraded":"2024-05-14T15:24:50.981463+0000","last_fullsized":"2024-05-14T15:24:50.981463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:47:40.017526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725622+0000","last_change":"2024-05-14T15:20:52.958151+0000","last_active":"2024-05-14T15:24:50.725622+0000","last_peered":"2024-05-14T15:24:50.725622+0000","last_clean":"2024-05-14T15:24:50.725622+0000","last_became_active":"2024-05-14T15:20:52.957400+0000","last_became_peered":"2024-05-14T15:20:52.957400+0000","last_unstale":"2024-05-14T15:24:50.725622+0000","last_undegraded":"2024-05-14T15:24:50.725622+0000","last_fullsized":"2024-05-14T15:24:50.725622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:35.523924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982136+0000","last_change":"2024-05-14T15:20:55.929248+0000","last_active":"2024-05-14T15:24:50.982136+0000","last_peered":"2024-05-14T15:24:50.982136+0000","last_clean":"2024-05-14T15:24:50.982136+0000","last_became_active":"2024-05-14T15:20:55.928888+0000","last_became_peered":"2024-05-14T15:20:55.928888+0000","last_unstale":"2024-05-14T15:24:50.982136+0000","last_undegraded":"2024-05-14T15:24:50.982136+0000","last_fullsized":"2024-05-14T15:24:50.982136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:22:02.688570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981579+0000","last_change":"2024-05-14T15:21:42.667773+0000","last_active":"2024-05-14T15:24:50.981579+0000","last_peered":"2024-05-14T15:24:50.981579+0000","last_clean":"2024-05-14T15:24:50.981579+0000","last_became_active":"2024-05-14T15:21:42.667253+0000","last_became_peered":"2024-05-14T15:21:42.667253+0000","last_unstale":"2024-05-14T15:24:50.981579+0000","last_undegraded":"2024-05-14T15:24:50.981579+0000","last_fullsized":"2024-05-14T15:24:50.981579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:24:54.624266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725311+0000","last_change":"2024-05-14T15:20:52.898779+0000","last_active":"2024-05-14T15:24:50.725311+0000","last_peered":"2024-05-14T15:24:50.725311+0000","last_clean":"2024-05-14T15:24:50.725311+0000","last_became_active":"2024-05-14T15:20:52.898422+0000","last_became_peered":"2024-05-14T15:20:52.898422+0000","last_unstale":"2024-05-14T15:24:50.725311+0000","last_undegraded":"2024-05-14T15:24:50.725311+0000","last_fullsized":"2024-05-14T15:24:50.725311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:51:39.644722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618077+0000","last_change":"2024-05-14T15:20:55.928441+0000","last_active":"2024-05-14T15:24:50.618077+0000","last_peered":"2024-05-14T15:24:50.618077+0000","last_clean":"2024-05-14T15:24:50.618077+0000","last_became_active":"2024-05-14T15:20:55.928109+0000","last_became_peered":"2024-05-14T15:20:55.928109+0000","last_unstale":"2024-05-14T15:24:50.618077+0000","last_undegraded":"2024-05-14T15:24:50.618077+0000","last_fullsized":"2024-05-14T15:24:50.618077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:58:45.233436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981599+0000","last_change":"2024-05-14T15:21:06.536820+0000","last_active":"2024-05-14T15:24:50.981599+0000","last_peered":"2024-05-14T15:24:50.981599+0000","last_clean":"2024-05-14T15:24:50.981599+0000","last_became_active":"2024-05-14T15:21:06.536275+0000","last_became_peered":"2024-05-14T15:21:06.536275+0000","last_unstale":"2024-05-14T15:24:50.981599+0000","last_undegraded":"2024-05-14T15:24:50.981599+0000","last_fullsized":"2024-05-14T15:24:50.981599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:04:47.918365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725116+0000","last_change":"2024-05-14T15:20:52.900222+0000","last_active":"2024-05-14T15:24:50.725116+0000","last_peered":"2024-05-14T15:24:50.725116+0000","last_clean":"2024-05-14T15:24:50.725116+0000","last_became_active":"2024-05-14T15:20:52.899841+0000","last_became_peered":"2024-05-14T15:20:52.899841+0000","last_unstale":"2024-05-14T15:24:50.725116+0000","last_undegraded":"2024-05-14T15:24:50.725116+0000","last_fullsized":"2024-05-14T15:24:50.725116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:18:58.433405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736594+0000","last_change":"2024-05-14T15:20:55.936424+0000","last_active":"2024-05-14T15:24:50.736594+0000","last_peered":"2024-05-14T15:24:50.736594+0000","last_clean":"2024-05-14T15:24:50.736594+0000","last_became_active":"2024-05-14T15:20:55.936092+0000","last_became_peered":"2024-05-14T15:20:55.936092+0000","last_unstale":"2024-05-14T15:24:50.736594+0000","last_undegraded":"2024-05-14T15:24:50.736594+0000","last_fullsized":"2024-05-14T15:24:50.736594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:29:22.011570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621169+0000","last_change":"2024-05-14T15:21:06.538695+0000","last_active":"2024-05-14T15:24:50.621169+0000","last_peered":"2024-05-14T15:24:50.621169+0000","last_clean":"2024-05-14T15:24:50.621169+0000","last_became_active":"2024-05-14T15:21:06.538199+0000","last_became_peered":"2024-05-14T15:21:06.538199+0000","last_unstale":"2024-05-14T15:24:50.621169+0000","last_undegraded":"2024-05-14T15:24:50.621169+0000","last_fullsized":"2024-05-14T15:24:50.621169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:14.448494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.036543+0000","last_change":"2024-05-14T15:20:52.899179+0000","last_active":"2024-05-14T15:24:51.036543+0000","last_peered":"2024-05-14T15:24:51.036543+0000","last_clean":"2024-05-14T15:24:51.036543+0000","last_became_active":"2024-05-14T15:20:52.898408+0000","last_became_peered":"2024-05-14T15:20:52.898408+0000","last_unstale":"2024-05-14T15:24:51.036543+0000","last_undegraded":"2024-05-14T15:24:51.036543+0000","last_fullsized":"2024-05-14T15:24:51.036543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:53:37.581136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724843+0000","last_change":"2024-05-14T15:20:55.924597+0000","last_active":"2024-05-14T15:24:50.724843+0000","last_peered":"2024-05-14T15:24:50.724843+0000","last_clean":"2024-05-14T15:24:50.724843+0000","last_became_active":"2024-05-14T15:20:55.924257+0000","last_became_peered":"2024-05-14T15:20:55.924257+0000","last_unstale":"2024-05-14T15:24:50.724843+0000","last_undegraded":"2024-05-14T15:24:50.724843+0000","last_fullsized":"2024-05-14T15:24:50.724843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:15:10.522417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.620804+0000","last_change":"2024-05-14T15:21:06.537506+0000","last_active":"2024-05-14T15:24:50.620804+0000","last_peered":"2024-05-14T15:24:50.620804+0000","last_clean":"2024-05-14T15:24:50.620804+0000","last_became_active":"2024-05-14T15:21:06.537030+0000","last_became_peered":"2024-05-14T15:21:06.537030+0000","last_unstale":"2024-05-14T15:24:50.620804+0000","last_undegraded":"2024-05-14T15:24:50.620804+0000","last_fullsized":"2024-05-14T15:24:50.620804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:24:39.369520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739485+0000","last_change":"2024-05-14T15:22:43.233044+0000","last_active":"2024-05-14T15:24:50.739485+0000","last_peered":"2024-05-14T15:24:50.739485+0000","last_clean":"2024-05-14T15:24:50.739485+0000","last_became_active":"2024-05-14T15:22:43.229808+0000","last_became_peered":"2024-05-14T15:22:43.229808+0000","last_unstale":"2024-05-14T15:24:50.739485+0000","last_undegraded":"2024-05-14T15:24:50.739485+0000","last_fullsized":"2024-05-14T15:24:50.739485+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:07.226460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736522+0000","last_change":"2024-05-14T15:20:55.926626+0000","last_active":"2024-05-14T15:24:50.736522+0000","last_peered":"2024-05-14T15:24:50.736522+0000","last_clean":"2024-05-14T15:24:50.736522+0000","last_became_active":"2024-05-14T15:20:55.925712+0000","last_became_peered":"2024-05-14T15:20:55.925712+0000","last_unstale":"2024-05-14T15:24:50.736522+0000","last_undegraded":"2024-05-14T15:24:50.736522+0000","last_fullsized":"2024-05-14T15:24:50.736522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:40.806194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003934+0000","last_change":"2024-05-14T15:21:42.622805+0000","last_active":"2024-05-14T15:24:51.003934+0000","last_peered":"2024-05-14T15:24:51.003934+0000","last_clean":"2024-05-14T15:24:51.003934+0000","last_became_active":"2024-05-14T15:21:42.622281+0000","last_became_peered":"2024-05-14T15:21:42.622281+0000","last_unstale":"2024-05-14T15:24:51.003934+0000","last_undegraded":"2024-05-14T15:24:51.003934+0000","last_fullsized":"2024-05-14T15:24:51.003934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:17.269627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981845+0000","last_change":"2024-05-14T15:20:52.897391+0000","last_active":"2024-05-14T15:24:50.981845+0000","last_peered":"2024-05-14T15:24:50.981845+0000","last_clean":"2024-05-14T15:24:50.981845+0000","last_became_active":"2024-05-14T15:20:52.897025+0000","last_became_peered":"2024-05-14T15:20:52.897025+0000","last_unstale":"2024-05-14T15:24:50.981845+0000","last_undegraded":"2024-05-14T15:24:50.981845+0000","last_fullsized":"2024-05-14T15:24:50.981845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:39:58.514503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724897+0000","last_change":"2024-05-14T15:20:55.917731+0000","last_active":"2024-05-14T15:24:50.724897+0000","last_peered":"2024-05-14T15:24:50.724897+0000","last_clean":"2024-05-14T15:24:50.724897+0000","last_became_active":"2024-05-14T15:20:55.917378+0000","last_became_peered":"2024-05-14T15:20:55.917378+0000","last_unstale":"2024-05-14T15:24:50.724897+0000","last_undegraded":"2024-05-14T15:24:50.724897+0000","last_fullsized":"2024-05-14T15:24:50.724897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:00:41.277831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982168+0000","last_change":"2024-05-14T15:21:06.538075+0000","last_active":"2024-05-14T15:24:50.982168+0000","last_peered":"2024-05-14T15:24:50.982168+0000","last_clean":"2024-05-14T15:24:50.982168+0000","last_became_active":"2024-05-14T15:21:06.537651+0000","last_became_peered":"2024-05-14T15:21:06.537651+0000","last_unstale":"2024-05-14T15:24:50.982168+0000","last_undegraded":"2024-05-14T15:24:50.982168+0000","last_fullsized":"2024-05-14T15:24:50.982168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:50.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003726+0000","last_change":"2024-05-14T15:22:42.900544+0000","last_active":"2024-05-14T15:24:51.003726+0000","last_peered":"2024-05-14T15:24:51.003726+0000","last_clean":"2024-05-14T15:24:51.003726+0000","last_became_active":"2024-05-14T15:22:42.900015+0000","last_became_peered":"2024-05-14T15:22:42.900015+0000","last_unstale":"2024-05-14T15:24:51.003726+0000","last_undegraded":"2024-05-14T15:24:51.003726+0000","last_fullsized":"2024-05-14T15:24:51.003726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:15.904051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616117+0000","last_change":"2024-05-14T15:20:55.923701+0000","last_active":"2024-05-14T15:24:50.616117+0000","last_peered":"2024-05-14T15:24:50.616117+0000","last_clean":"2024-05-14T15:24:50.616117+0000","last_became_active":"2024-05-14T15:20:55.923371+0000","last_became_peered":"2024-05-14T15:20:55.923371+0000","last_unstale":"2024-05-14T15:24:50.616117+0000","last_undegraded":"2024-05-14T15:24:50.616117+0000","last_fullsized":"2024-05-14T15:24:50.616117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:18.620410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724836+0000","last_change":"2024-05-14T15:21:06.537692+0000","last_active":"2024-05-14T15:24:50.724836+0000","last_peered":"2024-05-14T15:24:50.724836+0000","last_clean":"2024-05-14T15:24:50.724836+0000","last_became_active":"2024-05-14T15:21:06.537265+0000","last_became_peered":"2024-05-14T15:21:06.537265+0000","last_unstale":"2024-05-14T15:24:50.724836+0000","last_undegraded":"2024-05-14T15:24:50.724836+0000","last_fullsized":"2024-05-14T15:24:50.724836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:45.738633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.886998+0000","last_change":"2024-05-14T15:20:52.902593+0000","last_active":"2024-05-14T15:24:50.886998+0000","last_peered":"2024-05-14T15:24:50.886998+0000","last_clean":"2024-05-14T15:24:50.886998+0000","last_became_active":"2024-05-14T15:20:52.902214+0000","last_became_peered":"2024-05-14T15:20:52.902214+0000","last_unstale":"2024-05-14T15:24:50.886998+0000","last_undegraded":"2024-05-14T15:24:50.886998+0000","last_fullsized":"2024-05-14T15:24:50.886998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:29:32.669147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618501+0000","last_change":"2024-05-14T15:20:55.924945+0000","last_active":"2024-05-14T15:24:50.618501+0000","last_peered":"2024-05-14T15:24:50.618501+0000","last_clean":"2024-05-14T15:24:50.618501+0000","last_became_active":"2024-05-14T15:20:55.924584+0000","last_became_peered":"2024-05-14T15:20:55.924584+0000","last_unstale":"2024-05-14T15:24:50.618501+0000","last_undegraded":"2024-05-14T15:24:50.618501+0000","last_fullsized":"2024-05-14T15:24:50.618501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:39:36.896162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.615998+0000","last_change":"2024-05-14T15:21:06.531787+0000","last_active":"2024-05-14T15:24:50.615998+0000","last_peered":"2024-05-14T15:24:50.615998+0000","last_clean":"2024-05-14T15:24:50.615998+0000","last_became_active":"2024-05-14T15:21:06.531340+0000","last_became_peered":"2024-05-14T15:21:06.531340+0000","last_unstale":"2024-05-14T15:24:50.615998+0000","last_undegraded":"2024-05-14T15:24:50.615998+0000","last_fullsized":"2024-05-14T15:24:50.615998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:22:59.656740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887655+0000","last_change":"2024-05-14T15:20:52.897356+0000","last_active":"2024-05-14T15:24:50.887655+0000","last_peered":"2024-05-14T15:24:50.887655+0000","last_clean":"2024-05-14T15:24:50.887655+0000","last_became_active":"2024-05-14T15:20:52.896965+0000","last_became_peered":"2024-05-14T15:20:52.896965+0000","last_unstale":"2024-05-14T15:24:50.887655+0000","last_undegraded":"2024-05-14T15:24:50.887655+0000","last_fullsized":"2024-05-14T15:24:50.887655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:02:59.410721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982157+0000","last_change":"2024-05-14T15:21:42.667723+0000","last_active":"2024-05-14T15:24:50.982157+0000","last_peered":"2024-05-14T15:24:50.982157+0000","last_clean":"2024-05-14T15:24:50.982157+0000","last_became_active":"2024-05-14T15:21:42.667295+0000","last_became_peered":"2024-05-14T15:21:42.667295+0000","last_unstale":"2024-05-14T15:24:50.982157+0000","last_undegraded":"2024-05-14T15:24:50.982157+0000","last_fullsized":"2024-05-14T15:24:50.982157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:07:00.679984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981722+0000","last_change":"2024-05-14T15:21:06.535309+0000","last_active":"2024-05-14T15:24:50.981722+0000","last_peered":"2024-05-14T15:24:50.981722+0000","last_clean":"2024-05-14T15:24:50.981722+0000","last_became_active":"2024-05-14T15:21:06.534791+0000","last_became_peered":"2024-05-14T15:21:06.534791+0000","last_unstale":"2024-05-14T15:24:50.981722+0000","last_undegraded":"2024-05-14T15:24:50.981722+0000","last_fullsized":"2024-05-14T15:24:50.981722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:43:31.378421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736561+0000","last_change":"2024-05-14T15:20:52.957331+0000","last_active":"2024-05-14T15:24:50.736561+0000","last_peered":"2024-05-14T15:24:50.736561+0000","last_clean":"2024-05-14T15:24:50.736561+0000","last_became_active":"2024-05-14T15:20:52.956932+0000","last_became_peered":"2024-05-14T15:20:52.956932+0000","last_unstale":"2024-05-14T15:24:50.736561+0000","last_undegraded":"2024-05-14T15:24:50.736561+0000","last_fullsized":"2024-05-14T15:24:50.736561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:53.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.618076+0000","last_change":"2024-05-14T15:20:55.937929+0000","last_active":"2024-05-14T15:24:50.618076+0000","last_peered":"2024-05-14T15:24:50.618076+0000","last_clean":"2024-05-14T15:24:50.618076+0000","last_became_active":"2024-05-14T15:20:55.936265+0000","last_became_peered":"2024-05-14T15:20:55.936265+0000","last_unstale":"2024-05-14T15:24:50.618076+0000","last_undegraded":"2024-05-14T15:24:50.618076+0000","last_fullsized":"2024-05-14T15:24:50.618076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:29:06.215575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616173+0000","last_change":"2024-05-14T15:21:06.537338+0000","last_active":"2024-05-14T15:24:50.616173+0000","last_peered":"2024-05-14T15:24:50.616173+0000","last_clean":"2024-05-14T15:24:50.616173+0000","last_became_active":"2024-05-14T15:21:06.536916+0000","last_became_peered":"2024-05-14T15:21:06.536916+0000","last_unstale":"2024-05-14T15:24:50.616173+0000","last_undegraded":"2024-05-14T15:24:50.616173+0000","last_fullsized":"2024-05-14T15:24:50.616173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:37:12.112866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":447,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.036303+0000","last_change":"2024-05-14T15:20:52.901029+0000","last_active":"2024-05-14T15:24:51.036303+0000","last_peered":"2024-05-14T15:24:51.036303+0000","last_clean":"2024-05-14T15:24:51.036303+0000","last_became_active":"2024-05-14T15:20:52.900247+0000","last_became_peered":"2024-05-14T15:20:52.900247+0000","last_unstale":"2024-05-14T15:24:51.036303+0000","last_undegraded":"2024-05-14T15:24:51.036303+0000","last_fullsized":"2024-05-14T15:24:51.036303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:11:30.064208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69369,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":249,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725097+0000","last_change":"2024-05-14T15:20:55.935151+0000","last_active":"2024-05-14T15:24:50.725097+0000","last_peered":"2024-05-14T15:24:50.725097+0000","last_clean":"2024-05-14T15:24:50.725097+0000","last_became_active":"2024-05-14T15:20:55.934794+0000","last_became_peered":"2024-05-14T15:20:55.934794+0000","last_unstale":"2024-05-14T15:24:50.725097+0000","last_undegraded":"2024-05-14T15:24:50.725097+0000","last_fullsized":"2024-05-14T15:24:50.725097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:40:18.203914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736912+0000","last_change":"2024-05-14T15:21:06.544057+0000","last_active":"2024-05-14T15:24:50.736912+0000","last_peered":"2024-05-14T15:24:50.736912+0000","last_clean":"2024-05-14T15:24:50.736912+0000","last_became_active":"2024-05-14T15:21:06.543567+0000","last_became_peered":"2024-05-14T15:21:06.543567+0000","last_unstale":"2024-05-14T15:24:50.736912+0000","last_undegraded":"2024-05-14T15:24:50.736912+0000","last_fullsized":"2024-05-14T15:24:50.736912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:59:18.308738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.620086+0000","last_change":"2024-05-14T15:20:52.900213+0000","last_active":"2024-05-14T15:24:50.620086+0000","last_peered":"2024-05-14T15:24:50.620086+0000","last_clean":"2024-05-14T15:24:50.620086+0000","last_became_active":"2024-05-14T15:20:52.899843+0000","last_became_peered":"2024-05-14T15:20:52.899843+0000","last_unstale":"2024-05-14T15:24:50.620086+0000","last_undegraded":"2024-05-14T15:24:50.620086+0000","last_fullsized":"2024-05-14T15:24:50.620086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:11:44.148912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982061+0000","last_change":"2024-05-14T15:20:55.933018+0000","last_active":"2024-05-14T15:24:50.982061+0000","last_peered":"2024-05-14T15:24:50.982061+0000","last_clean":"2024-05-14T15:24:50.982061+0000","last_became_active":"2024-05-14T15:20:55.932682+0000","last_became_peered":"2024-05-14T15:20:55.932682+0000","last_unstale":"2024-05-14T15:24:50.982061+0000","last_undegraded":"2024-05-14T15:24:50.982061+0000","last_fullsized":"2024-05-14T15:24:50.982061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:28:24.502449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003977+0000","last_change":"2024-05-14T15:21:06.536405+0000","last_active":"2024-05-14T15:24:51.003977+0000","last_peered":"2024-05-14T15:24:51.003977+0000","last_clean":"2024-05-14T15:24:51.003977+0000","last_became_active":"2024-05-14T15:21:06.535951+0000","last_became_peered":"2024-05-14T15:21:06.535951+0000","last_unstale":"2024-05-14T15:24:51.003977+0000","last_undegraded":"2024-05-14T15:24:51.003977+0000","last_fullsized":"2024-05-14T15:24:51.003977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:21:32.413845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736331+0000","last_change":"2024-05-14T15:20:52.900768+0000","last_active":"2024-05-14T15:24:50.736331+0000","last_peered":"2024-05-14T15:24:50.736331+0000","last_clean":"2024-05-14T15:24:50.736331+0000","last_became_active":"2024-05-14T15:20:52.900291+0000","last_became_peered":"2024-05-14T15:20:52.900291+0000","last_unstale":"2024-05-14T15:24:50.736331+0000","last_undegraded":"2024-05-14T15:24:50.736331+0000","last_fullsized":"2024-05-14T15:24:50.736331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:07:44.252833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887498+0000","last_change":"2024-05-14T15:20:55.930806+0000","last_active":"2024-05-14T15:24:50.887498+0000","last_peered":"2024-05-14T15:24:50.887498+0000","last_clean":"2024-05-14T15:24:50.887498+0000","last_became_active":"2024-05-14T15:20:55.928621+0000","last_became_peered":"2024-05-14T15:20:55.928621+0000","last_unstale":"2024-05-14T15:24:50.887498+0000","last_undegraded":"2024-05-14T15:24:50.887498+0000","last_fullsized":"2024-05-14T15:24:50.887498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:01:53.005642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616369+0000","last_change":"2024-05-14T15:21:06.539890+0000","last_active":"2024-05-14T15:24:50.616369+0000","last_peered":"2024-05-14T15:24:50.616369+0000","last_clean":"2024-05-14T15:24:50.616369+0000","last_became_active":"2024-05-14T15:21:06.538218+0000","last_became_peered":"2024-05-14T15:21:06.538218+0000","last_unstale":"2024-05-14T15:24:50.616369+0000","last_undegraded":"2024-05-14T15:24:50.616369+0000","last_fullsized":"2024-05-14T15:24:50.616369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:58:41.148333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982014+0000","last_change":"2024-05-14T15:20:52.903059+0000","last_active":"2024-05-14T15:24:50.982014+0000","last_peered":"2024-05-14T15:24:50.982014+0000","last_clean":"2024-05-14T15:24:50.982014+0000","last_became_active":"2024-05-14T15:20:52.902620+0000","last_became_peered":"2024-05-14T15:20:52.902620+0000","last_unstale":"2024-05-14T15:24:50.982014+0000","last_undegraded":"2024-05-14T15:24:50.982014+0000","last_fullsized":"2024-05-14T15:24:50.982014+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:15:32.648412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887383+0000","last_change":"2024-05-14T15:20:55.929029+0000","last_active":"2024-05-14T15:24:50.887383+0000","last_peered":"2024-05-14T15:24:50.887383+0000","last_clean":"2024-05-14T15:24:50.887383+0000","last_became_active":"2024-05-14T15:20:55.928691+0000","last_became_peered":"2024-05-14T15:20:55.928691+0000","last_unstale":"2024-05-14T15:24:50.887383+0000","last_undegraded":"2024-05-14T15:24:50.887383+0000","last_fullsized":"2024-05-14T15:24:50.887383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:05:21.455304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982928+0000","last_change":"2024-05-14T15:21:06.541193+0000","last_active":"2024-05-14T15:24:50.982928+0000","last_peered":"2024-05-14T15:24:50.982928+0000","last_clean":"2024-05-14T15:24:50.982928+0000","last_became_active":"2024-05-14T15:21:06.540788+0000","last_became_peered":"2024-05-14T15:21:06.540788+0000","last_unstale":"2024-05-14T15:24:50.982928+0000","last_undegraded":"2024-05-14T15:24:50.982928+0000","last_fullsized":"2024-05-14T15:24:50.982928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:35.912708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724711+0000","last_change":"2024-05-14T15:20:52.957398+0000","last_active":"2024-05-14T15:24:50.724711+0000","last_peered":"2024-05-14T15:24:50.724711+0000","last_clean":"2024-05-14T15:24:50.724711+0000","last_became_active":"2024-05-14T15:20:52.956928+0000","last_became_peered":"2024-05-14T15:20:52.956928+0000","last_unstale":"2024-05-14T15:24:50.724711+0000","last_undegraded":"2024-05-14T15:24:50.724711+0000","last_fullsized":"2024-05-14T15:24:50.724711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:37:42.307216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736231+0000","last_change":"2024-05-14T15:20:55.931475+0000","last_active":"2024-05-14T15:24:50.736231+0000","last_peered":"2024-05-14T15:24:50.736231+0000","last_clean":"2024-05-14T15:24:50.736231+0000","last_became_active":"2024-05-14T15:20:55.931169+0000","last_became_peered":"2024-05-14T15:20:55.931169+0000","last_unstale":"2024-05-14T15:24:50.736231+0000","last_undegraded":"2024-05-14T15:24:50.736231+0000","last_fullsized":"2024-05-14T15:24:50.736231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:49:50.720904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736232+0000","last_change":"2024-05-14T15:21:06.541269+0000","last_active":"2024-05-14T15:24:50.736232+0000","last_peered":"2024-05-14T15:24:50.736232+0000","last_clean":"2024-05-14T15:24:50.736232+0000","last_became_active":"2024-05-14T15:21:06.540638+0000","last_became_peered":"2024-05-14T15:21:06.540638+0000","last_unstale":"2024-05-14T15:24:50.736232+0000","last_undegraded":"2024-05-14T15:24:50.736232+0000","last_fullsized":"2024-05-14T15:24:50.736232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:06.926870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725227+0000","last_change":"2024-05-14T15:20:52.898986+0000","last_active":"2024-05-14T15:24:50.725227+0000","last_peered":"2024-05-14T15:24:50.725227+0000","last_clean":"2024-05-14T15:24:50.725227+0000","last_became_active":"2024-05-14T15:20:52.898586+0000","last_became_peered":"2024-05-14T15:20:52.898586+0000","last_unstale":"2024-05-14T15:24:50.725227+0000","last_undegraded":"2024-05-14T15:24:50.725227+0000","last_fullsized":"2024-05-14T15:24:50.725227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:50:30.066608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004205+0000","last_change":"2024-05-14T15:20:55.930981+0000","last_active":"2024-05-14T15:24:51.004205+0000","last_peered":"2024-05-14T15:24:51.004205+0000","last_clean":"2024-05-14T15:24:51.004205+0000","last_became_active":"2024-05-14T15:20:55.930650+0000","last_became_peered":"2024-05-14T15:20:55.930650+0000","last_unstale":"2024-05-14T15:24:51.004205+0000","last_undegraded":"2024-05-14T15:24:51.004205+0000","last_fullsized":"2024-05-14T15:24:51.004205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:45:08.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724662+0000","last_change":"2024-05-14T15:21:06.670757+0000","last_active":"2024-05-14T15:24:50.724662+0000","last_peered":"2024-05-14T15:24:50.724662+0000","last_clean":"2024-05-14T15:24:50.724662+0000","last_became_active":"2024-05-14T15:21:06.670237+0000","last_became_peered":"2024-05-14T15:21:06.670237+0000","last_unstale":"2024-05-14T15:24:50.724662+0000","last_undegraded":"2024-05-14T15:24:50.724662+0000","last_fullsized":"2024-05-14T15:24:50.724662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:17:56.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616333+0000","last_change":"2024-05-14T15:21:42.620211+0000","last_active":"2024-05-14T15:24:50.616333+0000","last_peered":"2024-05-14T15:24:50.616333+0000","last_clean":"2024-05-14T15:24:50.616333+0000","last_became_active":"2024-05-14T15:21:42.619681+0000","last_became_peered":"2024-05-14T15:21:42.619681+0000","last_unstale":"2024-05-14T15:24:50.616333+0000","last_undegraded":"2024-05-14T15:24:50.616333+0000","last_fullsized":"2024-05-14T15:24:50.616333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:11:26.479851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739775+0000","last_change":"2024-05-14T15:20:55.926661+0000","last_active":"2024-05-14T15:24:50.739775+0000","last_peered":"2024-05-14T15:24:50.739775+0000","last_clean":"2024-05-14T15:24:50.739775+0000","last_became_active":"2024-05-14T15:20:55.926328+0000","last_became_peered":"2024-05-14T15:20:55.926328+0000","last_unstale":"2024-05-14T15:24:50.739775+0000","last_undegraded":"2024-05-14T15:24:50.739775+0000","last_fullsized":"2024-05-14T15:24:50.739775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:06:38.425351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982918+0000","last_change":"2024-05-14T15:21:06.534984+0000","last_active":"2024-05-14T15:24:50.982918+0000","last_peered":"2024-05-14T15:24:50.982918+0000","last_clean":"2024-05-14T15:24:50.982918+0000","last_became_active":"2024-05-14T15:21:06.534489+0000","last_became_peered":"2024-05-14T15:21:06.534489+0000","last_unstale":"2024-05-14T15:24:50.982918+0000","last_undegraded":"2024-05-14T15:24:50.982918+0000","last_fullsized":"2024-05-14T15:24:50.982918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:52:31.290882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887350+0000","last_change":"2024-05-14T15:20:52.897944+0000","last_active":"2024-05-14T15:24:50.887350+0000","last_peered":"2024-05-14T15:24:50.887350+0000","last_clean":"2024-05-14T15:24:50.887350+0000","last_became_active":"2024-05-14T15:20:52.897579+0000","last_became_peered":"2024-05-14T15:20:52.897579+0000","last_unstale":"2024-05-14T15:24:50.887350+0000","last_undegraded":"2024-05-14T15:24:50.887350+0000","last_fullsized":"2024-05-14T15:24:50.887350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:16:15.137020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.739500+0000","last_change":"2024-05-14T15:20:55.937409+0000","last_active":"2024-05-14T15:24:50.739500+0000","last_peered":"2024-05-14T15:24:50.739500+0000","last_clean":"2024-05-14T15:24:50.739500+0000","last_became_active":"2024-05-14T15:20:55.937078+0000","last_became_peered":"2024-05-14T15:20:55.937078+0000","last_unstale":"2024-05-14T15:24:50.739500+0000","last_undegraded":"2024-05-14T15:24:50.739500+0000","last_fullsized":"2024-05-14T15:24:50.739500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:06:08.672818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983481+0000","last_change":"2024-05-14T15:21:06.671887+0000","last_active":"2024-05-14T15:24:50.983481+0000","last_peered":"2024-05-14T15:24:50.983481+0000","last_clean":"2024-05-14T15:24:50.983481+0000","last_became_active":"2024-05-14T15:21:06.671404+0000","last_became_peered":"2024-05-14T15:21:06.671404+0000","last_unstale":"2024-05-14T15:24:50.983481+0000","last_undegraded":"2024-05-14T15:24:50.983481+0000","last_fullsized":"2024-05-14T15:24:50.983481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:38.613067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736244+0000","last_change":"2024-05-14T15:20:55.926337+0000","last_active":"2024-05-14T15:24:50.736244+0000","last_peered":"2024-05-14T15:24:50.736244+0000","last_clean":"2024-05-14T15:24:50.736244+0000","last_became_active":"2024-05-14T15:20:55.925839+0000","last_became_peered":"2024-05-14T15:20:55.925839+0000","last_unstale":"2024-05-14T15:24:50.736244+0000","last_undegraded":"2024-05-14T15:24:50.736244+0000","last_fullsized":"2024-05-14T15:24:50.736244+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:10.065461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981800+0000","last_change":"2024-05-14T15:20:52.956834+0000","last_active":"2024-05-14T15:24:50.981800+0000","last_peered":"2024-05-14T15:24:50.981800+0000","last_clean":"2024-05-14T15:24:50.981800+0000","last_became_active":"2024-05-14T15:20:52.956411+0000","last_became_peered":"2024-05-14T15:20:52.956411+0000","last_unstale":"2024-05-14T15:24:50.981800+0000","last_undegraded":"2024-05-14T15:24:50.981800+0000","last_fullsized":"2024-05-14T15:24:50.981800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:19:47.112567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887629+0000","last_change":"2024-05-14T15:21:06.537336+0000","last_active":"2024-05-14T15:24:50.887629+0000","last_peered":"2024-05-14T15:24:50.887629+0000","last_clean":"2024-05-14T15:24:50.887629+0000","last_became_active":"2024-05-14T15:21:06.536913+0000","last_became_peered":"2024-05-14T15:21:06.536913+0000","last_unstale":"2024-05-14T15:24:50.887629+0000","last_undegraded":"2024-05-14T15:24:50.887629+0000","last_fullsized":"2024-05-14T15:24:50.887629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:23:59.079068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725131+0000","last_change":"2024-05-14T15:20:55.930176+0000","last_active":"2024-05-14T15:24:50.725131+0000","last_peered":"2024-05-14T15:24:50.725131+0000","last_clean":"2024-05-14T15:24:50.725131+0000","last_became_active":"2024-05-14T15:20:55.929108+0000","last_became_peered":"2024-05-14T15:20:55.929108+0000","last_unstale":"2024-05-14T15:24:50.725131+0000","last_undegraded":"2024-05-14T15:24:50.725131+0000","last_fullsized":"2024-05-14T15:24:50.725131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:17:11.550999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981587+0000","last_change":"2024-05-14T15:20:52.901440+0000","last_active":"2024-05-14T15:24:50.981587+0000","last_peered":"2024-05-14T15:24:50.981587+0000","last_clean":"2024-05-14T15:24:50.981587+0000","last_became_active":"2024-05-14T15:20:52.901008+0000","last_became_peered":"2024-05-14T15:20:52.901008+0000","last_unstale":"2024-05-14T15:24:50.981587+0000","last_undegraded":"2024-05-14T15:24:50.981587+0000","last_fullsized":"2024-05-14T15:24:50.981587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:28:34.570173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887139+0000","last_change":"2024-05-14T15:19:27.937626+0000","last_active":"2024-05-14T15:24:50.887139+0000","last_peered":"2024-05-14T15:24:50.887139+0000","last_clean":"2024-05-14T15:24:50.887139+0000","last_became_active":"2024-05-14T15:19:27.932504+0000","last_became_peered":"2024-05-14T15:19:27.932504+0000","last_unstale":"2024-05-14T15:24:50.887139+0000","last_undegraded":"2024-05-14T15:24:50.887139+0000","last_fullsized":"2024-05-14T15:24:50.887139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:16:06.398354+0000","last_clean_scrub_stamp":"2024-05-14T15:16:06.398354+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:27:38.159575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.735953+0000","last_change":"2024-05-14T15:21:06.670746+0000","last_active":"2024-05-14T15:24:50.735953+0000","last_peered":"2024-05-14T15:24:50.735953+0000","last_clean":"2024-05-14T15:24:50.735953+0000","last_became_active":"2024-05-14T15:21:06.670209+0000","last_became_peered":"2024-05-14T15:21:06.670209+0000","last_unstale":"2024-05-14T15:24:50.735953+0000","last_undegraded":"2024-05-14T15:24:50.735953+0000","last_fullsized":"2024-05-14T15:24:50.735953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:52:41.365657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981579+0000","last_change":"2024-05-14T15:20:55.925896+0000","last_active":"2024-05-14T15:24:50.981579+0000","last_peered":"2024-05-14T15:24:50.981579+0000","last_clean":"2024-05-14T15:24:50.981579+0000","last_became_active":"2024-05-14T15:20:55.925524+0000","last_became_peered":"2024-05-14T15:20:55.925524+0000","last_unstale":"2024-05-14T15:24:50.981579+0000","last_undegraded":"2024-05-14T15:24:50.981579+0000","last_fullsized":"2024-05-14T15:24:50.981579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:26:59.096602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887352+0000","last_change":"2024-05-14T15:22:42.903272+0000","last_active":"2024-05-14T15:24:50.887352+0000","last_peered":"2024-05-14T15:24:50.887352+0000","last_clean":"2024-05-14T15:24:50.887352+0000","last_became_active":"2024-05-14T15:22:42.900088+0000","last_became_peered":"2024-05-14T15:22:42.900088+0000","last_unstale":"2024-05-14T15:24:50.887352+0000","last_undegraded":"2024-05-14T15:24:50.887352+0000","last_fullsized":"2024-05-14T15:24:50.887352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:44:23.975904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003825+0000","last_change":"2024-05-14T15:21:06.539089+0000","last_active":"2024-05-14T15:24:51.003825+0000","last_peered":"2024-05-14T15:24:51.003825+0000","last_clean":"2024-05-14T15:24:51.003825+0000","last_became_active":"2024-05-14T15:21:06.538588+0000","last_became_peered":"2024-05-14T15:21:06.538588+0000","last_unstale":"2024-05-14T15:24:51.003825+0000","last_undegraded":"2024-05-14T15:24:51.003825+0000","last_fullsized":"2024-05-14T15:24:51.003825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:10.685075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725154+0000","last_change":"2024-05-14T15:20:52.957389+0000","last_active":"2024-05-14T15:24:50.725154+0000","last_peered":"2024-05-14T15:24:50.725154+0000","last_clean":"2024-05-14T15:24:50.725154+0000","last_became_active":"2024-05-14T15:20:52.956934+0000","last_became_peered":"2024-05-14T15:20:52.956934+0000","last_unstale":"2024-05-14T15:24:50.725154+0000","last_undegraded":"2024-05-14T15:24:50.725154+0000","last_fullsized":"2024-05-14T15:24:50.725154+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:56:27.691601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983138+0000","last_change":"2024-05-14T15:20:55.921785+0000","last_active":"2024-05-14T15:24:50.983138+0000","last_peered":"2024-05-14T15:24:50.983138+0000","last_clean":"2024-05-14T15:24:50.983138+0000","last_became_active":"2024-05-14T15:20:55.921392+0000","last_became_peered":"2024-05-14T15:20:55.921392+0000","last_unstale":"2024-05-14T15:24:50.983138+0000","last_undegraded":"2024-05-14T15:24:50.983138+0000","last_fullsized":"2024-05-14T15:24:50.983138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:00:11.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724761+0000","last_change":"2024-05-14T15:21:06.670880+0000","last_active":"2024-05-14T15:24:50.724761+0000","last_peered":"2024-05-14T15:24:50.724761+0000","last_clean":"2024-05-14T15:24:50.724761+0000","last_became_active":"2024-05-14T15:21:06.670378+0000","last_became_peered":"2024-05-14T15:21:06.670378+0000","last_unstale":"2024-05-14T15:24:50.724761+0000","last_undegraded":"2024-05-14T15:24:50.724761+0000","last_fullsized":"2024-05-14T15:24:50.724761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:13:25.517324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.855611+0000","last_change":"2024-05-14T15:20:52.895781+0000","last_active":"2024-05-14T15:24:50.855611+0000","last_peered":"2024-05-14T15:24:50.855611+0000","last_clean":"2024-05-14T15:24:50.855611+0000","last_became_active":"2024-05-14T15:20:52.895412+0000","last_became_peered":"2024-05-14T15:20:52.895412+0000","last_unstale":"2024-05-14T15:24:50.855611+0000","last_undegraded":"2024-05-14T15:24:50.855611+0000","last_fullsized":"2024-05-14T15:24:50.855611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:03:17.728524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725675+0000","last_change":"2024-05-14T15:20:55.935139+0000","last_active":"2024-05-14T15:24:50.725675+0000","last_peered":"2024-05-14T15:24:50.725675+0000","last_clean":"2024-05-14T15:24:50.725675+0000","last_became_active":"2024-05-14T15:20:55.934893+0000","last_became_peered":"2024-05-14T15:20:55.934893+0000","last_unstale":"2024-05-14T15:24:50.725675+0000","last_undegraded":"2024-05-14T15:24:50.725675+0000","last_fullsized":"2024-05-14T15:24:50.725675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:09:27.987163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619735+0000","last_change":"2024-05-14T15:21:06.533419+0000","last_active":"2024-05-14T15:24:50.619735+0000","last_peered":"2024-05-14T15:24:50.619735+0000","last_clean":"2024-05-14T15:24:50.619735+0000","last_became_active":"2024-05-14T15:21:06.532970+0000","last_became_peered":"2024-05-14T15:21:06.532970+0000","last_unstale":"2024-05-14T15:24:50.619735+0000","last_undegraded":"2024-05-14T15:24:50.619735+0000","last_fullsized":"2024-05-14T15:24:50.619735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:04:31.797482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981860+0000","last_change":"2024-05-14T15:20:52.998546+0000","last_active":"2024-05-14T15:24:50.981860+0000","last_peered":"2024-05-14T15:24:50.981860+0000","last_clean":"2024-05-14T15:24:50.981860+0000","last_became_active":"2024-05-14T15:20:52.998149+0000","last_became_peered":"2024-05-14T15:20:52.998149+0000","last_unstale":"2024-05-14T15:24:50.981860+0000","last_undegraded":"2024-05-14T15:24:50.981860+0000","last_fullsized":"2024-05-14T15:24:50.981860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:58:46.001967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982282+0000","last_change":"2024-05-14T15:21:42.718180+0000","last_active":"2024-05-14T15:24:50.982282+0000","last_peered":"2024-05-14T15:24:50.982282+0000","last_clean":"2024-05-14T15:24:50.982282+0000","last_became_active":"2024-05-14T15:21:42.717752+0000","last_became_peered":"2024-05-14T15:21:42.717752+0000","last_unstale":"2024-05-14T15:24:50.982282+0000","last_undegraded":"2024-05-14T15:24:50.982282+0000","last_fullsized":"2024-05-14T15:24:50.982282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:41:30.867001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981334+0000","last_change":"2024-05-14T15:21:06.670741+0000","last_active":"2024-05-14T15:24:50.981334+0000","last_peered":"2024-05-14T15:24:50.981334+0000","last_clean":"2024-05-14T15:24:50.981334+0000","last_became_active":"2024-05-14T15:21:06.670256+0000","last_became_peered":"2024-05-14T15:21:06.670256+0000","last_unstale":"2024-05-14T15:24:50.981334+0000","last_undegraded":"2024-05-14T15:24:50.981334+0000","last_fullsized":"2024-05-14T15:24:50.981334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:54:07.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617877+0000","last_change":"2024-05-14T15:20:52.957528+0000","last_active":"2024-05-14T15:24:50.617877+0000","last_peered":"2024-05-14T15:24:50.617877+0000","last_clean":"2024-05-14T15:24:50.617877+0000","last_became_active":"2024-05-14T15:20:52.957222+0000","last_became_peered":"2024-05-14T15:20:52.957222+0000","last_unstale":"2024-05-14T15:24:50.617877+0000","last_undegraded":"2024-05-14T15:24:50.617877+0000","last_fullsized":"2024-05-14T15:24:50.617877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:49:27.333179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724725+0000","last_change":"2024-05-14T15:20:55.922758+0000","last_active":"2024-05-14T15:24:50.724725+0000","last_peered":"2024-05-14T15:24:50.724725+0000","last_clean":"2024-05-14T15:24:50.724725+0000","last_became_active":"2024-05-14T15:20:55.922411+0000","last_became_peered":"2024-05-14T15:20:55.922411+0000","last_unstale":"2024-05-14T15:24:50.724725+0000","last_undegraded":"2024-05-14T15:24:50.724725+0000","last_fullsized":"2024-05-14T15:24:50.724725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:21:41.712668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.616775+0000","last_change":"2024-05-14T15:21:06.539805+0000","last_active":"2024-05-14T15:24:50.616775+0000","last_peered":"2024-05-14T15:24:50.616775+0000","last_clean":"2024-05-14T15:24:50.616775+0000","last_became_active":"2024-05-14T15:21:06.539185+0000","last_became_peered":"2024-05-14T15:21:06.539185+0000","last_unstale":"2024-05-14T15:24:50.616775+0000","last_undegraded":"2024-05-14T15:24:50.616775+0000","last_fullsized":"2024-05-14T15:24:50.616775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:46:57.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724872+0000","last_change":"2024-05-14T15:20:52.898526+0000","last_active":"2024-05-14T15:24:50.724872+0000","last_peered":"2024-05-14T15:24:50.724872+0000","last_clean":"2024-05-14T15:24:50.724872+0000","last_became_active":"2024-05-14T15:20:52.898118+0000","last_became_peered":"2024-05-14T15:20:52.898118+0000","last_unstale":"2024-05-14T15:24:50.724872+0000","last_undegraded":"2024-05-14T15:24:50.724872+0000","last_fullsized":"2024-05-14T15:24:50.724872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:29:07.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.003974+0000","last_change":"2024-05-14T15:20:55.920561+0000","last_active":"2024-05-14T15:24:51.003974+0000","last_peered":"2024-05-14T15:24:51.003974+0000","last_clean":"2024-05-14T15:24:51.003974+0000","last_became_active":"2024-05-14T15:20:55.919826+0000","last_became_peered":"2024-05-14T15:20:55.919826+0000","last_unstale":"2024-05-14T15:24:51.003974+0000","last_undegraded":"2024-05-14T15:24:51.003974+0000","last_fullsized":"2024-05-14T15:24:51.003974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:55:34.666406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981584+0000","last_change":"2024-05-14T15:21:06.531963+0000","last_active":"2024-05-14T15:24:50.981584+0000","last_peered":"2024-05-14T15:24:50.981584+0000","last_clean":"2024-05-14T15:24:50.981584+0000","last_became_active":"2024-05-14T15:21:06.531529+0000","last_became_peered":"2024-05-14T15:21:06.531529+0000","last_unstale":"2024-05-14T15:24:50.981584+0000","last_undegraded":"2024-05-14T15:24:50.981584+0000","last_fullsized":"2024-05-14T15:24:50.981584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T15:58:30.870002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619978+0000","last_change":"2024-05-14T15:20:52.896889+0000","last_active":"2024-05-14T15:24:50.619978+0000","last_peered":"2024-05-14T15:24:50.619978+0000","last_clean":"2024-05-14T15:24:50.619978+0000","last_became_active":"2024-05-14T15:20:52.895774+0000","last_became_peered":"2024-05-14T15:20:52.895774+0000","last_unstale":"2024-05-14T15:24:50.619978+0000","last_undegraded":"2024-05-14T15:24:50.619978+0000","last_fullsized":"2024-05-14T15:24:50.619978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:37:20.785188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725647+0000","last_change":"2024-05-14T15:20:55.925111+0000","last_active":"2024-05-14T15:24:50.725647+0000","last_peered":"2024-05-14T15:24:50.725647+0000","last_clean":"2024-05-14T15:24:50.725647+0000","last_became_active":"2024-05-14T15:20:55.924735+0000","last_became_peered":"2024-05-14T15:20:55.924735+0000","last_unstale":"2024-05-14T15:24:50.725647+0000","last_undegraded":"2024-05-14T15:24:50.725647+0000","last_fullsized":"2024-05-14T15:24:50.725647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:46:32.884658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981327+0000","last_change":"2024-05-14T15:21:06.670963+0000","last_active":"2024-05-14T15:24:50.981327+0000","last_peered":"2024-05-14T15:24:50.981327+0000","last_clean":"2024-05-14T15:24:50.981327+0000","last_became_active":"2024-05-14T15:21:06.670059+0000","last_became_peered":"2024-05-14T15:21:06.670059+0000","last_unstale":"2024-05-14T15:24:50.981327+0000","last_undegraded":"2024-05-14T15:24:50.981327+0000","last_fullsized":"2024-05-14T15:24:50.981327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:01.247686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724801+0000","last_change":"2024-05-14T15:22:42.901814+0000","last_active":"2024-05-14T15:24:50.724801+0000","last_peered":"2024-05-14T15:24:50.724801+0000","last_clean":"2024-05-14T15:24:50.724801+0000","last_became_active":"2024-05-14T15:22:42.896795+0000","last_became_peered":"2024-05-14T15:22:42.896795+0000","last_unstale":"2024-05-14T15:24:50.724801+0000","last_undegraded":"2024-05-14T15:24:50.724801+0000","last_fullsized":"2024-05-14T15:24:50.724801+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:57:15.981532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887356+0000","last_change":"2024-05-14T15:20:55.932569+0000","last_active":"2024-05-14T15:24:50.887356+0000","last_peered":"2024-05-14T15:24:50.887356+0000","last_clean":"2024-05-14T15:24:50.887356+0000","last_became_active":"2024-05-14T15:20:55.931484+0000","last_became_peered":"2024-05-14T15:20:55.931484+0000","last_unstale":"2024-05-14T15:24:50.887356+0000","last_undegraded":"2024-05-14T15:24:50.887356+0000","last_fullsized":"2024-05-14T15:24:50.887356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:18:54.155520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617801+0000","last_change":"2024-05-14T15:21:06.539101+0000","last_active":"2024-05-14T15:24:50.617801+0000","last_peered":"2024-05-14T15:24:50.617801+0000","last_clean":"2024-05-14T15:24:50.617801+0000","last_became_active":"2024-05-14T15:21:06.538673+0000","last_became_peered":"2024-05-14T15:21:06.538673+0000","last_unstale":"2024-05-14T15:24:50.617801+0000","last_undegraded":"2024-05-14T15:24:50.617801+0000","last_fullsized":"2024-05-14T15:24:50.617801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:06:37.882323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621026+0000","last_change":"2024-05-14T15:20:52.998491+0000","last_active":"2024-05-14T15:24:50.621026+0000","last_peered":"2024-05-14T15:24:50.621026+0000","last_clean":"2024-05-14T15:24:50.621026+0000","last_became_active":"2024-05-14T15:20:52.998078+0000","last_became_peered":"2024-05-14T15:20:52.998078+0000","last_unstale":"2024-05-14T15:24:50.621026+0000","last_undegraded":"2024-05-14T15:24:50.621026+0000","last_fullsized":"2024-05-14T15:24:50.621026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:42:32.431018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887834+0000","last_change":"2024-05-14T15:20:55.932888+0000","last_active":"2024-05-14T15:24:50.887834+0000","last_peered":"2024-05-14T15:24:50.887834+0000","last_clean":"2024-05-14T15:24:50.887834+0000","last_became_active":"2024-05-14T15:20:55.932523+0000","last_became_peered":"2024-05-14T15:20:55.932523+0000","last_unstale":"2024-05-14T15:24:50.887834+0000","last_undegraded":"2024-05-14T15:24:50.887834+0000","last_fullsized":"2024-05-14T15:24:50.887834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:51:00.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.981549+0000","last_change":"2024-05-14T15:21:06.535729+0000","last_active":"2024-05-14T15:24:50.981549+0000","last_peered":"2024-05-14T15:24:50.981549+0000","last_clean":"2024-05-14T15:24:50.981549+0000","last_became_active":"2024-05-14T15:21:06.535261+0000","last_became_peered":"2024-05-14T15:21:06.535261+0000","last_unstale":"2024-05-14T15:24:50.981549+0000","last_undegraded":"2024-05-14T15:24:50.981549+0000","last_fullsized":"2024-05-14T15:24:50.981549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:07:20.147022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":93,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.850897+0000","last_change":"2024-05-14T15:20:52.899189+0000","last_active":"2024-05-14T15:24:50.850897+0000","last_peered":"2024-05-14T15:24:50.850897+0000","last_clean":"2024-05-14T15:24:50.850897+0000","last_became_active":"2024-05-14T15:20:52.898780+0000","last_became_peered":"2024-05-14T15:20:52.898780+0000","last_unstale":"2024-05-14T15:24:50.850897+0000","last_undegraded":"2024-05-14T15:24:50.850897+0000","last_fullsized":"2024-05-14T15:24:50.850897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:15:48.235107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887586+0000","last_change":"2024-05-14T15:20:55.932377+0000","last_active":"2024-05-14T15:24:50.887586+0000","last_peered":"2024-05-14T15:24:50.887586+0000","last_clean":"2024-05-14T15:24:50.887586+0000","last_became_active":"2024-05-14T15:20:55.931403+0000","last_became_peered":"2024-05-14T15:20:55.931403+0000","last_unstale":"2024-05-14T15:24:50.887586+0000","last_undegraded":"2024-05-14T15:24:50.887586+0000","last_fullsized":"2024-05-14T15:24:50.887586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:02:19.979106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736013+0000","last_change":"2024-05-14T15:21:06.538359+0000","last_active":"2024-05-14T15:24:50.736013+0000","last_peered":"2024-05-14T15:24:50.736013+0000","last_clean":"2024-05-14T15:24:50.736013+0000","last_became_active":"2024-05-14T15:21:06.537873+0000","last_became_peered":"2024-05-14T15:21:06.537873+0000","last_unstale":"2024-05-14T15:24:50.736013+0000","last_undegraded":"2024-05-14T15:24:50.736013+0000","last_fullsized":"2024-05-14T15:24:50.736013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:21:05.514628+0000","last_clean_scrub_stamp":"2024-05-14T15:21:05.514628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:30.996257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983098+0000","last_change":"2024-05-14T15:22:43.235879+0000","last_active":"2024-05-14T15:24:50.983098+0000","last_peered":"2024-05-14T15:24:50.983098+0000","last_clean":"2024-05-14T15:24:50.983098+0000","last_became_active":"2024-05-14T15:22:43.230116+0000","last_became_peered":"2024-05-14T15:22:43.230116+0000","last_unstale":"2024-05-14T15:24:50.983098+0000","last_undegraded":"2024-05-14T15:24:50.983098+0000","last_fullsized":"2024-05-14T15:24:50.983098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:51.872440+0000","last_clean_scrub_stamp":"2024-05-14T15:20:51.872440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:25:50.192419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.887809+0000","last_change":"2024-05-14T15:20:55.934916+0000","last_active":"2024-05-14T15:24:50.887809+0000","last_peered":"2024-05-14T15:24:50.887809+0000","last_clean":"2024-05-14T15:24:50.887809+0000","last_became_active":"2024-05-14T15:20:55.934427+0000","last_became_peered":"2024-05-14T15:20:55.934427+0000","last_unstale":"2024-05-14T15:24:50.887809+0000","last_undegraded":"2024-05-14T15:24:50.887809+0000","last_fullsized":"2024-05-14T15:24:50.887809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:42:33.435635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004023+0000","last_change":"2024-05-14T15:20:55.928655+0000","last_active":"2024-05-14T15:24:51.004023+0000","last_peered":"2024-05-14T15:24:51.004023+0000","last_clean":"2024-05-14T15:24:51.004023+0000","last_became_active":"2024-05-14T15:20:55.928286+0000","last_became_peered":"2024-05-14T15:20:55.928286+0000","last_unstale":"2024-05-14T15:24:51.004023+0000","last_undegraded":"2024-05-14T15:24:51.004023+0000","last_fullsized":"2024-05-14T15:24:51.004023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:28:06.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619978+0000","last_change":"2024-05-14T15:20:55.924448+0000","last_active":"2024-05-14T15:24:50.619978+0000","last_peered":"2024-05-14T15:24:50.619978+0000","last_clean":"2024-05-14T15:24:50.619978+0000","last_became_active":"2024-05-14T15:20:55.924133+0000","last_became_peered":"2024-05-14T15:20:55.924133+0000","last_unstale":"2024-05-14T15:24:50.619978+0000","last_undegraded":"2024-05-14T15:24:50.619978+0000","last_fullsized":"2024-05-14T15:24:50.619978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:36.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.619801+0000","last_change":"2024-05-14T15:21:42.621443+0000","last_active":"2024-05-14T15:24:50.619801+0000","last_peered":"2024-05-14T15:24:50.619801+0000","last_clean":"2024-05-14T15:24:50.619801+0000","last_became_active":"2024-05-14T15:21:42.621029+0000","last_became_peered":"2024-05-14T15:21:42.621029+0000","last_unstale":"2024-05-14T15:24:50.619801+0000","last_undegraded":"2024-05-14T15:24:50.619801+0000","last_fullsized":"2024-05-14T15:24:50.619801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:21:09.268430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982632+0000","last_change":"2024-05-14T15:20:55.917777+0000","last_active":"2024-05-14T15:24:50.982632+0000","last_peered":"2024-05-14T15:24:50.982632+0000","last_clean":"2024-05-14T15:24:50.982632+0000","last_became_active":"2024-05-14T15:20:55.917421+0000","last_became_peered":"2024-05-14T15:20:55.917421+0000","last_unstale":"2024-05-14T15:24:50.982632+0000","last_undegraded":"2024-05-14T15:24:50.982632+0000","last_fullsized":"2024-05-14T15:24:50.982632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:12.071389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.983315+0000","last_change":"2024-05-14T15:20:55.919148+0000","last_active":"2024-05-14T15:24:50.983315+0000","last_peered":"2024-05-14T15:24:50.983315+0000","last_clean":"2024-05-14T15:24:50.983315+0000","last_became_active":"2024-05-14T15:20:55.918828+0000","last_became_peered":"2024-05-14T15:20:55.918828+0000","last_unstale":"2024-05-14T15:24:50.983315+0000","last_undegraded":"2024-05-14T15:24:50.983315+0000","last_fullsized":"2024-05-14T15:24:50.983315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:00:18.562362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004200+0000","last_change":"2024-05-14T15:21:42.667394+0000","last_active":"2024-05-14T15:24:51.004200+0000","last_peered":"2024-05-14T15:24:51.004200+0000","last_clean":"2024-05-14T15:24:51.004200+0000","last_became_active":"2024-05-14T15:21:42.666826+0000","last_became_peered":"2024-05-14T15:21:42.666826+0000","last_unstale":"2024-05-14T15:24:51.004200+0000","last_undegraded":"2024-05-14T15:24:51.004200+0000","last_fullsized":"2024-05-14T15:24:51.004200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:39:18.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.621083+0000","last_change":"2024-05-14T15:20:55.923961+0000","last_active":"2024-05-14T15:24:50.621083+0000","last_peered":"2024-05-14T15:24:50.621083+0000","last_clean":"2024-05-14T15:24:50.621083+0000","last_became_active":"2024-05-14T15:20:55.923627+0000","last_became_peered":"2024-05-14T15:20:55.923627+0000","last_unstale":"2024-05-14T15:24:50.621083+0000","last_undegraded":"2024-05-14T15:24:50.621083+0000","last_fullsized":"2024-05-14T15:24:50.621083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:37:53.079034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.725796+0000","last_change":"2024-05-14T15:20:55.919532+0000","last_active":"2024-05-14T15:24:50.725796+0000","last_peered":"2024-05-14T15:24:50.725796+0000","last_clean":"2024-05-14T15:24:50.725796+0000","last_became_active":"2024-05-14T15:20:55.919163+0000","last_became_peered":"2024-05-14T15:20:55.919163+0000","last_unstale":"2024-05-14T15:24:50.725796+0000","last_undegraded":"2024-05-14T15:24:50.725796+0000","last_fullsized":"2024-05-14T15:24:50.725796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:38:25.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.736083+0000","last_change":"2024-05-14T15:20:55.922260+0000","last_active":"2024-05-14T15:24:50.736083+0000","last_peered":"2024-05-14T15:24:50.736083+0000","last_clean":"2024-05-14T15:24:50.736083+0000","last_became_active":"2024-05-14T15:20:55.921868+0000","last_became_peered":"2024-05-14T15:20:55.921868+0000","last_unstale":"2024-05-14T15:24:50.736083+0000","last_undegraded":"2024-05-14T15:24:50.736083+0000","last_fullsized":"2024-05-14T15:24:50.736083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:57:59.384006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.724963+0000","last_change":"2024-05-14T15:20:55.920231+0000","last_active":"2024-05-14T15:24:50.724963+0000","last_peered":"2024-05-14T15:24:50.724963+0000","last_clean":"2024-05-14T15:24:50.724963+0000","last_became_active":"2024-05-14T15:20:55.919843+0000","last_became_peered":"2024-05-14T15:20:55.919843+0000","last_unstale":"2024-05-14T15:24:50.724963+0000","last_undegraded":"2024-05-14T15:24:50.724963+0000","last_fullsized":"2024-05-14T15:24:50.724963+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:34:37.538312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.617965+0000","last_change":"2024-05-14T15:20:55.935865+0000","last_active":"2024-05-14T15:24:50.617965+0000","last_peered":"2024-05-14T15:24:50.617965+0000","last_clean":"2024-05-14T15:24:50.617965+0000","last_became_active":"2024-05-14T15:20:55.935392+0000","last_became_peered":"2024-05-14T15:20:55.935392+0000","last_unstale":"2024-05-14T15:24:50.617965+0000","last_undegraded":"2024-05-14T15:24:50.617965+0000","last_fullsized":"2024-05-14T15:24:50.617965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:43:04.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:50.982527+0000","last_change":"2024-05-14T15:20:55.931615+0000","last_active":"2024-05-14T15:24:50.982527+0000","last_peered":"2024-05-14T15:24:50.982527+0000","last_clean":"2024-05-14T15:24:50.982527+0000","last_became_active":"2024-05-14T15:20:55.931193+0000","last_became_peered":"2024-05-14T15:20:55.931193+0000","last_unstale":"2024-05-14T15:24:50.982527+0000","last_undegraded":"2024-05-14T15:24:50.982527+0000","last_fullsized":"2024-05-14T15:24:50.982527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:34:16.830213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-14T15:24:51.004115+0000","last_change":"2024-05-14T15:20:55.919962+0000","last_active":"2024-05-14T15:24:51.004115+0000","last_peered":"2024-05-14T15:24:51.004115+0000","last_clean":"2024-05-14T15:24:51.004115+0000","last_became_active":"2024-05-14T15:20:55.919607+0000","last_became_peered":"2024-05-14T15:20:55.919607+0000","last_unstale":"2024-05-14T15:24:51.004115+0000","last_undegraded":"2024-05-14T15:24:51.004115+0000","last_fullsized":"2024-05-14T15:24:51.004115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T15:20:54.891446+0000","last_clean_scrub_stamp":"2024-05-14T15:20:54.891446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:00:33.407247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108306,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599109,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1169,"num_read_kb":697,"num_write":295,"num_write_kb":1443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1797327,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678079,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":528169,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874307,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1171456,"data_stored":1316406,"data_compressed":77588,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037834,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":925717,"data_compressed":54476,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234063,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":921541,"data_compressed":54476,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430291,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":1125157,"data_compressed":71432,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593815,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":513649,"data_compressed":32366,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":592078,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":524969,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":595524,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149798,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":1048591,"data_compressed":60632,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953570,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":528231,"data_compressed":35330,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790044,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1252345,"data_compressed":77588,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36102,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134390,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202547,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T15:24:57.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:24:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:24:58.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:57 smithi005 ceph-mon[31559]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 205 B/s wr, 0 op/s 2024-05-14T15:24:58.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:57 smithi144 ceph-mon[35650]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 205 B/s wr, 0 op/s 2024-05-14T15:24:58.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:57 smithi193 ceph-mon[36001]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 205 B/s wr, 0 op/s 2024-05-14T15:24:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:58 smithi144 ceph-mon[35650]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:58 smithi144 ceph-mon[35650]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:58 smithi144 ceph-mon[35650]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:58 smithi193 ceph-mon[36001]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:58 smithi193 ceph-mon[36001]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:58 smithi193 ceph-mon[36001]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:58 smithi005 ceph-mon[31559]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:58 smithi005 ceph-mon[31559]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:58 smithi005 ceph-mon[31559]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T15:24:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.971+0000 7fe87b7fb700 1 -- 172.21.15.5:0/667839767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 msgr2=0x7fe8741034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.971+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/667839767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe8741034b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe864009990 tx=0x7fe864009ca0 comp rx=0 tx=0).stop 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 -- 172.21.15.5:0/667839767 shutdown_connections 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/667839767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe874110c50 0x7fe874113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/667839767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe8741034b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/667839767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe874103a80 0x7fe874110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 -- 172.21.15.5:0/667839767 >> 172.21.15.5:0/667839767 conn(0x7fe8740fef30 msgr2=0x7fe874101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 -- 172.21.15.5:0/667839767 shutdown_connections 2024-05-14T15:24:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 -- 172.21.15.5:0/667839767 wait complete. 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 Processor -- start 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.972+0000 7fe87b7fb700 1 -- start start 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe87b7fb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe87419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe87b7fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe874103a80 0x7fe87419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe87b7fb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe874110c50 0x7fe8741a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe87b7fb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8741166a0 con 0x7fe8741030a0 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe87b7fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe874116520 con 0x7fe874103a80 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe87b7fb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe874116820 con 0x7fe874110c50 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe87419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe87419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60714/0 (socket says 172.21.15.5:60714) 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 -- 172.21.15.5:0/1112918414 learned_addr learned my addr 172.21.15.5:0/1112918414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:24:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879d98700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe874110c50 0x7fe8741a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe878d96700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe874103a80 0x7fe87419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 -- 172.21.15.5:0/1112918414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe874110c50 msgr2=0x7fe8741a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe874110c50 0x7fe8741a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 -- 172.21.15.5:0/1112918414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe874103a80 msgr2=0x7fe87419d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe874103a80 0x7fe87419d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 -- 172.21.15.5:0/1112918414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8741a1d90 con 0x7fe8741030a0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.973+0000 7fe879597700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe87419cf70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe864000c00 tx=0x7fe8640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.974+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864027070 con 0x7fe8741030a0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.974+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8640041d0 con 0x7fe8741030a0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.974+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a2020 con 0x7fe8741030a0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.974+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400f460 con 0x7fe8741030a0 2024-05-14T15:24:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.974+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a2500 con 0x7fe8741030a0 2024-05-14T15:24:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.975+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8740512c0 con 0x7fe8741030a0 2024-05-14T15:24:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.976+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fe86402a070 con 0x7fe8741030a0 2024-05-14T15:24:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.977+0000 7fe8727fc700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe85c0b2b10 0x7fe85c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:24:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.977+0000 7fe878d96700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe85c0b2b10 0x7fe85c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:24:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.977+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fe8640e3e10 con 0x7fe8741030a0 2024-05-14T15:24:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.978+0000 7fe878d96700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe85c0b2b10 0x7fe85c0b5020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8741122a0 tx=0x7fe868005d70 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:24:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:24:59.982+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe864013070 con 0x7fe8741030a0 2024-05-14T15:25:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.253+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87402e5f0 con 0x7fe8741030a0 2024-05-14T15:25:00.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:24:59 smithi005 ceph-mon[31559]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-05-14T15:25:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.253+0000 7fe8727fc700 1 -- 172.21.15.5:0/1112918414 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11864 (secure 0 0 0) 0x7fe864022470 con 0x7fe8741030a0 2024-05-14T15:25:00.255 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:25:00.256 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:24:47.806060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe85c0b2b10 msgr2=0x7fe85c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe85c0b2b10 0x7fe85c0b5020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8741122a0 tx=0x7fe868005d70 comp rx=0 tx=0).stop 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 msgr2=0x7fe87419cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe87419cf70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe864000c00 tx=0x7fe8640047b0 comp rx=0 tx=0).stop 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 shutdown_connections 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe85c0b2b10 0x7fe85c0b5020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe874110c50 0x7fe8741a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8741030a0 0x7fe87419cf70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 --2- 172.21.15.5:0/1112918414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe874103a80 0x7fe87419d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 >> 172.21.15.5:0/1112918414 conn(0x7fe8740fef30 msgr2=0x7fe8741125b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:00.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 shutdown_connections 2024-05-14T15:25:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:00.256+0000 7fe87b7fb700 1 -- 172.21.15.5:0/1112918414 wait complete. 2024-05-14T15:25:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-14T15:25:00.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:24:59 smithi144 ceph-mon[35650]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-05-14T15:25:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:24:59 smithi193 ceph-mon[36001]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.1 KiB/s wr, 16 op/s 2024-05-14T15:25:00.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:25:01.146 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:01.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:00 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1112918414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:25:01.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:00 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1112918414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:25:01.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:00 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1112918414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:25:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:01 smithi144 ceph-mon[35650]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 911 B/s wr, 12 op/s 2024-05-14T15:25:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:01 smithi193 ceph-mon[36001]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 911 B/s wr, 12 op/s 2024-05-14T15:25:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:01 smithi005 ceph-mon[31559]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 911 B/s wr, 12 op/s 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 -- 172.21.15.5:0/3755555026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c1045f0 msgr2=0x7f9d1c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 --2- 172.21.15.5:0/3755555026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c1045f0 0x7f9d1c104a00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9d180099c0 tx=0x7f9d18009cd0 comp rx=0 tx=0).stop 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 -- 172.21.15.5:0/3755555026 shutdown_connections 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 --2- 172.21.15.5:0/3755555026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c109870 0x7f9d1c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 --2- 172.21.15.5:0/3755555026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c1045f0 0x7f9d1c104a00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 --2- 172.21.15.5:0/3755555026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d1c108f20 0x7f9d1c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 -- 172.21.15.5:0/3755555026 >> 172.21.15.5:0/3755555026 conn(0x7f9d1c100040 msgr2=0x7f9d1c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:02.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 -- 172.21.15.5:0/3755555026 shutdown_connections 2024-05-14T15:25:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 -- 172.21.15.5:0/3755555026 wait complete. 2024-05-14T15:25:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.906+0000 7f9d238f9700 1 Processor -- start 2024-05-14T15:25:02.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 -- start start 2024-05-14T15:25:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d1c1045f0 0x7f9d1c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108f20 0x7f9d1c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 0x7f9d1c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:02.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1144a0 con 0x7f9d1c109870 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c114320 con 0x7f9d1c108f20 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d238f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c114620 con 0x7f9d1c1045f0 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 0x7f9d1c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 0x7f9d1c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60738/0 (socket says 172.21.15.5:60738) 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d20e94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108f20 0x7f9d1c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 -- 172.21.15.5:0/852688050 learned_addr learned my addr 172.21.15.5:0/852688050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21695700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d1c1045f0 0x7f9d1c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 -- 172.21.15.5:0/852688050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d1c1045f0 msgr2=0x7f9d1c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d1c1045f0 0x7f9d1c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 -- 172.21.15.5:0/852688050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108f20 msgr2=0x7f9d1c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108f20 0x7f9d1c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d21e96700 1 -- 172.21.15.5:0/852688050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c1a91b0 con 0x7f9d1c109870 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.907+0000 7f9d20e94700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108f20 0x7f9d1c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.908+0000 7f9d21e96700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 0x7f9d1c076540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400ef30 tx=0x7f9d1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:02.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.908+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140092e0 con 0x7f9d1c109870 2024-05-14T15:25:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.908+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d14004bb0 con 0x7f9d1c109870 2024-05-14T15:25:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.908+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004020 con 0x7f9d1c109870 2024-05-14T15:25:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.908+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c1a9440 con 0x7f9d1c109870 2024-05-14T15:25:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.908+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a9930 con 0x7f9d1c109870 2024-05-14T15:25:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.909+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c0512c0 con 0x7f9d1c109870 2024-05-14T15:25:02.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.910+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f9d1401b030 con 0x7f9d1c109870 2024-05-14T15:25:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.912+0000 7f9d127fc700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d080b2b10 0x7f9d080b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.912+0000 7f9d21695700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d080b2b10 0x7f9d080b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.912+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f9d140e3110 con 0x7f9d1c109870 2024-05-14T15:25:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.912+0000 7f9d21695700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d080b2b10 0x7f9d080b5020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d18003760 tx=0x7f9d1801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:02.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:02.915+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d14022020 con 0x7f9d1c109870 2024-05-14T15:25:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.130+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d1c02e7b0 con 0x7f9d1c109870 2024-05-14T15:25:03.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.132+0000 7f9d127fc700 1 -- 172.21.15.5:0/852688050 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25130 (secure 0 0 0) 0x7f9d14010040 con 0x7f9d1c109870 2024-05-14T15:25:03.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:25:03.134 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:24:50.611181+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2413445605":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/365942426":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/4280832550":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/3326284548":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d080b2b10 msgr2=0x7f9d080b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d080b2b10 0x7f9d080b5020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d18003760 tx=0x7f9d1801a040 comp rx=0 tx=0).stop 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 msgr2=0x7f9d1c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 0x7f9d1c076540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400ef30 tx=0x7f9d1400c3a0 comp rx=0 tx=0).stop 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 shutdown_connections 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d080b2b10 0x7f9d080b5020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d1c109870 0x7f9d1c076540 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d1c1045f0 0x7f9d1c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 --2- 172.21.15.5:0/852688050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108f20 0x7f9d1c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 >> 172.21.15.5:0/852688050 conn(0x7f9d1c100040 msgr2=0x7f9d1c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 shutdown_connections 2024-05-14T15:25:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:03.135+0000 7f9d238f9700 1 -- 172.21.15.5:0/852688050 wait complete. 2024-05-14T15:25:03.875 DEBUG:tasks.fwd_scrub:Yielding 2024-05-14T15:25:03.875 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T15:25:03.884 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T15:25:03.884 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:25:03.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-14T15:25:03.885 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-14T15:25:03.885 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:25:03.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 -- 172.21.15.5:0/373752028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170109040 msgr2=0x7fe170102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 --2- 172.21.15.5:0/373752028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170109040 0x7fe170102c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe158009ca0 tx=0x7fe15802a8c0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 -- 172.21.15.5:0/373752028 shutdown_connections 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 --2- 172.21.15.5:0/373752028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1701032f0 0x7fe17010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 --2- 172.21.15.5:0/373752028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170109040 0x7fe170102c80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 --2- 172.21.15.5:0/373752028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe170108ba0 0x7fe170102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 -- 172.21.15.5:0/373752028 >> 172.21.15.5:0/373752028 conn(0x7fe1700782e0 msgr2=0x7fe170078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:04.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 -- 172.21.15.5:0/373752028 shutdown_connections 2024-05-14T15:25:04.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.010+0000 7fe176b84640 1 -- 172.21.15.5:0/373752028 wait complete. 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.011+0000 7fe176b84640 1 Processor -- start 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.011+0000 7fe176b84640 1 -- start start 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe176b84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1701032f0 0x7fe17019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe176b84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 0x7fe17019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe176b84640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe170109040 0x7fe17019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe176b84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe170112d90 con 0x7fe170108ba0 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe176b84640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe170112c10 con 0x7fe1701032f0 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe176b84640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe170112f10 con 0x7fe170109040 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 0x7fe17019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 0x7fe17019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60764/0 (socket says 172.21.15.5:60764) 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 -- 172.21.15.5:0/3731336980 learned_addr learned my addr 172.21.15.5:0/3731336980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe174920640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1701032f0 0x7fe17019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe175121640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe170109040 0x7fe17019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 -- 172.21.15.5:0/3731336980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe170109040 msgr2=0x7fe17019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe170109040 0x7fe17019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 -- 172.21.15.5:0/3731336980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1701032f0 msgr2=0x7fe17019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1701032f0 0x7fe17019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.012+0000 7fe16ffff640 1 -- 172.21.15.5:0/3731336980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701a6b20 con 0x7fe170108ba0 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.013+0000 7fe16ffff640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 0x7fe17019bd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1580319f0 tx=0x7fe158031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.013+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158003bf0 con 0x7fe170108ba0 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.013+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe158003d90 con 0x7fe170108ba0 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.013+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701a6db0 con 0x7fe170108ba0 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.013+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15802fb70 con 0x7fe170108ba0 2024-05-14T15:25:04.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.013+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a7210 con 0x7fe170108ba0 2024-05-14T15:25:04.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.014+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17006a240 con 0x7fe170108ba0 2024-05-14T15:25:04.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.016+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fe15802fd10 con 0x7fe170108ba0 2024-05-14T15:25:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.017+0000 7fe16dffb640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe15c0b2a70 0x7fe15c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.017+0000 7fe174920640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe15c0b2a70 0x7fe15c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.017+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fe1581041e0 con 0x7fe170108ba0 2024-05-14T15:25:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.017+0000 7fe174920640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe15c0b2a70 0x7fe15c0b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe160005fd0 tx=0x7fe16000b510 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.021+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1580cb6a0 con 0x7fe170108ba0 2024-05-14T15:25:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:04.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.245+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fe1701a74e0 con 0x7fe170108ba0 2024-05-14T15:25:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.249+0000 7fe16dffb640 1 -- 172.21.15.5:0/3731336980 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7fe1580098b0 con 0x7fe170108ba0 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe15c0b2a70 msgr2=0x7fe15c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe15c0b2a70 0x7fe15c0b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe160005fd0 tx=0x7fe16000b510 comp rx=0 tx=0).stop 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 msgr2=0x7fe17019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 0x7fe17019bd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1580319f0 tx=0x7fe158031a20 comp rx=0 tx=0).stop 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 shutdown_connections 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe15c0b2a70 0x7fe15c0b4f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe170109040 0x7fe17019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170108ba0 0x7fe17019bd50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 --2- 172.21.15.5:0/3731336980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1701032f0 0x7fe17019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 >> 172.21.15.5:0/3731336980 conn(0x7fe1700782e0 msgr2=0x7fe1701012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 shutdown_connections 2024-05-14T15:25:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.252+0000 7fe176b84640 1 -- 172.21.15.5:0/3731336980 wait complete. 2024-05-14T15:25:04.264 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:03 smithi005 ceph-mon[31559]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 745 B/s wr, 13 op/s 2024-05-14T15:25:04.264 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:03 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/852688050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:25:04.265 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-14T15:25:04.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:03 smithi144 ceph-mon[35650]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 745 B/s wr, 13 op/s 2024-05-14T15:25:04.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:03 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/852688050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:25:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:03 smithi193 ceph-mon[36001]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 745 B/s wr, 13 op/s 2024-05-14T15:25:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:03 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/852688050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 -- 172.21.15.5:0/2972452681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 msgr2=0x7f637c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 --2- 172.21.15.5:0/2972452681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c102c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6364009900 tx=0x7f636402a410 comp rx=0 tx=0).stop 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 -- 172.21.15.5:0/2972452681 shutdown_connections 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 --2- 172.21.15.5:0/2972452681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637c103300 0x7f637c10f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 --2- 172.21.15.5:0/2972452681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c102c90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 --2- 172.21.15.5:0/2972452681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c108bb0 0x7f637c102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 -- 172.21.15.5:0/2972452681 >> 172.21.15.5:0/2972452681 conn(0x7f637c0792a0 msgr2=0x7f637c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 -- 172.21.15.5:0/2972452681 shutdown_connections 2024-05-14T15:25:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.385+0000 7f638235e640 1 -- 172.21.15.5:0/2972452681 wait complete. 2024-05-14T15:25:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 Processor -- start 2024-05-14T15:25:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 -- start start 2024-05-14T15:25:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c103300 0x7f637c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637c108bb0 0x7f637c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f637c112da0 con 0x7f637c109050 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f637c112c20 con 0x7f637c103300 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f638235e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f637c112f20 con 0x7f637c108bb0 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f63808fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f63808fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60780/0 (socket says 172.21.15.5:60780) 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f63808fb640 1 -- 172.21.15.5:0/864180691 learned_addr learned my addr 172.21.15.5:0/864180691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f637bfff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c103300 0x7f637c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.386+0000 7f6373fff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637c108bb0 0x7f637c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f637bfff640 1 -- 172.21.15.5:0/864180691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637c108bb0 msgr2=0x7f637c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f637bfff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637c108bb0 0x7f637c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f637bfff640 1 -- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 msgr2=0x7f637c19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f637bfff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f637bfff640 1 -- 172.21.15.5:0/864180691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c1a6a50 con 0x7f637c103300 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f63808fb640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f637bfff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c103300 0x7f637c19b740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9c0 tx=0x7f636c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c00ccc0 con 0x7f637c103300 2024-05-14T15:25:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c1a6d40 con 0x7f637c103300 2024-05-14T15:25:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c1a71f0 con 0x7f637c103300 2024-05-14T15:25:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636c0108e0 con 0x7f637c103300 2024-05-14T15:25:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.387+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c0187a0 con 0x7f637c103300 2024-05-14T15:25:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.388+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637c050870 con 0x7f637c103300 2024-05-14T15:25:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.391+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f636c010430 con 0x7f637c103300 2024-05-14T15:25:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.392+0000 7f6379ffb640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63680b2a70 0x7f63680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.392+0000 7f6373fff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63680b2a70 0x7f63680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.392+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f636c0e3420 con 0x7f637c103300 2024-05-14T15:25:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.393+0000 7f6373fff640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63680b2a70 0x7f63680b4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6364009230 tx=0x7f6364031400 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.395+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f636c0aa140 con 0x7f637c103300 2024-05-14T15:25:04.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.596+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f637c1a7510 con 0x7f637c103300 2024-05-14T15:25:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.604+0000 7f6379ffb640 1 -- 172.21.15.5:0/864180691 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f636c0ae440 con 0x7f637c103300 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63680b2a70 msgr2=0x7f63680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63680b2a70 0x7f63680b4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6364009230 tx=0x7f6364031400 comp rx=0 tx=0).stop 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c103300 msgr2=0x7f637c19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c103300 0x7f637c19b740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9c0 tx=0x7f636c00ee80 comp rx=0 tx=0).stop 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 shutdown_connections 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63680b2a70 0x7f63680b4f80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637c108bb0 0x7f637c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f637c109050 0x7f637c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 --2- 172.21.15.5:0/864180691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c103300 0x7f637c19b740 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 >> 172.21.15.5:0/864180691 conn(0x7f637c0792a0 msgr2=0x7f637c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 shutdown_connections 2024-05-14T15:25:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.607+0000 7f638235e640 1 -- 172.21.15.5:0/864180691 wait complete. 2024-05-14T15:25:04.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-14T15:25:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 -- 172.21.15.5:0/1964418077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010e1f0 msgr2=0x7f50e01106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 --2- 172.21.15.5:0/1964418077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010e1f0 0x7f50e01106e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009900 tx=0x7f50d802a410 comp rx=0 tx=0).stop 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 -- 172.21.15.5:0/1964418077 shutdown_connections 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 --2- 172.21.15.5:0/1964418077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e010b800 0x7f50e010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 --2- 172.21.15.5:0/1964418077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010e1f0 0x7f50e01106e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 --2- 172.21.15.5:0/1964418077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e0104da0 0x7f50e01051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 -- 172.21.15.5:0/1964418077 >> 172.21.15.5:0/1964418077 conn(0x7f50e00782e0 msgr2=0x7f50e0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 -- 172.21.15.5:0/1964418077 shutdown_connections 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.740+0000 7f50e5c15640 1 -- 172.21.15.5:0/1964418077 wait complete. 2024-05-14T15:25:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.741+0000 7f50e5c15640 1 Processor -- start 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.741+0000 7f50e5c15640 1 -- start start 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50e5c15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e0104da0 0x7f50e01a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50e5c15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 0x7f50e01a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50e5c15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e010e1f0 0x7f50e019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50e5c15640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0113e50 con 0x7f50e010b800 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50e5c15640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0113cd0 con 0x7f50e010e1f0 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50e5c15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0113fd0 con 0x7f50e0104da0 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 0x7f50e01a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 0x7f50e01a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60794/0 (socket says 172.21.15.5:60794) 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 -- 172.21.15.5:0/3885254244 learned_addr learned my addr 172.21.15.5:0/3885254244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50df7fe640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e0104da0 0x7f50e01a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50dffff640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e010e1f0 0x7f50e019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 -- 172.21.15.5:0/3885254244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e0104da0 msgr2=0x7f50e01a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e0104da0 0x7f50e01a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 -- 172.21.15.5:0/3885254244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e010e1f0 msgr2=0x7f50e019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e010e1f0 0x7f50e019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 -- 172.21.15.5:0/3885254244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e019ada0 con 0x7f50e010b800 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50dffff640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e010e1f0 0x7f50e019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.742+0000 7f50deffd640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 0x7f50e01a0a30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9f0 tx=0x7f50c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.743+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c800cd80 con 0x7f50e010b800 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.743+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c80054d0 con 0x7f50e010b800 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.743+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8010690 con 0x7f50e010b800 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.743+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e019b030 con 0x7f50e010b800 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.743+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e01a7290 con 0x7f50e010b800 2024-05-14T15:25:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.744+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e006add0 con 0x7f50e010b800 2024-05-14T15:25:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.745+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f50c8004e30 con 0x7f50e010b800 2024-05-14T15:25:04.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.746+0000 7f50dcff9640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f50cc0b2af0 0x7f50cc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:04.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.746+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f50c80e3250 con 0x7f50e010b800 2024-05-14T15:25:04.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.746+0000 7f50df7fe640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f50cc0b2af0 0x7f50cc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:04.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.747+0000 7f50df7fe640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f50cc0b2af0 0x7f50cc0b5000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005fd0 tx=0x7f50d0009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:04.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.750+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50c8054df0 con 0x7f50e010b800 2024-05-14T15:25:04.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.949+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f50e01a74d0 con 0x7f50e010b800 2024-05-14T15:25:04.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.953+0000 7f50dcff9640 1 -- 172.21.15.5:0/3885254244 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f50c80aa620 con 0x7f50e010b800 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f50cc0b2af0 msgr2=0x7f50cc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f50cc0b2af0 0x7f50cc0b5000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50d0005fd0 tx=0x7f50d0009500 comp rx=0 tx=0).stop 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 msgr2=0x7f50e01a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 0x7f50e01a0a30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9f0 tx=0x7f50c800edf0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 shutdown_connections 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f50cc0b2af0 0x7f50cc0b5000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e010e1f0 0x7f50e019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50e010b800 0x7f50e01a0a30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 --2- 172.21.15.5:0/3885254244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50e0104da0 0x7f50e01a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 >> 172.21.15.5:0/3885254244 conn(0x7f50e00782e0 msgr2=0x7f50e01012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:04.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 shutdown_connections 2024-05-14T15:25:04.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:04.956+0000 7f50e5c15640 1 -- 172.21.15.5:0/3885254244 wait complete. 2024-05-14T15:25:04.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 -- 172.21.15.5:0/3686170025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c1043f0 msgr2=0x7fef8c1107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 --2- 172.21.15.5:0/3686170025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c1043f0 0x7fef8c1107c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ca0 tx=0x7fef8402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 -- 172.21.15.5:0/3686170025 shutdown_connections 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 --2- 172.21.15.5:0/3686170025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c1043f0 0x7fef8c1107c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 --2- 172.21.15.5:0/3686170025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c10a140 0x7fef8c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 --2- 172.21.15.5:0/3686170025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef8c109ca0 0x7fef8c103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 -- 172.21.15.5:0/3686170025 >> 172.21.15.5:0/3686170025 conn(0x7fef8c0ff340 msgr2=0x7fef8c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 -- 172.21.15.5:0/3686170025 shutdown_connections 2024-05-14T15:25:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.089+0000 7fef921ce640 1 -- 172.21.15.5:0/3686170025 wait complete. 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 Processor -- start 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 -- start start 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c1043f0 0x7fef8c1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 0x7fef8c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef8c10a140 0x7fef8c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c113e90 con 0x7fef8c109ca0 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c113d10 con 0x7fef8c1043f0 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.090+0000 7fef921ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c114010 con 0x7fef8c10a140 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 0x7fef8c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 0x7fef8c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60812/0 (socket says 172.21.15.5:60812) 2024-05-14T15:25:05.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 -- 172.21.15.5:0/250313767 learned_addr learned my addr 172.21.15.5:0/250313767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef8b7fe640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c1043f0 0x7fef8c1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef8bfff640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef8c10a140 0x7fef8c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 -- 172.21.15.5:0/250313767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef8c10a140 msgr2=0x7fef8c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef8c10a140 0x7fef8c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 -- 172.21.15.5:0/250313767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c1043f0 msgr2=0x7fef8c1a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c1043f0 0x7fef8c1a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 -- 172.21.15.5:0/250313767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c19ad30 con 0x7fef8c109ca0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef8b7fe640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c1043f0 0x7fef8c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef82ffd640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 0x7fef8c1a0a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c970 tx=0x7fef7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74007c70 con 0x7fef8c109ca0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c19b020 con 0x7fef8c109ca0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef74007e10 con 0x7fef8c109ca0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74018760 con 0x7fef8c109ca0 2024-05-14T15:25:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.091+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c1a72c0 con 0x7fef8c109ca0 2024-05-14T15:25:05.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.092+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c104f80 con 0x7fef8c109ca0 2024-05-14T15:25:05.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.094+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fef7400fab0 con 0x7fef8c109ca0 2024-05-14T15:25:05.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.095+0000 7fef897fa640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fef780b2a70 0x7fef780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.095+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fef740e2e40 con 0x7fef8c109ca0 2024-05-14T15:25:05.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.095+0000 7fef8b7fe640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fef780b2a70 0x7fef780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.096+0000 7fef8b7fe640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fef780b2a70 0x7fef780b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0058c0 tx=0x7fef7c005ec0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:05.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.099+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef74014030 con 0x7fef8c109ca0 2024-05-14T15:25:05.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.316+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fef8c063110 con 0x7fef780b2a70 2024-05-14T15:25:05.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.388+0000 7fef897fa640 1 -- 172.21.15.5:0/250313767 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fef8c063110 con 0x7fef780b2a70 2024-05-14T15:25:05.389 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fef780b2a70 msgr2=0x7fef780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fef780b2a70 0x7fef780b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0058c0 tx=0x7fef7c005ec0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 msgr2=0x7fef8c1a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 0x7fef8c1a0a60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c970 tx=0x7fef7400ce30 comp rx=0 tx=0).stop 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 shutdown_connections 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fef780b2a70 0x7fef780b4f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef8c10a140 0x7fef8c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef8c109ca0 0x7fef8c1a0a60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 --2- 172.21.15.5:0/250313767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef8c1043f0 0x7fef8c1a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 >> 172.21.15.5:0/250313767 conn(0x7fef8c0ff340 msgr2=0x7fef8c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 shutdown_connections 2024-05-14T15:25:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.390+0000 7fef921ce640 1 -- 172.21.15.5:0/250313767 wait complete. 2024-05-14T15:25:05.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-14T15:25:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/2593386821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 msgr2=0x7f7d0810f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/2593386821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d0810f6e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009ca0 tx=0x7f7d0002a8c0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/2593386821 shutdown_connections 2024-05-14T15:25:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/2593386821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d08109060 0x7f7d08102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/2593386821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d0810f6e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/2593386821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d08108bc0 0x7f7d08102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/2593386821 >> 172.21.15.5:0/2593386821 conn(0x7f7d080781b0 msgr2=0x7f7d08078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/2593386821 shutdown_connections 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.528+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/2593386821 wait complete. 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 Processor -- start 2024-05-14T15:25:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 -- start start 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d081a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d08108bc0 0x7f7d081a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d08109060 0x7f7d0819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112db0 con 0x7f7d08103310 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112c30 con 0x7f7d08109060 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.529+0000 7f7d0e4e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112f30 con 0x7f7d08108bc0 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d081a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d081a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60834/0 (socket says 172.21.15.5:60834) 2024-05-14T15:25:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 -- 172.21.15.5:0/410689073 learned_addr learned my addr 172.21.15.5:0/410689073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d0ca7d640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d08109060 0x7f7d0819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d077fe640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d08108bc0 0x7f7d081a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 -- 172.21.15.5:0/410689073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d08108bc0 msgr2=0x7f7d081a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d08108bc0 0x7f7d081a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 -- 172.21.15.5:0/410689073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d08109060 msgr2=0x7f7d0819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d08109060 0x7f7d0819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 -- 172.21.15.5:0/410689073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0819aca0 con 0x7f7d08103310 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d07fff640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d081a04e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c970 tx=0x7f7cf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8007bb0 con 0x7f7d08103310 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0819af90 con 0x7f7d08103310 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf8007d50 con 0x7f7d08103310 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.530+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800f450 con 0x7f7d08103310 2024-05-14T15:25:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.531+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a7280 con 0x7f7d08103310 2024-05-14T15:25:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.532+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d08103ea0 con 0x7f7d08103310 2024-05-14T15:25:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.533+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f7cf8004ec0 con 0x7f7d08103310 2024-05-14T15:25:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.534+0000 7f7d057fa640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7cf00b2a70 0x7f7cf00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.534+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f7cf80e2940 con 0x7f7d08103310 2024-05-14T15:25:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.534+0000 7f7d077fe640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7cf00b2a70 0x7f7cf00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.535+0000 7f7d077fe640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7cf00b2a70 0x7f7cf00b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d0819be00 tx=0x7f7cfc005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:05.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.540+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cf8014030 con 0x7f7d08103310 2024-05-14T15:25:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3731336980' entity='client.admin' 2024-05-14T15:25:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='client.? ' entity='client.admin' 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3885254244' entity='client.admin' 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3731336980' entity='client.admin' 2024-05-14T15:25:05.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='client.? ' entity='client.admin' 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3885254244' entity='client.admin' 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3731336980' entity='client.admin' 2024-05-14T15:25:05.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='client.? ' entity='client.admin' 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3885254244' entity='client.admin' 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:25:05.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:25:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.764+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7d08063110 con 0x7f7cf00b2a70 2024-05-14T15:25:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.871+0000 7f7d057fa640 1 -- 172.21.15.5:0/410689073 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7d08063110 con 0x7f7cf00b2a70 2024-05-14T15:25:05.872 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7cf00b2a70 msgr2=0x7f7cf00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7cf00b2a70 0x7f7cf00b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d0819be00 tx=0x7f7cfc005f90 comp rx=0 tx=0).stop 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 msgr2=0x7f7d081a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d081a04e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c970 tx=0x7f7cf800ce30 comp rx=0 tx=0).stop 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 shutdown_connections 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7cf00b2a70 0x7f7cf00b4f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d08108bc0 0x7f7d081a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d08109060 0x7f7d0819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 --2- 172.21.15.5:0/410689073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d08103310 0x7f7d081a04e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 >> 172.21.15.5:0/410689073 conn(0x7f7d080781b0 msgr2=0x7f7d0806b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 shutdown_connections 2024-05-14T15:25:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:05.874+0000 7f7d0e4e0640 1 -- 172.21.15.5:0/410689073 wait complete. 2024-05-14T15:25:05.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T15:25:06.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2058575072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4109030 msgr2=0x7fcab4102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2058575072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4109030 0x7fcab4102c70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402a410 comp rx=0 tx=0).stop 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2058575072 shutdown_connections 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2058575072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcab41032e0 0x7fcab410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2058575072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4109030 0x7fcab4102c70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2058575072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4108b90 0x7fcab4102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2058575072 >> 172.21.15.5:0/2058575072 conn(0x7fcab40fe390 msgr2=0x7fcab41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2058575072 shutdown_connections 2024-05-14T15:25:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.006+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2058575072 wait complete. 2024-05-14T15:25:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 Processor -- start 2024-05-14T15:25:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 -- start start 2024-05-14T15:25:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcab41032e0 0x7fcab419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 0x7fcab419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4109030 0x7fcab419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab4112dd0 con 0x7fcab4108b90 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcab4112c50 con 0x7fcab4109030 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcabb9a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcab4112f50 con 0x7fcab41032e0 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcab8f44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 0x7fcab419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcab8f44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 0x7fcab419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44494/0 (socket says 172.21.15.5:44494) 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcab8f44640 1 -- 172.21.15.5:0/2592321772 learned_addr learned my addr 172.21.15.5:0/2592321772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.007+0000 7fcab9f46640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4109030 0x7fcab419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab9745640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcab41032e0 0x7fcab419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab8f44640 1 -- 172.21.15.5:0/2592321772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcab41032e0 msgr2=0x7fcab419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab8f44640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcab41032e0 0x7fcab419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab8f44640 1 -- 172.21.15.5:0/2592321772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4109030 msgr2=0x7fcab419c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab8f44640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4109030 0x7fcab419c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab8f44640 1 -- 172.21.15.5:0/2592321772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab41a6b00 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab9f46640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4109030 0x7fcab419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcab8f44640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 0x7fcab419bd30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005ab0 tx=0x7fcaa4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa403f070 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa4031b50 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab41a6d90 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.008+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4035030 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.009+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab41a7220 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.010+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab4050870 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.012+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fcaa4042070 con 0x7fcab4108b90 2024-05-14T15:25:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.013+0000 7fcaaa7fc640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcaa00b2a70 0x7fcaa00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.013+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fcaa4103900 con 0x7fcab4108b90 2024-05-14T15:25:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.013+0000 7fcab9745640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcaa00b2a70 0x7fcaa00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.013+0000 7fcab9745640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcaa00b2a70 0x7fcaa00b4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005d00 tx=0x7fcab000b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.017+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcaa40ca750 con 0x7fcab4108b90 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 -- 172.21.15.5:0/1924817007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc1045d0 msgr2=0x7f59bc1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 --2- 172.21.15.5:0/1924817007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc1045d0 0x7f59bc1049e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0099c0 tx=0x7f59ac009cd0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 -- 172.21.15.5:0/1924817007 shutdown_connections 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 --2- 172.21.15.5:0/1924817007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59bc109850 0x7f59bc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 --2- 172.21.15.5:0/1924817007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc1045d0 0x7f59bc1049e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 --2- 172.21.15.5:0/1924817007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc108f00 0x7f59bc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 -- 172.21.15.5:0/1924817007 >> 172.21.15.5:0/1924817007 conn(0x7f59bc0ffea0 msgr2=0x7f59bc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.054+0000 7f59c1382700 1 -- 172.21.15.5:0/1924817007 shutdown_connections 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.055+0000 7f59c1382700 1 -- 172.21.15.5:0/1924817007 wait complete. 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.055+0000 7f59c1382700 1 Processor -- start 2024-05-14T15:25:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.055+0000 7f59c1382700 1 -- start start 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59c1382700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc1045d0 0x7f59bc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59c1382700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 0x7f59bc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59c1382700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59bc109850 0x7f59bc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59c1382700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1144d0 con 0x7f59bc108f00 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59c1382700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc114350 con 0x7f59bc1045d0 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59c1382700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc114650 con 0x7f59bc109850 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 0x7f59bc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 0x7f59bc19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44512/0 (socket says 172.21.15.5:44512) 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2640339492 learned_addr learned my addr 172.21.15.5:0/2640339492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59bb7fe700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59bc109850 0x7f59bc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59baffd700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc1045d0 0x7f59bc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2640339492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59bc109850 msgr2=0x7f59bc19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59bc109850 0x7f59bc19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2640339492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc1045d0 msgr2=0x7f59bc19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc1045d0 0x7f59bc19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2640339492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc19fd10 con 0x7f59bc108f00 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59baffd700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc1045d0 0x7f59bc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.056+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 0x7f59bc19b3e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59b000e970 tx=0x7f59b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.057+0000 7f59a3fff700 1 -- 172.21.15.5:0/2640339492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b000cbd0 con 0x7f59bc108f00 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.057+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc1a0000 con 0x7f59bc108f00 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.057+0000 7f59a3fff700 1 -- 172.21.15.5:0/2640339492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b000cd70 con 0x7f59bc108f00 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.057+0000 7f59a3fff700 1 -- 172.21.15.5:0/2640339492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0005680 con 0x7f59bc108f00 2024-05-14T15:25:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.057+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1a7920 con 0x7f59bc108f00 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.060+0000 7f59a3fff700 1 -- 172.21.15.5:0/2640339492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f59b0004750 con 0x7f59bc108f00 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59a3fff700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f59a40b2b10 0x7f59a40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 Processor -- start 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 -- start start 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59baffd700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f59a40b2b10 0x7f59a40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59a3fff700 1 -- 172.21.15.5:0/2640339492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f59b00e2f30 con 0x7f59bc108f00 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80bbe00 0x55c4f80b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4f7fd79c0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4f7f3ce20 con 0x55c4f80bbe00 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4f7fd3980 con 0x55c4f80b5e90 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59baffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59baffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44516/0 (socket says 172.21.15.5:44516) 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59baffd700 1 -- 172.21.15.5:0/1773665144 learned_addr learned my addr 172.21.15.5:0/1773665144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.061+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80bbe00 0x55c4f80b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59bb7fe700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f59a40b2b10 0x7f59a40b5020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59ac003760 tx=0x7f59ac005fb0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 -- 172.21.15.5:0/1773665144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 msgr2=0x55c4f80b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 -- 172.21.15.5:0/1773665144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80bbe00 msgr2=0x55c4f80b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80bbe00 0x55c4f80b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 -- 172.21.15.5:0/1773665144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4f7f24950 con 0x55c4f7f37ec0 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80bbe00 0x55c4f80b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59bb7fe700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59baffd700 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bb8c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59ac015000 tx=0x7f59ac025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73de80dc5a43d456 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59a17fa700 1 -- 172.21.15.5:0/1773665144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac032070 con 0x55c4f7f37ec0 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59a17fa700 1 -- 172.21.15.5:0/1773665144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ac005500 con 0x55c4f7f37ec0 2024-05-14T15:25:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.062+0000 7f59a17fa700 1 -- 172.21.15.5:0/1773665144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00be50 con 0x55c4f7f37ec0 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1773665144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 msgr2=0x55c4f80bb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bb8c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59ac015000 tx=0x7f59ac025000 comp rx=0 tx=0).stop 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1773665144 shutdown_connections 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bb8c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1773665144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80bbe00 0x55c4f80b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1773665144 >> 172.21.15.5:0/1773665144 conn(0x55c4f7f7f6b0 msgr2=0x55c4f80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1773665144 shutdown_connections 2024-05-14T15:25:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.063+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1773665144 wait complete. 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.064+0000 7f59cfd96b80 1 -- start start 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.064+0000 7f59cfd96b80 1 Processor -- start 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.064+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.064+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80bbe00 0x55c4f80c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4f80c26c0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4f7f90f50 con 0x55c4f80b5e90 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4f80c2840 con 0x55c4f80bbe00 2024-05-14T15:25:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44528/0 (socket says 172.21.15.5:44528) 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 -- 172.21.15.5:0/2238036135 learned_addr learned my addr 172.21.15.5:0/2238036135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59cfd96b80 10 client.0 _ll_register_callbacks cb 0x55c4f7f6f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59bb7fe700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80bbe00 0x55c4f80c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 -- 172.21.15.5:0/2238036135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80bbe00 msgr2=0x55c4f80c0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80bbe00 0x55c4f80c0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 -- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 msgr2=0x55c4f80bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80bfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 -- 172.21.15.5:0/2238036135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4f7f8e7e0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f59baffd700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bf5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098e0 tx=0x7f59ac009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96f87f04cd07a7c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f599eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.065+0000 7f599eff5700 1 -- 172.21.15.5:0/2238036135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac032030 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac005500 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f599eff5700 1 -- 172.21.15.5:0/2238036135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f59ac005260 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f599eff5700 1 -- 172.21.15.5:0/2238036135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59ac047050 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 10 client.15426 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c4f80c26c0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f599eff5700 1 -- 172.21.15.5:0/2238036135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ac005500 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f599eff5700 1 -- 172.21.15.5:0/2238036135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f59ac038070 con 0x55c4f7f37ec0 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 10 client.15426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 20 client.15426 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.066+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f80cf650 0x55c4f80d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.067+0000 7f59cfd96b80 4 client.15426 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.067+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4f7fe53a0 con 0x55c4f80cf650 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.067+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f80cf650 0x55c4f80d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.067+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f80cf650 0x55c4f80d1a70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ae8fec181cbcb4 server_cookie=e934d04b6fde99ac in_seq=0 out_seq=0 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.067+0000 7f599eff5700 10 client.15426 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f599eff5700 1 -- 172.21.15.5:0/2238036135 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4f7fe53a0 con 0x55c4f80cf650 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f599eff5700 10 client.15426 handle_command_reply: tid=0 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 client.15426 shutdown 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 msgr2=0x55c4f80bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bf5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098e0 tx=0x7f59ac009cd0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 shutdown_connections 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f80cf650 0x55c4f80d1a70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80bbe00 0x55c4f80c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f7f37ec0 0x55c4f80bf5f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/2238036135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 >> 172.21.15.5:0/2238036135 conn(0x55c4f7f7f6b0 msgr2=0x55c4f80b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f599eff5700 0 client.15426 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 shutdown_connections 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 1 -- 172.21.15.5:0/2238036135 wait complete. 2024-05-14T15:25:06.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.068+0000 7f59cfd96b80 20 client.15426 trim_cache size 0 max 16384 2024-05-14T15:25:06.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.104+0000 7f59cfd96b80 1 Processor -- start 2024-05-14T15:25:06.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 -- start start 2024-05-14T15:25:06.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f7f37ec0 0x55c4f80b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b2f60 con 0x55c4f80bbe00 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b07f0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1ae640 con 0x55c4f80b5e90 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52866/0 (socket says 172.21.15.5:52866) 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80bfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44532/0 (socket says 172.21.15.5:44532) 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 -- 172.21.15.5:0/1200751173 learned_addr learned my addr 172.21.15.5:0/1200751173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59baffd700 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f7f37ec0 0x55c4f80b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 -- 172.21.15.5:0/1200751173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f7f37ec0 msgr2=0x55c4f80b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f7f37ec0 0x55c4f80b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 -- 172.21.15.5:0/1200751173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 msgr2=0x55c4f80bfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80bfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.105+0000 7f59ba7fc700 1 -- 172.21.15.5:0/1200751173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4f80aa210 con 0x55c4f80b5e90 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.106+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80c16d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59b00a9900 tx=0x7f59b000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1452c5d53b29c6b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.106+0000 7f59a0ff9700 1 -- 172.21.15.5:0/1200751173 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00a9710 con 0x55c4f80b5e90 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.106+0000 7f59a0ff9700 1 -- 172.21.15.5:0/1200751173 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b008e070 con 0x55c4f80b5e90 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.106+0000 7f59a0ff9700 1 -- 172.21.15.5:0/1200751173 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00b25d0 con 0x55c4f80b5e90 2024-05-14T15:25:06.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1200751173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 msgr2=0x55c4f80c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80c16d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59b00a9900 tx=0x7f59b000c6e0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1200751173 shutdown_connections 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80bfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f7f37ec0 0x55c4f80b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/1200751173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f80b5e90 0x55c4f80c16d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1200751173 >> 172.21.15.5:0/1200751173 conn(0x55c4f7f7f6b0 msgr2=0x55c4f80be550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1200751173 shutdown_connections 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 -- 172.21.15.5:0/1200751173 wait complete. 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 -- start start 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.107+0000 7f59cfd96b80 1 Processor -- start 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 0x55c4f80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1aecb0 con 0x55c4f80bbe00 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b0120 con 0x55c4f80b5e90 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1af380 con 0x55c4f7f37ec0 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59cfd96b80 10 client.0 _ll_register_callbacks cb 0x55c4f7f6f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 0x55c4f80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44546/0 (socket says 172.21.15.5:44546) 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59bb7fe700 1 -- 172.21.15.5:0/3616175886 learned_addr learned my addr 172.21.15.5:0/3616175886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 0x55c4f80c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52874/0 (socket says 172.21.15.5:52874) 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 -- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 msgr2=0x55c4f80c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 -- 172.21.15.5:0/3616175886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 msgr2=0x55c4f80c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59baffd700 1 -- 172.21.15.5:0/3616175886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4f80c4fe0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.108+0000 7f59bb7fe700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f59baffd700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 0x55c4f80c7ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59ac003aa0 tx=0x7f59ac005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744072d015635f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f599eff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f599eff5700 1 -- 172.21.15.5:0/3616175886 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac053060 con 0x55c4f7f37ec0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b008e070 con 0x55c4f7f37ec0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f599eff5700 1 -- 172.21.15.5:0/3616175886 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f59ac025ca0 con 0x55c4f7f37ec0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f599eff5700 1 -- 172.21.15.5:0/3616175886 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59ac05a050 con 0x55c4f7f37ec0 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f59cfd96b80 10 client.25156 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:06.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c4f813a410 con 0x55c4f7f37ec0 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f599eff5700 1 -- 172.21.15.5:0/3616175886 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b008e070 con 0x55c4f7f37ec0 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.109+0000 7f599eff5700 1 -- 172.21.15.5:0/3616175886 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f59ac038040 con 0x55c4f7f37ec0 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59cfd96b80 10 client.25156 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59cfd96b80 10 client.25156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59cfd96b80 20 client.25156 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f813a9a0 0x55c4f813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59cfd96b80 4 client.25156 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4f7fdf440 con 0x55c4f813a9a0 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.110+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f813a9a0 0x55c4f813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.111+0000 7f59ba7fc700 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f813a9a0 0x55c4f813cdc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4ecae48a9d6984 server_cookie=63d8415005f07fc9 in_seq=0 out_seq=0 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.111+0000 7f599eff5700 10 client.25156 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:06.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f599eff5700 1 -- 172.21.15.5:0/3616175886 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4f7fdf440 con 0x55c4f813a9a0 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f599eff5700 10 client.25156 handle_command_reply: tid=0 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 client.25156 shutdown 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 msgr2=0x55c4f80c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 0x55c4f80c7ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f59ac003aa0 tx=0x7f59ac005590 comp rx=0 tx=0).stop 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 shutdown_connections 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c4f813a9a0 0x55c4f813cdc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f80bbe00 0x55c4f80c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4f7f37ec0 0x55c4f80c7ad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 --2- 172.21.15.5:0/3616175886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4f80b5e90 0x55c4f80c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 >> 172.21.15.5:0/3616175886 conn(0x55c4f7f7f6b0 msgr2=0x55c4f80c0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.112+0000 7f599eff5700 0 client.25156 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:06.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 shutdown_connections 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59cfd96b80 1 -- 172.21.15.5:0/3616175886 wait complete. 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59cfd96b80 20 client.25156 trim_cache size 0 max 16384 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "f26e5dc4-1f46-4410-b84f-ba30afe91b8d", 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:25:06.121 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:25:06.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f59a40b2b10 msgr2=0x7f59a40b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f59a40b2b10 0x7f59a40b5020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59ac003760 tx=0x7f59ac005fb0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 msgr2=0x7f59bc19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 0x7f59bc19b3e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59b000e970 tx=0x7f59b000ec80 comp rx=0 tx=0).stop 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 shutdown_connections 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f59a40b2b10 0x7f59a40b5020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59bc109850 0x7f59bc19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59bc1045d0 0x7f59bc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 --2- 172.21.15.5:0/2640339492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59bc108f00 0x7f59bc19b3e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 >> 172.21.15.5:0/2640339492 conn(0x7f59bc0ffea0 msgr2=0x7f59bc10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.113+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 shutdown_connections 2024-05-14T15:25:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.114+0000 7f59c1382700 1 -- 172.21.15.5:0/2640339492 wait complete. 2024-05-14T15:25:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.267+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fcab410ee80 con 0x7fcaa00b2a70 2024-05-14T15:25:06.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.276+0000 7fcaaa7fc640 1 -- 172.21.15.5:0/2592321772 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fcab410ee80 con 0x7fcaa00b2a70 2024-05-14T15:25:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T15:25:05", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-14T15:25:06.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcaa00b2a70 msgr2=0x7fcaa00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcaa00b2a70 0x7fcaa00b4f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005d00 tx=0x7fcab000b410 comp rx=0 tx=0).stop 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 msgr2=0x7fcab419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 0x7fcab419bd30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005ab0 tx=0x7fcaa4003b90 comp rx=0 tx=0).stop 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 shutdown_connections 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcaa00b2a70 0x7fcaa00b4f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcab4109030 0x7fcab419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab4108b90 0x7fcab419bd30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 --2- 172.21.15.5:0/2592321772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcab41032e0 0x7fcab419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 >> 172.21.15.5:0/2592321772 conn(0x7fcab40fe390 msgr2=0x7fcab410e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 shutdown_connections 2024-05-14T15:25:06.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.278+0000 7fcabb9a9640 1 -- 172.21.15.5:0/2592321772 wait complete. 2024-05-14T15:25:06.290 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T15:25:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.407+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/349321362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0103310 msgr2=0x7f0dc010f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.407+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/349321362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0103310 0x7f0dc010f6e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009ca0 tx=0x7f0db802a8c0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/349321362 shutdown_connections 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/349321362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0103310 0x7f0dc010f6e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/349321362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dc0109060 0x7f0dc0102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/349321362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc0108bc0 0x7f0dc0102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/349321362 >> 172.21.15.5:0/349321362 conn(0x7f0dc00fe390 msgr2=0x7f0dc01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/349321362 shutdown_connections 2024-05-14T15:25:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/349321362 wait complete. 2024-05-14T15:25:06.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 Processor -- start 2024-05-14T15:25:06.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.408+0000 7f0dc4ae9640 1 -- start start 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dc4ae9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dc0103310 0x7f0dc00fe870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dc4ae9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc0108bc0 0x7f0dc00fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dc4ae9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 0x7f0dc00ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dc4ae9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0112dd0 con 0x7f0dc0109060 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dc4ae9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0112c50 con 0x7f0dc0103310 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dc4ae9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0112f50 con 0x7f0dc0108bc0 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 0x7f0dc00ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 0x7f0dc00ff2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44572/0 (socket says 172.21.15.5:44572) 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 -- 172.21.15.5:0/1085748899 learned_addr learned my addr 172.21.15.5:0/1085748899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbdd9b640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc0108bc0 0x7f0dc00fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbe59c640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dc0103310 0x7f0dc00fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 -- 172.21.15.5:0/1085748899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc0108bc0 msgr2=0x7f0dc00fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc0108bc0 0x7f0dc00fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 -- 172.21.15.5:0/1085748899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dc0103310 msgr2=0x7f0dc00fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dc0103310 0x7f0dc00fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 -- 172.21.15.5:0/1085748899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc01a8c00 con 0x7f0dc0109060 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.409+0000 7f0dbed9d640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 0x7f0dc00ff2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005cc0 tx=0x7f0db802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.410+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8003bf0 con 0x7f0dc0109060 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.410+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc01a8e90 con 0x7f0dc0109060 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.410+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db8003d90 con 0x7f0dc0109060 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.410+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db802fbc0 con 0x7f0dc0109060 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.410+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc01a9320 con 0x7f0dc0109060 2024-05-14T15:25:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.411+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc0050870 con 0x7f0dc0109060 2024-05-14T15:25:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.412+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f0db802fd60 con 0x7f0dc0109060 2024-05-14T15:25:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.413+0000 7f0daf7fe640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0da80b2a70 0x7f0da80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.413+0000 7f0dbe59c640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0da80b2a70 0x7f0da80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.413+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0db81043b0 con 0x7f0dc0109060 2024-05-14T15:25:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.414+0000 7f0dbe59c640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0da80b2a70 0x7f0da80b4f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0db0005fd0 tx=0x7f0db000b390 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.417+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0db80cb890 con 0x7f0dc0109060 2024-05-14T15:25:06.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.633+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0dc01051a0 con 0x7f0da80b2a70 2024-05-14T15:25:06.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.642+0000 7f0daf7fe640 1 -- 172.21.15.5:0/1085748899 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0dc01051a0 con 0x7f0da80b2a70 2024-05-14T15:25:06.642 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0da80b2a70 msgr2=0x7f0da80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0da80b2a70 0x7f0da80b4f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0db0005fd0 tx=0x7f0db000b390 comp rx=0 tx=0).stop 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 msgr2=0x7f0dc00ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 0x7f0dc00ff2f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005cc0 tx=0x7f0db802f380 comp rx=0 tx=0).stop 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 shutdown_connections 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0da80b2a70 0x7f0da80b4f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc0108bc0 0x7f0dc00fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc0109060 0x7f0dc00ff2f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 --2- 172.21.15.5:0/1085748899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dc0103310 0x7f0dc00fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 >> 172.21.15.5:0/1085748899 conn(0x7f0dc00fe390 msgr2=0x7f0dc0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 shutdown_connections 2024-05-14T15:25:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:06.644+0000 7f0dc4ae9640 1 -- 172.21.15.5:0/1085748899 wait complete. 2024-05-14T15:25:06.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-14T15:25:06.686 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-14T15:25:06.701 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-14T15:25:06.770 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:25:06.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-14T15:25:06.773 INFO:teuthology.run_tasks:Running task workunit... 2024-05-14T15:25:06.796 INFO:tasks.workunit:Pulling workunits from ref e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:25:06.796 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-14T15:25:06.796 INFO:tasks.workunit:timeout=3h 2024-05-14T15:25:06.796 INFO:tasks.workunit:cleanup=True 2024-05-14T15:25:06.796 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-14T15:25:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-14T15:25:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-14T15:25:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-14T15:25:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T15:25:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-14T15:25:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-14 15:22:12.009009711 +0000 2024-05-14T15:25:06.815 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-14 15:22:12.009009711 +0000 2024-05-14T15:25:06.815 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-14 15:24:47.043095081 +0000 2024-05-14T15:25:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-14 15:22:12.009009711 +0000 2024-05-14T15:25:06.815 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-14T15:25:06.815 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-14T15:25:06.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:25:06.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:06 smithi005 ceph-mon[31559]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:06.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:06 smithi005 ceph-mon[31559]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:06.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:06 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:25:06.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:06 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:06.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:06 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:25:06.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:06 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:06.902 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e2b755a64f0d72176b57299d02e880cf0620d780 2024-05-14T15:25:06.959 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-14T15:25:07.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:06 smithi144 ceph-mon[35650]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:06 smithi144 ceph-mon[35650]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:06 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:25:07.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:06 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:07.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:06 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:25:07.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:06 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:07.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:06 smithi193 ceph-mon[36001]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:06 smithi193 ceph-mon[36001]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:06 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:25:07.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:06 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:07.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:06 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:25:07.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:06 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:07.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:07 smithi005 ceph-mon[31559]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:07 smithi005 ceph-mon[31559]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:07 smithi005 ceph-mon[31559]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 KiB/s wr, 38 op/s 2024-05-14T15:25:07.902 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:07 smithi144 ceph-mon[35650]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.903 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:07 smithi144 ceph-mon[35650]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.903 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:07 smithi144 ceph-mon[35650]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 KiB/s wr, 38 op/s 2024-05-14T15:25:07.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:07 smithi193 ceph-mon[36001]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:07 smithi193 ceph-mon[36001]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:25:07.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:07 smithi193 ceph-mon[36001]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 38 KiB/s wr, 38 op/s 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 -- 172.21.15.5:0/1894454635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 msgr2=0x7f5efc109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 --2- 172.21.15.5:0/1894454635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc109710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009990 tx=0x7f5eec009ca0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 -- 172.21.15.5:0/1894454635 shutdown_connections 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 --2- 172.21.15.5:0/1894454635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5efc1030e0 0x7f5efc111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 --2- 172.21.15.5:0/1894454635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc109710 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 --2- 172.21.15.5:0/1894454635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5efc102790 0x7f5efc102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 -- 172.21.15.5:0/1894454635 >> 172.21.15.5:0/1894454635 conn(0x7f5efc0fe4e0 msgr2=0x7f5efc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 -- 172.21.15.5:0/1894454635 shutdown_connections 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.933+0000 7f5f01377700 1 -- 172.21.15.5:0/1894454635 wait complete. 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.934+0000 7f5f01377700 1 Processor -- start 2024-05-14T15:25:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.934+0000 7f5f01377700 1 -- start start 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5f01377700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5efc102790 0x7f5efc19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5f01377700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5efc1030e0 0x7f5efc19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5f01377700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5f01377700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc114590 con 0x7f5efc109300 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5f01377700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc114410 con 0x7f5efc1030e0 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5f01377700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc114710 con 0x7f5efc102790 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efaffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5efc102790 0x7f5efc19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 -- 172.21.15.5:0/1280400844 learned_addr learned my addr 172.21.15.5:0/1280400844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efa7fc700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5efc1030e0 0x7f5efc19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 -- 172.21.15.5:0/1280400844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5efc102790 msgr2=0x7f5efc19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5efc102790 0x7f5efc19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 -- 172.21.15.5:0/1280400844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5efc1030e0 msgr2=0x7f5efc19b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5efc1030e0 0x7f5efc19b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.935+0000 7f5efb7fe700 1 -- 172.21.15.5:0/1280400844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc19fe30 con 0x7f5efc109300 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.936+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc19f790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4007ee0 tx=0x7f5ef400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.936+0000 7f5ee3fff700 1 -- 172.21.15.5:0/1280400844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4004bb0 con 0x7f5efc109300 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.936+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc1a0120 con 0x7f5efc109300 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.936+0000 7f5ee3fff700 1 -- 172.21.15.5:0/1280400844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ef4004d50 con 0x7f5efc109300 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.936+0000 7f5ee3fff700 1 -- 172.21.15.5:0/1280400844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4005920 con 0x7f5efc109300 2024-05-14T15:25:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.936+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc1a7a40 con 0x7f5efc109300 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.939+0000 7f5ee3fff700 1 -- 172.21.15.5:0/1280400844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f5ef401a020 con 0x7f5efc109300 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.940+0000 7f5ee3fff700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ee40b2b10 0x7f5ee40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.940+0000 7f5f0fd8bb80 1 Processor -- start 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.940+0000 7f5efaffd700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ee40b2b10 0x7f5ee40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.940+0000 7f5ee3fff700 1 -- 172.21.15.5:0/1280400844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5ef40e2300 con 0x7f5efc109300 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 -- start start 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ee40b2b10 0x7f5ee40b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0059d0 tx=0x7f5eec005960 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e25c4c4940 con 0x55e25c424e40 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e25c429da0 con 0x55e25c5a2e10 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e25c4c0900 con 0x55e25c5a8d80 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44612/0 (socket says 172.21.15.5:44612) 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 -- 172.21.15.5:0/3872268342 learned_addr learned my addr 172.21.15.5:0/3872268342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efa7fc700 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 -- 172.21.15.5:0/3872268342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 msgr2=0x55e25c5a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 -- 172.21.15.5:0/3872268342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 msgr2=0x55e25c5a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5a3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efaffd700 1 -- 172.21.15.5:0/3872268342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25c4118d0 con 0x55e25c424e40 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.941+0000 7f5efa7fc700 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.942+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a8840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5efc06be30 tx=0x7f5eec01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e665f29ac6e47d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.942+0000 7f5ee17fa700 1 -- 172.21.15.5:0/3872268342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec005430 con 0x55e25c424e40 2024-05-14T15:25:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.942+0000 7f5ee17fa700 1 -- 172.21.15.5:0/3872268342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec0055d0 con 0x55e25c424e40 2024-05-14T15:25:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.942+0000 7f5ee17fa700 1 -- 172.21.15.5:0/3872268342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00fa00 con 0x55e25c424e40 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3872268342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 msgr2=0x55e25c5a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a8840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5efc06be30 tx=0x7f5eec01e000 comp rx=0 tx=0).stop 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3872268342 shutdown_connections 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a8840 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3872268342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3872268342 >> 172.21.15.5:0/3872268342 conn(0x55e25c46c630 msgr2=0x55e25c59da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3872268342 shutdown_connections 2024-05-14T15:25:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3872268342 wait complete. 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 -- start start 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.943+0000 7f5f0fd8bb80 1 Processor -- start 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e25c5af760 con 0x55e25c5a8d80 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e25c47ded0 con 0x55e25c424e40 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e25c5af8e0 con 0x55e25c5a2e10 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5f0fd8bb80 10 client.0 _ll_register_callbacks cb 0x55e25c45c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44624/0 (socket says 172.21.15.5:44624) 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 -- 172.21.15.5:0/3144687259 learned_addr learned my addr 172.21.15.5:0/3144687259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efa7fc700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 -- 172.21.15.5:0/3144687259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 msgr2=0x55e25c5ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 -- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 msgr2=0x55e25c5ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 -- 172.21.15.5:0/3144687259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25c47b760 con 0x55e25c5a8d80 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5acf00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40a5030 tx=0x7f5ef4005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48938ac07adbfce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.944+0000 7f5edeff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5edeff5700 1 -- 172.21.15.5:0/3144687259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40a96b0 con 0x55e25c5a8d80 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5edeff5700 1 -- 172.21.15.5:0/3144687259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5ef40b1b90 con 0x55e25c5a8d80 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec0055d0 con 0x55e25c5a8d80 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5edeff5700 1 -- 172.21.15.5:0/3144687259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ef40a8fb0 con 0x55e25c5a8d80 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 10 client.15450 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e25c5ba350 con 0x55e25c5a8d80 2024-05-14T15:25:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5edeff5700 1 -- 172.21.15.5:0/3144687259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec0055d0 con 0x55e25c5a8d80 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5edeff5700 1 -- 172.21.15.5:0/3144687259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5ef40a8210 con 0x55e25c5a8d80 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 10 client.15450 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 20 client.15450 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c5bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 4 client.15450 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.945+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e25c4d2320 con 0x55e25c5bc990 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.946+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c5bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.946+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c5bedb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e8b624be7b58f0 server_cookie=5b9dc3eb36dd8926 in_seq=0 out_seq=0 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.946+0000 7f5edeff5700 10 client.15450 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5edeff5700 1 -- 172.21.15.5:0/3144687259 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e25c4d2320 con 0x55e25c5bc990 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5edeff5700 10 client.15450 handle_command_reply: tid=0 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 client.15450 shutdown 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 msgr2=0x55e25c5acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5acf00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40a5030 tx=0x7f5ef4005260 comp rx=0 tx=0).stop 2024-05-14T15:25:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 shutdown_connections 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c5bedb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5acf00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5edeff5700 0 client.15450 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3144687259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 >> 172.21.15.5:0/3144687259 conn(0x55e25c46c630 msgr2=0x55e25c5a18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 shutdown_connections 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3144687259 wait complete. 2024-05-14T15:25:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.947+0000 7f5f0fd8bb80 20 client.15450 trim_cache size 0 max 16384 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 Processor -- start 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 -- start start 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b2d70 con 0x55e25c424e40 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1b0600 con 0x55e25c5a2e10 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1ae450 con 0x55e25c5a8d80 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5efaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5efaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44630/0 (socket says 172.21.15.5:44630) 2024-05-14T15:25:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.968+0000 7f5efaffd700 1 -- 172.21.15.5:0/3526677251 learned_addr learned my addr 172.21.15.5:0/3526677251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efa7fc700 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efaffd700 1 -- 172.21.15.5:0/3526677251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 msgr2=0x55e25c5acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efaffd700 1 -- 172.21.15.5:0/3526677251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 msgr2=0x55e25c5ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efaffd700 1 -- 172.21.15.5:0/3526677251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e25c597190 con 0x55e25c424e40 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5efaffd700 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a28d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00a010 tx=0x7f5eec021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a458c5d315951 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5ee0ff9700 1 -- 172.21.15.5:0/3526677251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec041070 con 0x55e25c424e40 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5ee0ff9700 1 -- 172.21.15.5:0/3526677251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec00b7d0 con 0x55e25c424e40 2024-05-14T15:25:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.969+0000 7f5ee0ff9700 1 -- 172.21.15.5:0/3526677251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec03c340 con 0x55e25c424e40 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3526677251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 msgr2=0x55e25c5a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a28d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00a010 tx=0x7f5eec021000 comp rx=0 tx=0).stop 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3526677251 shutdown_connections 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a8d80 0x55e25c5acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c424e40 0x55e25c5a28d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/3526677251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c5a2e10 0x55e25c5ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3526677251 >> 172.21.15.5:0/3526677251 conn(0x55e25c46c630 msgr2=0x55e25c597580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3526677251 shutdown_connections 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/3526677251 wait complete. 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 -- start start 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.970+0000 7f5f0fd8bb80 1 Processor -- start 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1aeac0 con 0x55e25c5a8d80 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1aff30 con 0x55e25c424e40 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc1af190 con 0x55e25c5a2e10 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5f0fd8bb80 10 client.0 _ll_register_callbacks cb 0x55e25c45c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44632/0 (socket says 172.21.15.5:44632) 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 -- 172.21.15.5:0/2690653356 learned_addr learned my addr 172.21.15.5:0/2690653356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efa7fc700 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 -- 172.21.15.5:0/2690653356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 msgr2=0x55e25c5c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 -- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 msgr2=0x55e25c5b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.971+0000 7f5efb7fe700 1 -- 172.21.15.5:0/2690653356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e25c5c4340 con 0x55e25c5a8d80 2024-05-14T15:25:08.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5efb7fe700 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5c3e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40a85d0 tx=0x7f5ef40094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd6ec7c7124c2c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5edeff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5edeff5700 1 -- 172.21.15.5:0/2690653356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef40a8d80 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec00b7d0 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5edeff5700 1 -- 172.21.15.5:0/2690653356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5ef40a49f0 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5edeff5700 1 -- 172.21.15.5:0/2690653356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ef40a4bf0 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 10 client.15456 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e25c626bf0 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5edeff5700 1 -- 172.21.15.5:0/2690653356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec00b7d0 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5edeff5700 1 -- 172.21.15.5:0/2690653356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5ef40df960 con 0x55e25c5a8d80 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 20 client.15456 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c629070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 4 client.15456 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:25:08.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.972+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e25c4cc3c0 con 0x55e25c5bc990 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5efaffd700 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c629070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5efaffd700 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c629070 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3601cff289f23d3 server_cookie=cc5ef0832887f0fd in_seq=0 out_seq=0 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5edeff5700 10 client.15456 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5edeff5700 1 -- 172.21.15.5:0/2690653356 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e25c4cc3c0 con 0x55e25c5bc990 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5edeff5700 10 client.15456 handle_command_reply: tid=0 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 client.15456 shutdown 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 msgr2=0x55e25c5c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5c3e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40a85d0 tx=0x7f5ef40094d0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 shutdown_connections 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e25c5bc990 0x55e25c629070 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e25c5a8d80 0x55e25c5c3e00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e25c424e40 0x55e25c5b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5edeff5700 0 client.15456 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 --2- 172.21.15.5:0/2690653356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e25c5a2e10 0x55e25c5c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.973+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 >> 172.21.15.5:0/2690653356 conn(0x55e25c46c630 msgr2=0x55e25c597580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:08.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 shutdown_connections 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f0fd8bb80 1 -- 172.21.15.5:0/2690653356 wait complete. 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f0fd8bb80 20 client.15456 trim_cache size 0 max 16384 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: "f26e5dc4-1f46-4410-b84f-ba30afe91b8d": { 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "f26e5dc4-1f46-4410-b84f-ba30afe91b8d", 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:25:08.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ee40b2b10 msgr2=0x7f5ee40b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ee40b2b10 0x7f5ee40b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0059d0 tx=0x7f5eec005960 comp rx=0 tx=0).stop 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 msgr2=0x7f5efc19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc19f790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4007ee0 tx=0x7f5ef400d3b0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 shutdown_connections 2024-05-14T15:25:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ee40b2b10 0x7f5ee40b5020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5efc109300 0x7f5efc19f790 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5efc102790 0x7f5efc19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 --2- 172.21.15.5:0/1280400844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5efc1030e0 0x7f5efc19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.974+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 >> 172.21.15.5:0/1280400844 conn(0x7f5efc0fe4e0 msgr2=0x7f5efc107140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.975+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 shutdown_connections 2024-05-14T15:25:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:08.975+0000 7f5f01377700 1 -- 172.21.15.5:0/1280400844 wait complete. 2024-05-14T15:25:09.744 INFO:tasks.cephfs.filesystem:scrub status for tag:f26e5dc4-1f46-4410-b84f-ba30afe91b8d - {'path': '/', 'tag': 'f26e5dc4-1f46-4410-b84f-ba30afe91b8d', 'options': 'recursive,force'} 2024-05-14T15:25:10.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:09 smithi144 ceph-mon[35650]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-14T15:25:10.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:09 smithi193 ceph-mon[36001]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-14T15:25:10.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:09 smithi005 ceph-mon[31559]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-14T15:25:12.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:11 smithi144 ceph-mon[35650]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-05-14T15:25:12.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:11 smithi193 ceph-mon[36001]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-05-14T15:25:12.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:11 smithi005 ceph-mon[31559]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-05-14T15:25:14.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:13 smithi144 ceph-mon[35650]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-14T15:25:14.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:13 smithi193 ceph-mon[36001]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-14T15:25:14.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:13 smithi005 ceph-mon[31559]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-14T15:25:16.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:15 smithi144 ceph-mon[35650]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-14T15:25:16.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:15 smithi193 ceph-mon[36001]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-14T15:25:16.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:15 smithi005 ceph-mon[31559]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-14T15:25:18.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:17 smithi144 ceph-mon[35650]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-14T15:25:18.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:17 smithi193 ceph-mon[36001]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-14T15:25:18.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:17 smithi005 ceph-mon[31559]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-14T15:25:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 -- 172.21.15.5:0/4238220882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 msgr2=0x7fdb6810f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4238220882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb6810f6b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ca0 tx=0x7fdb6002a8c0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 -- 172.21.15.5:0/4238220882 shutdown_connections 2024-05-14T15:25:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4238220882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb68109030 0x7fdb68102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4238220882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb6810f6b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4238220882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb68108b90 0x7fdb68102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 -- 172.21.15.5:0/4238220882 >> 172.21.15.5:0/4238220882 conn(0x7fdb680fe390 msgr2=0x7fdb681007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 -- 172.21.15.5:0/4238220882 shutdown_connections 2024-05-14T15:25:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.934+0000 7fdb6df38640 1 -- 172.21.15.5:0/4238220882 wait complete. 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.935+0000 7fdb6df38640 1 Processor -- start 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.935+0000 7fdb6df38640 1 -- start start 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb6df38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb681a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb6df38640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb68108b90 0x7fdb681a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb6df38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb68109030 0x7fdb6819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb6df38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68112d80 con 0x7fdb681032e0 2024-05-14T15:25:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb6df38640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68112c00 con 0x7fdb68108b90 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb6df38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68112f00 con 0x7fdb68109030 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb681a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb681a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59230/0 (socket says 172.21.15.5:59230) 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 -- 172.21.15.5:0/4268430295 learned_addr learned my addr 172.21.15.5:0/4268430295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb68108b90 0x7fdb681a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb67fff640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb68109030 0x7fdb6819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 -- 172.21.15.5:0/4268430295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb68109030 msgr2=0x7fdb6819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb68109030 0x7fdb6819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 -- 172.21.15.5:0/4268430295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb68108b90 msgr2=0x7fdb681a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb68108b90 0x7fdb681a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.936+0000 7fdb677fe640 1 -- 172.21.15.5:0/4268430295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6819ae20 con 0x7fdb681032e0 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.937+0000 7fdb677fe640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb681a0570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800e9c0 tx=0x7fdb5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.937+0000 7fdb64ff9640 1 -- 172.21.15.5:0/4268430295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800cd80 con 0x7fdb681032e0 2024-05-14T15:25:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.937+0000 7fdb64ff9640 1 -- 172.21.15.5:0/4268430295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb580054d0 con 0x7fdb681032e0 2024-05-14T15:25:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.937+0000 7fdb64ff9640 1 -- 172.21.15.5:0/4268430295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58010430 con 0x7fdb681032e0 2024-05-14T15:25:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.937+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6819b0b0 con 0x7fdb681032e0 2024-05-14T15:25:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.937+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681a7310 con 0x7fdb681032e0 2024-05-14T15:25:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.940+0000 7fdb74dec740 1 Processor -- start 2024-05-14T15:25:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 -- start start 2024-05-14T15:25:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb64ff9640 1 -- 172.21.15.5:0/4268430295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fdb58004fd0 con 0x7fdb681032e0 2024-05-14T15:25:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 0x5612de7d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7d8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5612de73d5d0 con 0x5612de40e2f0 2024-05-14T15:25:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612de6f6890 con 0x5612de4371b0 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb74dec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5612de739590 con 0x5612de4164d0 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7d8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 0x5612de7d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb66ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7d8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59234/0 (socket says 172.21.15.5:59234) 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 0x5612de7d7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:58094/0 (socket says 172.21.15.5:58094) 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.941+0000 7fdb677fe640 1 -- 172.21.15.5:0/327046203 learned_addr learned my addr 172.21.15.5:0/327046203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb677fe640 1 -- 172.21.15.5:0/327046203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 msgr2=0x5612de7d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb677fe640 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb677fe640 1 -- 172.21.15.5:0/327046203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 msgr2=0x5612de7d7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb677fe640 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 0x5612de7d7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb677fe640 1 -- 172.21.15.5:0/327046203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612de44f210 con 0x5612de40e2f0 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb67fff640 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 0x5612de7d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb64ff9640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb500b2a70 0x7fdb500b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb677fe640 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7d8010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb580e0410 tx=0x7fdb580eb320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2333549b714c7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb67fff640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb500b2a70 0x7fdb500b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb64ff9640 1 -- 172.21.15.5:0/4268430295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fdb580e30f0 con 0x7fdb681032e0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb4dffb640 1 -- 172.21.15.5:0/327046203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580a5c80 con 0x5612de40e2f0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb4dffb640 1 -- 172.21.15.5:0/327046203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb580046c0 con 0x5612de40e2f0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.942+0000 7fdb4dffb640 1 -- 172.21.15.5:0/327046203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580ab0f0 con 0x5612de40e2f0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 -- 172.21.15.5:0/327046203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 msgr2=0x5612de7d8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb67fff640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb500b2a70 0x7fdb500b4f80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5612de7d98d0 tx=0x7fdb600094a0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7d8010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb580e0410 tx=0x7fdb580eb320 comp rx=0 tx=0).stop 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 -- 172.21.15.5:0/327046203 shutdown_connections 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7d8010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 --2- 172.21.15.5:0/327046203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4371b0 0x5612de7d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 -- 172.21.15.5:0/327046203 >> 172.21.15.5:0/327046203 conn(0x5612de42b300 msgr2=0x5612de7d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 -- 172.21.15.5:0/327046203 shutdown_connections 2024-05-14T15:25:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.943+0000 7fdb74dec740 1 -- 172.21.15.5:0/327046203 wait complete. 2024-05-14T15:25:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 -- start start 2024-05-14T15:25:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 Processor -- start 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 0x5612de7e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de7ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5612de7e9120 con 0x5612de4164d0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612de6fa370 con 0x5612de40e2f0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5612de7e92a0 con 0x5612de4371b0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.944+0000 7fdb74dec740 10 client.0 _ll_register_callbacks cb 0x5612de44ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 0x5612de7e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 0x5612de7e6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59242/0 (socket says 172.21.15.5:59242) 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 -- 172.21.15.5:0/2317707384 learned_addr learned my addr 172.21.15.5:0/2317707384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb67fff640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de7ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 -- 172.21.15.5:0/2317707384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 msgr2=0x5612de7ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de7ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 -- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 msgr2=0x5612de7e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 -- 172.21.15.5:0/2317707384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612de51b620 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb67fff640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de7ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb677fe640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 0x5612de7e6bd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c012700 tx=0x7fdb5c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e933e9e0fb1d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb46ffd640 1 -- 172.21.15.5:0/2317707384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c004030 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb46ffd640 1 -- 172.21.15.5:0/2317707384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fdb5c0041d0 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.945+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612de74b1e0 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb46ffd640 1 -- 172.21.15.5:0/2317707384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb5c025050 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 10 client.15468 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612de7f55d0 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb46ffd640 1 -- 172.21.15.5:0/2317707384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5612de74b1e0 con 0x5612de4164d0 2024-05-14T15:25:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb46ffd640 1 -- 172.21.15.5:0/2317707384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdb5c003940 con 0x5612de4164d0 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 20 client.15468 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3f5d80 0x5612de7f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 4 client.15468 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612de745050 con 0x5612de3f5d80 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.946+0000 7fdb677fe640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3f5d80 0x5612de7f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.947+0000 7fdb46ffd640 10 client.15468 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.947+0000 7fdb677fe640 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3f5d80 0x5612de7f8aa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343b622ffc69aac0 server_cookie=51b9bb62aa94ec66 in_seq=0 out_seq=0 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.947+0000 7fdb46ffd640 1 -- 172.21.15.5:0/2317707384 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612de745050 con 0x5612de3f5d80 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.947+0000 7fdb46ffd640 10 client.15468 handle_command_reply: tid=0 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.947+0000 7fdb74dec740 1 client.15468 shutdown 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 msgr2=0x5612de7e6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 0x5612de7e6bd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c012700 tx=0x7fdb5c012bc0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 shutdown_connections 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3f5d80 0x5612de7f8aa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de7ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4164d0 0x5612de7e6bd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb46ffd640 0 client.15468 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 --2- 172.21.15.5:0/2317707384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 >> 172.21.15.5:0/2317707384 conn(0x5612de42b300 msgr2=0x5612de7d7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 shutdown_connections 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 1 -- 172.21.15.5:0/2317707384 wait complete. 2024-05-14T15:25:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.948+0000 7fdb74dec740 20 client.15468 trim_cache size 0 max 16384 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 Processor -- start 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 -- start start 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7ec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 0x5612de7fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681b1ae0 con 0x5612de4371b0 2024-05-14T15:25:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681af3b0 con 0x5612de40e2f0 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.963+0000 7fdb74dec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681ad200 con 0x5612de4164d0 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 0x5612de7fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 0x5612de7fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59250/0 (socket says 172.21.15.5:59250) 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 -- 172.21.15.5:0/134546496 learned_addr learned my addr 172.21.15.5:0/134546496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb677fe640 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7ec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 -- 172.21.15.5:0/134546496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 msgr2=0x5612de7eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 -- 172.21.15.5:0/134546496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 msgr2=0x5612de7ec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7ec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 -- 172.21.15.5:0/134546496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612de7e49d0 con 0x5612de4371b0 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb677fe640 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7ec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb67fff640 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 0x5612de7fe260 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdb60035000 tx=0x7fdb60041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe09546db4ea8883 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb4d7fa640 1 -- 172.21.15.5:0/134546496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60031e20 con 0x5612de4371b0 2024-05-14T15:25:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb4d7fa640 1 -- 172.21.15.5:0/134546496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb60036050 con 0x5612de4371b0 2024-05-14T15:25:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.964+0000 7fdb4d7fa640 1 -- 172.21.15.5:0/134546496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004740 con 0x5612de4371b0 2024-05-14T15:25:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 -- 172.21.15.5:0/134546496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 msgr2=0x5612de7fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 0x5612de7fe260 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdb60035000 tx=0x7fdb60041000 comp rx=0 tx=0).stop 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 -- 172.21.15.5:0/134546496 shutdown_connections 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4164d0 0x5612de7eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de4371b0 0x5612de7fe260 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 --2- 172.21.15.5:0/134546496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de40e2f0 0x5612de7ec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 -- 172.21.15.5:0/134546496 >> 172.21.15.5:0/134546496 conn(0x5612de42b300 msgr2=0x5612de7e4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 -- 172.21.15.5:0/134546496 shutdown_connections 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.965+0000 7fdb74dec740 1 -- 172.21.15.5:0/134546496 wait complete. 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 -- start start 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 Processor -- start 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4164d0 0x5612de7ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de85ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681ad870 con 0x5612de40e2f0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681aece0 con 0x5612de4164d0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681adf40 con 0x5612de4371b0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb74dec740 10 client.0 _ll_register_callbacks cb 0x7fdb5c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59260/0 (socket says 172.21.15.5:59260) 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb677fe640 1 -- 172.21.15.5:0/4041173905 learned_addr learned my addr 172.21.15.5:0/4041173905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4164d0 0x5612de7ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.966+0000 7fdb67fff640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de85ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb677fe640 1 -- 172.21.15.5:0/4041173905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 msgr2=0x5612de85ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb677fe640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de85ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb677fe640 1 -- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4164d0 msgr2=0x5612de7ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb677fe640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4164d0 0x5612de7ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb677fe640 1 -- 172.21.15.5:0/4041173905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612de85f340 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb677fe640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7eddc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800c6a0 tx=0x7fdb5800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949839cd599741de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb46ffd640 1 -- 172.21.15.5:0/4041173905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb580a9530 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb46ffd640 1 -- 172.21.15.5:0/4041173905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fdb580aa020 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb60036050 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb46ffd640 1 -- 172.21.15.5:0/4041173905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb580a9710 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb74dec740 10 client.15474 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612de8605a0 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb46ffd640 1 -- 172.21.15.5:0/4041173905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb60036050 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.967+0000 7fdb46ffd640 1 -- 172.21.15.5:0/4041173905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdb580e0bd0 con 0x5612de40e2f0 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb74dec740 10 client.15474 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb74dec740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb74dec740 20 client.15474 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb74dec740 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3cb1c0 0x5612de8628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb74dec740 4 client.15474 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:25:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612de6c05f0 con 0x5612de3cb1c0 2024-05-14T15:25:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3cb1c0 0x5612de8628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb46ffd640 10 client.15474 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.968+0000 7fdb66ffd640 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3cb1c0 0x5612de8628f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f56edf90b01bef server_cookie=10cea5ae9a85063 in_seq=0 out_seq=0 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.982+0000 7fdb46ffd640 1 -- 172.21.15.5:0/4041173905 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612de6c05f0 con 0x5612de3cb1c0 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.982+0000 7fdb46ffd640 10 client.15474 handle_command_reply: tid=0 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.982+0000 7fdb74dec740 1 client.15474 shutdown 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 msgr2=0x5612de7eddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7eddc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800c6a0 tx=0x7fdb5800e940 comp rx=0 tx=0).stop 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 shutdown_connections 2024-05-14T15:25:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5612de3cb1c0 0x5612de8628f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612de4371b0 0x5612de85ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612de4164d0 0x5612de7ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb46ffd640 0 client.15474 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 --2- 172.21.15.5:0/4041173905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612de40e2f0 0x5612de7eddc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 >> 172.21.15.5:0/4041173905 conn(0x5612de42b300 msgr2=0x5612de7d9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 shutdown_connections 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 1 -- 172.21.15.5:0/4041173905 wait complete. 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.983+0000 7fdb74dec740 20 client.15474 trim_cache size 0 max 16384 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:25:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:25:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:25:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:25:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb500b2a70 msgr2=0x7fdb500b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb500b2a70 0x7fdb500b4f80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5612de7d98d0 tx=0x7fdb600094a0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 msgr2=0x7fdb681a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb681a0570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800e9c0 tx=0x7fdb5800ee80 comp rx=0 tx=0).stop 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 shutdown_connections 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb500b2a70 0x7fdb500b4f80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb68109030 0x7fdb6819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb68108b90 0x7fdb681a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 --2- 172.21.15.5:0/4268430295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb681032e0 0x7fdb681a0570 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 >> 172.21.15.5:0/4268430295 conn(0x7fdb680fe390 msgr2=0x7fdb68100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 shutdown_connections 2024-05-14T15:25:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:19.984+0000 7fdb6df38640 1 -- 172.21.15.5:0/4268430295 wait complete. 2024-05-14T15:25:20.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:19 smithi144 ceph-mon[35650]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-05-14T15:25:20.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:19 smithi193 ceph-mon[36001]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-05-14T15:25:20.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:19 smithi005 ceph-mon[31559]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 40 KiB/s wr, 13 op/s 2024-05-14T15:25:21.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:20 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:25:21.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:20 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:25:21.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:20 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:25:22.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:21 smithi144 ceph-mon[35650]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-14T15:25:22.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:21 smithi193 ceph-mon[36001]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-14T15:25:22.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:21 smithi005 ceph-mon[31559]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-14T15:25:24.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:23 smithi144 ceph-mon[35650]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-05-14T15:25:24.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:23 smithi193 ceph-mon[36001]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-05-14T15:25:24.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:23 smithi005 ceph-mon[31559]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-05-14T15:25:26.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:25 smithi144 ceph-mon[35650]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T15:25:26.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:25 smithi193 ceph-mon[36001]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T15:25:26.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:25 smithi005 ceph-mon[31559]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T15:25:28.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:27 smithi144 ceph-mon[35650]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:28.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:27 smithi193 ceph-mon[36001]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:28.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:27 smithi005 ceph-mon[31559]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:30.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:29 smithi144 ceph-mon[35650]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:30.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:29 smithi193 ceph-mon[36001]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:30.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:29 smithi005 ceph-mon[31559]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:32.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:31 smithi144 ceph-mon[35650]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T15:25:32.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:31 smithi193 ceph-mon[36001]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T15:25:32.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:31 smithi005 ceph-mon[31559]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 -- 172.21.15.5:0/1263106016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 msgr2=0x7fcb8c1026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 --2- 172.21.15.5:0/1263106016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c1026d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009900 tx=0x7fcb7402a410 comp rx=0 tx=0).stop 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 -- 172.21.15.5:0/1263106016 shutdown_connections 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 --2- 172.21.15.5:0/1263106016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c10e660 0x7fcb8c110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 --2- 172.21.15.5:0/1263106016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c1026d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 --2- 172.21.15.5:0/1263106016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c108c10 0x7fcb8c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 -- 172.21.15.5:0/1263106016 >> 172.21.15.5:0/1263106016 conn(0x7fcb8c0fd9d0 msgr2=0x7fcb8c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 -- 172.21.15.5:0/1263106016 shutdown_connections 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.098+0000 7fcb915c5640 1 -- 172.21.15.5:0/1263106016 wait complete. 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.099+0000 7fcb915c5640 1 Processor -- start 2024-05-14T15:25:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.099+0000 7fcb915c5640 1 -- start start 2024-05-14T15:25:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb915c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb915c5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c108c10 0x7fcb8c1034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb915c5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c10e660 0x7fcb8c1039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb915c5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c113f40 con 0x7fcb8c1022c0 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb915c5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c113dc0 con 0x7fcb8c108c10 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb915c5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1140c0 con 0x7fcb8c10e660 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46154/0 (socket says 172.21.15.5:46154) 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 -- 172.21.15.5:0/4073620886 learned_addr learned my addr 172.21.15.5:0/4073620886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c108c10 0x7fcb8c1034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c10e660 0x7fcb8c1039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 -- 172.21.15.5:0/4073620886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c10e660 msgr2=0x7fcb8c1039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c10e660 0x7fcb8c1039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 -- 172.21.15.5:0/4073620886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c108c10 msgr2=0x7fcb8c1034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c108c10 0x7fcb8c1034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 -- 172.21.15.5:0/4073620886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c1a8c80 con 0x7fcb8c1022c0 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.100+0000 7fcb8affd640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c102f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00ea80 tx=0x7fcb7c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.101+0000 7fcb83fff640 1 -- 172.21.15.5:0/4073620886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00ce40 con 0x7fcb8c1022c0 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.101+0000 7fcb83fff640 1 -- 172.21.15.5:0/4073620886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7c011920 con 0x7fcb8c1022c0 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.101+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c1a8f70 con 0x7fcb8c1022c0 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.101+0000 7fcb83fff640 1 -- 172.21.15.5:0/4073620886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c01aa20 con 0x7fcb8c1022c0 2024-05-14T15:25:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.101+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c1a93d0 con 0x7fcb8c1022c0 2024-05-14T15:25:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 Processor -- start 2024-05-14T15:25:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb83fff640 1 -- 172.21.15.5:0/4073620886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fcb7c01abc0 con 0x7fcb8c1022c0 2024-05-14T15:25:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 -- start start 2024-05-14T15:25:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556dbdba55d0 con 0x556dbd87e4d0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556dbdb5e890 con 0x556dbd89f1b0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb98476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556dbdba1590 con 0x556dbd8762f0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb8a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb8a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc40660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46162/0 (socket says 172.21.15.5:46162) 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.104+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/3752544681 learned_addr learned my addr 172.21.15.5:0/3752544681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8affd640 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/3752544681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 msgr2=0x556dbdc40120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc40120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/3752544681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 msgr2=0x556dbdc3fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc3fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/3752544681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbd8b7210 con 0x556dbd87e4d0 2024-05-14T15:25:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb83fff640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb780b2af0 0x7fcb780b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc40660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009900 tx=0x7fcb74032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6c60048077d8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb780b2af0 0x7fcb780b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb83fff640 1 -- 172.21.15.5:0/4073620886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fcb7c0e30d0 con 0x7fcb8c1022c0 2024-05-14T15:25:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb817fa640 1 -- 172.21.15.5:0/3752544681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74040070 con 0x556dbd87e4d0 2024-05-14T15:25:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb817fa640 1 -- 172.21.15.5:0/3752544681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb74032e50 con 0x556dbd87e4d0 2024-05-14T15:25:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.105+0000 7fcb817fa640 1 -- 172.21.15.5:0/3752544681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74035040 con 0x556dbd87e4d0 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 -- 172.21.15.5:0/3752544681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 msgr2=0x556dbdc40660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb780b2af0 0x7fcb780b5000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x556dbdc3cd00 tx=0x7fcb8400b2e0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc40660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009900 tx=0x7fcb74032c60 comp rx=0 tx=0).stop 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 -- 172.21.15.5:0/3752544681 shutdown_connections 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc40120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc40660 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 --2- 172.21.15.5:0/3752544681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 -- 172.21.15.5:0/3752544681 >> 172.21.15.5:0/3752544681 conn(0x556dbd893300 msgr2=0x556dbdc3c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 -- 172.21.15.5:0/3752544681 shutdown_connections 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.106+0000 7fcb98476740 1 -- 172.21.15.5:0/3752544681 wait complete. 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 -- start start 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 Processor -- start 2024-05-14T15:25:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc4a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd87e4d0 0x556dbdc4a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc56160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556dbdc4ccb0 con 0x556dbd8762f0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556dbdb62370 con 0x556dbd89f1b0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556dbdc4ce30 con 0x556dbd87e4d0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb98476740 10 client.0 _ll_register_callbacks cb 0x556dbd8b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.107+0000 7fcb8affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc4a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc4a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46178/0 (socket says 172.21.15.5:46178) 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 -- 172.21.15.5:0/2490627219 learned_addr learned my addr 172.21.15.5:0/2490627219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd87e4d0 0x556dbdc4a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc56160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 -- 172.21.15.5:0/2490627219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd87e4d0 msgr2=0x556dbdc4a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd87e4d0 0x556dbdc4a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 -- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 msgr2=0x556dbdc56160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc56160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 -- 172.21.15.5:0/2490627219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbd983620 con 0x556dbd8762f0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd87e4d0 0x556dbdc4a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc4a1c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0a9aa0 tx=0x7fcb7c0a9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205cb6efcc6a1439 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb727fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb727fc640 1 -- 172.21.15.5:0/2490627219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0a98b0 con 0x556dbd8762f0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb727fc640 1 -- 172.21.15.5:0/2490627219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fcb7c0e0bb0 con 0x556dbd8762f0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb74032e50 con 0x556dbd8762f0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb727fc640 1 -- 172.21.15.5:0/2490627219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb7c0c9050 con 0x556dbd8762f0 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb98476740 10 client.15486 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.108+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556dbdc59070 con 0x556dbd8762f0 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb727fc640 1 -- 172.21.15.5:0/2490627219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb74032e50 con 0x556dbd8762f0 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb727fc640 1 -- 172.21.15.5:0/2490627219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fcb7c022030 con 0x556dbd8762f0 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb98476740 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb98476740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb98476740 20 client.15486 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb98476740 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd85dd80 0x556dbdc5c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb98476740 4 client.15486 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dbdbb31e0 con 0x556dbd85dd80 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.109+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd85dd80 0x556dbdc5c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.110+0000 7fcb727fc640 10 client.15486 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.110+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd85dd80 0x556dbdc5c5a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee9cb9b12c6d568 server_cookie=393ffcc7cada27d6 in_seq=0 out_seq=0 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.110+0000 7fcb727fc640 1 -- 172.21.15.5:0/2490627219 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556dbdbb31e0 con 0x556dbd85dd80 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.110+0000 7fcb727fc640 10 client.15486 handle_command_reply: tid=0 2024-05-14T15:25:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.110+0000 7fcb98476740 1 client.15486 shutdown 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 msgr2=0x556dbdc4a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc4a1c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0a9aa0 tx=0x7fcb7c0a9b70 comp rx=0 tx=0).stop 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 shutdown_connections 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd85dd80 0x556dbdc5c5a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd87e4d0 0x556dbdc4a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd89f1b0 0x556dbdc56160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb727fc640 0 client.15486 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 --2- 172.21.15.5:0/2490627219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc4a1c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 >> 172.21.15.5:0/2490627219 conn(0x556dbd893300 msgr2=0x556dbdc41900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 shutdown_connections 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 1 -- 172.21.15.5:0/2490627219 wait complete. 2024-05-14T15:25:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.111+0000 7fcb98476740 20 client.15486 trim_cache size 0 max 16384 2024-05-14T15:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.126+0000 7fcb98476740 1 Processor -- start 2024-05-14T15:25:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.126+0000 7fcb98476740 1 -- start start 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb98476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd85dd80 0x556dbdc41fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb98476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb98476740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd87e4d0 0x556dbdc622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb98476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b5d50 con 0x556dbd8762f0 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb98476740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b3620 con 0x556dbd87e4d0 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb98476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b1470 con 0x556dbd85dd80 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc61da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46182/0 (socket says 172.21.15.5:46182) 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/2031808912 learned_addr learned my addr 172.21.15.5:0/2031808912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd85dd80 0x556dbdc41fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd87e4d0 0x556dbdc622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/2031808912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd85dd80 msgr2=0x556dbdc41fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd85dd80 0x556dbdc41fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/2031808912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd87e4d0 msgr2=0x556dbdc622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd87e4d0 0x556dbdc622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8a7fc640 1 -- 172.21.15.5:0/2031808912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dbdc3cf80 con 0x556dbd8762f0 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.127+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd87e4d0 0x556dbdc622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.128+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc61da0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb740051f0 tx=0x7fcb74005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52846bc41d6dcde5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.128+0000 7fcb80ff9640 1 -- 172.21.15.5:0/2031808912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74035040 con 0x556dbd8762f0 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.128+0000 7fcb80ff9640 1 -- 172.21.15.5:0/2031808912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb74054020 con 0x556dbd8762f0 2024-05-14T15:25:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.128+0000 7fcb80ff9640 1 -- 172.21.15.5:0/2031808912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74040040 con 0x556dbd8762f0 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 -- 172.21.15.5:0/2031808912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 msgr2=0x556dbdc61da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc61da0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb740051f0 tx=0x7fcb74005220 comp rx=0 tx=0).stop 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 -- 172.21.15.5:0/2031808912 shutdown_connections 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd87e4d0 0x556dbdc622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd8762f0 0x556dbdc61da0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 --2- 172.21.15.5:0/2031808912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd85dd80 0x556dbdc41fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 -- 172.21.15.5:0/2031808912 >> 172.21.15.5:0/2031808912 conn(0x556dbd893300 msgr2=0x556dbdc41890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 -- 172.21.15.5:0/2031808912 shutdown_connections 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 -- 172.21.15.5:0/2031808912 wait complete. 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 -- start start 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.129+0000 7fcb98476740 1 Processor -- start 2024-05-14T15:25:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd85dd80 0x556dbdc52060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc4d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b1ae0 con 0x556dbd87e4d0 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b2f50 con 0x556dbd85dd80 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b21b0 con 0x556dbd8762f0 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb98476740 10 client.0 _ll_register_callbacks cb 0x7fcb7c01a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc4d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc4d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46188/0 (socket says 172.21.15.5:46188) 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 -- 172.21.15.5:0/2221457364 learned_addr learned my addr 172.21.15.5:0/2221457364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd85dd80 0x556dbdc52060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8a7fc640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 -- 172.21.15.5:0/2221457364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 msgr2=0x556dbdc525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 -- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd85dd80 msgr2=0x556dbdc52060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd85dd80 0x556dbdc52060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 -- 172.21.15.5:0/2221457364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dbdc4db50 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd85dd80 0x556dbdc52060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb8b7fe640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc4d490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb84003d00 tx=0x7fcb84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559036ea07cafc01 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb727fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.130+0000 7fcb727fc640 1 -- 172.21.15.5:0/2221457364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb840200d0 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb727fc640 1 -- 172.21.15.5:0/2221457364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fcb84017030 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb74054020 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb727fc640 1 -- 172.21.15.5:0/2221457364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb84033050 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 10 client.15492 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556dbdcc42e0 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb727fc640 1 -- 172.21.15.5:0/2221457364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb74054020 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb727fc640 1 -- 172.21.15.5:0/2221457364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fcb84004b00 con 0x556dbd87e4d0 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 10 client.15492 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 20 client.15492 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd89f1b0 0x556dbdcc6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 4 client.15492 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556dbdbad050 con 0x556dbd89f1b0 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.131+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd89f1b0 0x556dbdcc6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.132+0000 7fcb8affd640 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd89f1b0 0x556dbdcc6aa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf0bc8afa15123f server_cookie=40576c1f559dfdb in_seq=0 out_seq=0 2024-05-14T15:25:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.132+0000 7fcb727fc640 10 client.15492 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb727fc640 1 -- 172.21.15.5:0/2221457364 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556dbdbad050 con 0x556dbd89f1b0 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb727fc640 10 client.15492 handle_command_reply: tid=0 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 client.15492 shutdown 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 msgr2=0x556dbdc4d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc4d490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcb84003d00 tx=0x7fcb84004730 comp rx=0 tx=0).stop 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 shutdown_connections 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556dbd89f1b0 0x556dbdcc6aa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556dbd8762f0 0x556dbdc525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556dbd87e4d0 0x556dbdc4d490 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb727fc640 0 client.15492 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 --2- 172.21.15.5:0/2221457364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556dbd85dd80 0x556dbdc52060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 >> 172.21.15.5:0/2221457364 conn(0x556dbd893300 msgr2=0x556dbdc3f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 shutdown_connections 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 1 -- 172.21.15.5:0/2221457364 wait complete. 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.140+0000 7fcb98476740 20 client.15492 trim_cache size 0 max 16384 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:25:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:25:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:25:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb780b2af0 msgr2=0x7fcb780b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb780b2af0 0x7fcb780b5000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x556dbdc3cd00 tx=0x7fcb8400b2e0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 msgr2=0x7fcb8c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c102f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00ea80 tx=0x7fcb7c00ef40 comp rx=0 tx=0).stop 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 shutdown_connections 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb780b2af0 0x7fcb780b5000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c10e660 0x7fcb8c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c108c10 0x7fcb8c1034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 --2- 172.21.15.5:0/4073620886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c1022c0 0x7fcb8c102f60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 >> 172.21.15.5:0/4073620886 conn(0x7fcb8c0fd9d0 msgr2=0x7fcb8c10fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 shutdown_connections 2024-05-14T15:25:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:33.141+0000 7fcb915c5640 1 -- 172.21.15.5:0/4073620886 wait complete. 2024-05-14T15:25:34.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:33 smithi144 ceph-mon[35650]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:34.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:33 smithi193 ceph-mon[36001]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:34.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:33 smithi005 ceph-mon[31559]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-14T15:25:36.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:35 smithi144 ceph-mon[35650]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:25:36.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:35 smithi193 ceph-mon[36001]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:25:36.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:35 smithi005 ceph-mon[31559]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:25:38.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:37 smithi144 ceph-mon[35650]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:25:38.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:37 smithi193 ceph-mon[36001]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:25:38.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:37 smithi005 ceph-mon[31559]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:25:39.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:25:40.012 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:40.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:39 smithi144 ceph-mon[35650]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:40.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:39 smithi193 ceph-mon[36001]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:40.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:39 smithi005 ceph-mon[31559]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 -- 172.21.15.5:0/2863809502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 msgr2=0x7fd744103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2863809502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd744103700 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd7400099c0 tx=0x7fd740009cd0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 -- 172.21.15.5:0/2863809502 shutdown_connections 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2863809502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 0x7fd74410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2863809502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd744103700 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2863809502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 0x7fd744108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 -- 172.21.15.5:0/2863809502 >> 172.21.15.5:0/2863809502 conn(0x7fd744078b50 msgr2=0x7fd744078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 -- 172.21.15.5:0/2863809502 shutdown_connections 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.841+0000 7fd749ba1700 1 -- 172.21.15.5:0/2863809502 wait complete. 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 Processor -- start 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 -- start start 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd7441a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 0x7fd7441a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 0x7fd74419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441142f0 con 0x7fd7441032f0 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd744114170 con 0x7fd744108570 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd749ba1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd744114470 con 0x7fd744107c20 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd748b9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd7441a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd748b9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd7441a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47764/0 (socket says 172.21.15.5:47764) 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd748b9f700 1 -- 172.21.15.5:0/2426286779 learned_addr learned my addr 172.21.15.5:0/2426286779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd73bfff700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 0x7fd7441a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.842+0000 7fd7493a0700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 0x7fd74419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd748b9f700 1 -- 172.21.15.5:0/2426286779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 msgr2=0x7fd7441a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd748b9f700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 0x7fd7441a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd748b9f700 1 -- 172.21.15.5:0/2426286779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 msgr2=0x7fd74419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd748b9f700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 0x7fd74419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd748b9f700 1 -- 172.21.15.5:0/2426286779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74419b070 con 0x7fd7441032f0 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd73bfff700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 0x7fd7441a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd7493a0700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 0x7fd74419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd748b9f700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd7441a0810 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b580 tx=0x7fd7400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd739ffb700 1 -- 172.21.15.5:0/2426286779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740027070 con 0x7fd7441032f0 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd739ffb700 1 -- 172.21.15.5:0/2426286779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd740004020 con 0x7fd7441032f0 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74419b300 con 0x7fd7441032f0 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd739ffb700 1 -- 172.21.15.5:0/2426286779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740022e30 con 0x7fd7441032f0 2024-05-14T15:25:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.843+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74419b7e0 con 0x7fd7441032f0 2024-05-14T15:25:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.846+0000 7fd739ffb700 1 -- 172.21.15.5:0/2426286779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fd740022590 con 0x7fd7441032f0 2024-05-14T15:25:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.847+0000 7fd739ffb700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd72c0b2b10 0x7fd72c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.847+0000 7fd73bfff700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd72c0b2b10 0x7fd72c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.847+0000 7fd739ffb700 1 -- 172.21.15.5:0/2426286779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fd7400e39b0 con 0x7fd7441032f0 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.847+0000 7fd759817b80 1 Processor -- start 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.847+0000 7fd759817b80 1 -- start start 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.847+0000 7fd73bfff700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd72c0b2b10 0x7fd72c0b5020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd734005fd0 tx=0x7fd734005e20 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd759817b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fbff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd759817b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0440 0x561955fc9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd759817b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd759817b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561955ee1940 con 0x561955e41e40 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd759817b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561955e46da0 con 0x561955fc0440 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd759817b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561955edd900 con 0x561955fc0850 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fbff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fbff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47770/0 (socket says 172.21.15.5:47770) 2024-05-14T15:25:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 -- 172.21.15.5:0/1200614295 learned_addr learned my addr 172.21.15.5:0/1200614295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd73bfff700 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0440 0x561955fc9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd7493a0700 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 -- 172.21.15.5:0/1200614295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 msgr2=0x561955fcbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 -- 172.21.15.5:0/1200614295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0440 msgr2=0x561955fc9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0440 0x561955fc9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 -- 172.21.15.5:0/1200614295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561955e2e8d0 con 0x561955e41e40 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd73bfff700 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0440 0x561955fc9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.848+0000 7fd748b9f700 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fbff00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd7400aa410 tx=0x7fd740009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5e00abab86a4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.849+0000 7fd72affd700 1 -- 172.21.15.5:0/1200614295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7400922e0 con 0x561955e41e40 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.849+0000 7fd72affd700 1 -- 172.21.15.5:0/1200614295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7400aae10 con 0x561955e41e40 2024-05-14T15:25:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.849+0000 7fd72affd700 1 -- 172.21.15.5:0/1200614295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7400b32b0 con 0x561955e41e40 2024-05-14T15:25:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.849+0000 7fd759817b80 1 -- 172.21.15.5:0/1200614295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 msgr2=0x561955fbff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.849+0000 7fd759817b80 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fbff00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd7400aa410 tx=0x7fd740009690 comp rx=0 tx=0).stop 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 -- 172.21.15.5:0/1200614295 shutdown_connections 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fbff00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 --2- 172.21.15.5:0/1200614295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0440 0x561955fc9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 -- 172.21.15.5:0/1200614295 >> 172.21.15.5:0/1200614295 conn(0x561955e89630 msgr2=0x561955fbacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 -- 172.21.15.5:0/1200614295 shutdown_connections 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 -- 172.21.15.5:0/1200614295 wait complete. 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 -- start start 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.850+0000 7fd759817b80 1 Processor -- start 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0440 0x561955fc9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0850 0x561955fca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561955ee1940 con 0x561955e41e40 2024-05-14T15:25:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561955e9aed0 con 0x561955fc0850 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561955fcc780 con 0x561955fc0440 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd759817b80 10 client.0 _ll_register_callbacks cb 0x561955e79950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fc9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47782/0 (socket says 172.21.15.5:47782) 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 -- 172.21.15.5:0/4066859721 learned_addr learned my addr 172.21.15.5:0/4066859721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd73bfff700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0440 0x561955fc9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd7493a0700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0850 0x561955fca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 -- 172.21.15.5:0/4066859721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0440 msgr2=0x561955fc9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0440 0x561955fc9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 -- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0850 msgr2=0x561955fca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0850 0x561955fca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 -- 172.21.15.5:0/4066859721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561955e98760 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd748b9f700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fc9760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd7400e0cd0 tx=0x7fd7400aa4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b78acb4d142f990 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd723fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.851+0000 7fd723fff700 1 -- 172.21.15.5:0/4066859721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7400c4060 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd723fff700 1 -- 172.21.15.5:0/4066859721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fd7400b3280 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7400aae10 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd723fff700 1 -- 172.21.15.5:0/4066859721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7400aa710 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 10 client.15504 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561955fd7420 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd723fff700 1 -- 172.21.15.5:0/4066859721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74006c030 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd723fff700 1 -- 172.21.15.5:0/4066859721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd7400a9ac0 con 0x561955e41e40 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 20 client.15504 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561955fd9720 0x561955fdbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 4 client.15504 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.852+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561955eef320 con 0x561955fd9720 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.853+0000 7fd73bfff700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561955fd9720 0x561955fdbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.853+0000 7fd73bfff700 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561955fd9720 0x561955fdbb40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83865f15c7b7a96 server_cookie=8d79055bd2bbfd41 in_seq=0 out_seq=0 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.853+0000 7fd723fff700 10 client.15504 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd723fff700 1 -- 172.21.15.5:0/4066859721 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561955eef320 con 0x561955fd9720 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd723fff700 10 client.15504 handle_command_reply: tid=0 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 client.15504 shutdown 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 msgr2=0x561955fc9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fc9760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd7400e0cd0 tx=0x7fd7400aa4e0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 shutdown_connections 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561955fd9720 0x561955fdbb40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955fc0850 0x561955fca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955e41e40 0x561955fc9760 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 --2- 172.21.15.5:0/4066859721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0440 0x561955fc9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 >> 172.21.15.5:0/4066859721 conn(0x561955e89630 msgr2=0x561955fbe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd723fff700 0 client.15504 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.854+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 shutdown_connections 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.855+0000 7fd759817b80 1 -- 172.21.15.5:0/4066859721 wait complete. 2024-05-14T15:25:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.855+0000 7fd759817b80 20 client.15504 trim_cache size 0 max 16384 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 Processor -- start 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 -- start start 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fbe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 0x561955fc9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441b2bd0 con 0x561955fc0440 2024-05-14T15:25:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441b0460 con 0x561955e41e40 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd759817b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441ae2b0 con 0x561955fc0850 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd73bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 0x561955fc9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd73bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 0x561955fc9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47784/0 (socket says 172.21.15.5:47784) 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd73bfff700 1 -- 172.21.15.5:0/3690499579 learned_addr learned my addr 172.21.15.5:0/3690499579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.875+0000 7fd7493a0700 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd748b9f700 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fbe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd73bfff700 1 -- 172.21.15.5:0/3690499579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 msgr2=0x561955fca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd73bfff700 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd73bfff700 1 -- 172.21.15.5:0/3690499579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 msgr2=0x561955fbe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd73bfff700 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fbe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd73bfff700 1 -- 172.21.15.5:0/3690499579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561955fc7cd0 con 0x561955fc0440 2024-05-14T15:25:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd73bfff700 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 0x561955fc9cd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd744100e30 tx=0x7fd734013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c027c151e38a62a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:41.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd72a7fc700 1 -- 172.21.15.5:0/3690499579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73401da10 con 0x561955fc0440 2024-05-14T15:25:41.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd72a7fc700 1 -- 172.21.15.5:0/3690499579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73401dbb0 con 0x561955fc0440 2024-05-14T15:25:41.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.876+0000 7fd72a7fc700 1 -- 172.21.15.5:0/3690499579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400f710 con 0x561955fc0440 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 -- 172.21.15.5:0/3690499579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 msgr2=0x561955fc9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 0x561955fc9cd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd744100e30 tx=0x7fd734013000 comp rx=0 tx=0).stop 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 -- 172.21.15.5:0/3690499579 shutdown_connections 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fbe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 --2- 172.21.15.5:0/3690499579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fc0440 0x561955fc9cd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 -- 172.21.15.5:0/3690499579 >> 172.21.15.5:0/3690499579 conn(0x561955e89630 msgr2=0x561955fbaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 -- 172.21.15.5:0/3690499579 shutdown_connections 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.877+0000 7fd759817b80 1 -- 172.21.15.5:0/3690499579 wait complete. 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 -- start start 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 Processor -- start 2024-05-14T15:25:41.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fd1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 0x561955fcd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441ae920 con 0x561955fcd570 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441afd90 con 0x561955e41e40 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7441aeff0 con 0x561955fc0850 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd759817b80 10 client.0 _ll_register_callbacks cb 0x561955e79950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd7493a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 0x561955fcd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd7493a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 0x561955fcd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47798/0 (socket says 172.21.15.5:47798) 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.878+0000 7fd7493a0700 1 -- 172.21.15.5:0/382252389 learned_addr learned my addr 172.21.15.5:0/382252389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd73bfff700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd748b9f700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fd1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd7493a0700 1 -- 172.21.15.5:0/382252389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 msgr2=0x561955fcd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd7493a0700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd7493a0700 1 -- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 msgr2=0x561955fd1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd7493a0700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fd1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd7493a0700 1 -- 172.21.15.5:0/382252389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561955fd2490 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd7493a0700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 0x561955fcd980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0078a0 tx=0x7fd73c007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a811b4e224b0a8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd723fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd723fff700 1 -- 172.21.15.5:0/382252389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c01c030 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd723fff700 1 -- 172.21.15.5:0/382252389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fd73c012040 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73401dbb0 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd723fff700 1 -- 172.21.15.5:0/382252389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd73c029050 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd759817b80 10 client.15510 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.879+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561956043ef0 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd723fff700 1 -- 172.21.15.5:0/382252389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73401dbb0 con 0x561955fcd570 2024-05-14T15:25:41.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd723fff700 1 -- 172.21.15.5:0/382252389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd73c0189c0 con 0x561955fcd570 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd759817b80 10 client.15510 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd759817b80 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd759817b80 20 client.15510 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd759817b80 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561956044650 0x561956046a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd759817b80 4 client.15510 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561955ee93c0 con 0x561956044650 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd748b9f700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561956044650 0x561956046a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd748b9f700 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561956044650 0x561956046a70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ebba9be2a21b2e server_cookie=8d5bfe076f97a136 in_seq=0 out_seq=0 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.880+0000 7fd723fff700 10 client.15510 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd723fff700 1 -- 172.21.15.5:0/382252389 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561955ee93c0 con 0x561956044650 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd723fff700 10 client.15510 handle_command_reply: tid=0 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 client.15510 shutdown 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 msgr2=0x561955fcd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 0x561955fcd980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0078a0 tx=0x7fd73c007bb0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 shutdown_connections 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561956044650 0x561956046a70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561955fcd570 0x561955fcd980 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561955e41e40 0x561955fd1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 --2- 172.21.15.5:0/382252389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561955fc0850 0x561955fcd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd723fff700 0 client.15510 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 >> 172.21.15.5:0/382252389 conn(0x561955e89630 msgr2=0x561955fbaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 shutdown_connections 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 1 -- 172.21.15.5:0/382252389 wait complete. 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.881+0000 7fd759817b80 20 client.15510 trim_cache size 0 max 16384 2024-05-14T15:25:41.888 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd72c0b2b10 msgr2=0x7fd72c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd72c0b2b10 0x7fd72c0b5020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd734005fd0 tx=0x7fd734005e20 comp rx=0 tx=0).stop 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 msgr2=0x7fd7441a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:41.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd7441a0810 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b580 tx=0x7fd7400047b0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 shutdown_connections 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd72c0b2b10 0x7fd72c0b5020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd744108570 0x7fd74419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7441032f0 0x7fd7441a0810 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 --2- 172.21.15.5:0/2426286779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd744107c20 0x7fd7441a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 >> 172.21.15.5:0/2426286779 conn(0x7fd744078b50 msgr2=0x7fd744101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 shutdown_connections 2024-05-14T15:25:41.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:41.882+0000 7fd749ba1700 1 -- 172.21.15.5:0/2426286779 wait complete. 2024-05-14T15:25:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:41 smithi144 ceph-mon[35650]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:41 smithi193 ceph-mon[36001]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:42.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:41 smithi005 ceph-mon[31559]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:42.599 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:25:42.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:25:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:44.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:43 smithi144 ceph-mon[35650]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:25:44.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:43 smithi193 ceph-mon[36001]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:25:44.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:43 smithi005 ceph-mon[31559]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:25:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 -- 172.21.15.5:0/186093237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814076e40 msgr2=0x7f3814077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 --2- 172.21.15.5:0/186093237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814076e40 0x7f3814077250 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f380400b0a0 tx=0x7f380400b3b0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 -- 172.21.15.5:0/186093237 shutdown_connections 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 --2- 172.21.15.5:0/186093237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814110b80 0x7f3814112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 --2- 172.21.15.5:0/186093237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814076e40 0x7f3814077250 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 --2- 172.21.15.5:0/186093237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814077790 0x7f3814110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 -- 172.21.15.5:0/186093237 >> 172.21.15.5:0/186093237 conn(0x7f38140ffe80 msgr2=0x7f38141022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 -- 172.21.15.5:0/186093237 shutdown_connections 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.621+0000 7f381c095700 1 -- 172.21.15.5:0/186093237 wait complete. 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 Processor -- start 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 -- start start 2024-05-14T15:25:44.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814076e40 0x7f381419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814077790 0x7f381419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 0x7f38141a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3814116670 con 0x7f3814110b80 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.622+0000 7f381c095700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38141164f0 con 0x7f3814076e40 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381c095700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f38141167f0 con 0x7f3814077790 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 0x7f38141a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 0x7f38141a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47812/0 (socket says 172.21.15.5:47812) 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 -- 172.21.15.5:0/3976289295 learned_addr learned my addr 172.21.15.5:0/3976289295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f3819630700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814077790 0x7f381419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f3819e31700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814076e40 0x7f381419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 -- 172.21.15.5:0/3976289295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814077790 msgr2=0x7f381419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814077790 0x7f381419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 -- 172.21.15.5:0/3976289295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814076e40 msgr2=0x7f381419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814076e40 0x7f381419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 -- 172.21.15.5:0/3976289295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38141a1dc0 con 0x7f3814110b80 2024-05-14T15:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381a632700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 0x7f38141a17d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f380c00d950 tx=0x7f380c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c017070 con 0x7f3814110b80 2024-05-14T15:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.623+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38141a20b0 con 0x7f3814110b80 2024-05-14T15:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.624+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f380c005020 con 0x7f3814110b80 2024-05-14T15:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.624+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c009990 con 0x7f3814110b80 2024-05-14T15:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.624+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a25f0 con 0x7f3814110b80 2024-05-14T15:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.624+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38140512c0 con 0x7f3814110b80 2024-05-14T15:25:44.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.626+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f380c004a30 con 0x7f3814110b80 2024-05-14T15:25:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.627+0000 7f3812ffd700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f37fc0b2b10 0x7f37fc0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.627+0000 7f3819e31700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f37fc0b2b10 0x7f37fc0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.627+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f380c0e2850 con 0x7f3814110b80 2024-05-14T15:25:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.627+0000 7f3819e31700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f37fc0b2b10 0x7f37fc0b5020 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3804006130 tx=0x7f38040060a0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.631+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f380c0a9c20 con 0x7f3814110b80 2024-05-14T15:25:44.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.897+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f381402e5f0 con 0x7f3814110b80 2024-05-14T15:25:44.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.898+0000 7f3812ffd700 1 -- 172.21.15.5:0/3976289295 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11864 (secure 0 0 0) 0x7f380c0a9520 con 0x7f3814110b80 2024-05-14T15:25:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:25:44.900 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:24:47.806060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:25:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f37fc0b2b10 msgr2=0x7f37fc0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f37fc0b2b10 0x7f37fc0b5020 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3804006130 tx=0x7f38040060a0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 msgr2=0x7f38141a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 0x7f38141a17d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f380c00d950 tx=0x7f380c00dc60 comp rx=0 tx=0).stop 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 shutdown_connections 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f37fc0b2b10 0x7f37fc0b5020 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3814110b80 0x7f38141a17d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3814076e40 0x7f381419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 --2- 172.21.15.5:0/3976289295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3814077790 0x7f381419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 >> 172.21.15.5:0/3976289295 conn(0x7f38140ffe80 msgr2=0x7f3814100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 shutdown_connections 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:44.900+0000 7f381c095700 1 -- 172.21.15.5:0/3976289295 wait complete. 2024-05-14T15:25:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-14T15:25:45.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:25:45.983 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:46.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:45 smithi144 ceph-mon[35650]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:46.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:45 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3976289295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:25:46.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:45 smithi193 ceph-mon[36001]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:46.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:45 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3976289295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:25:46.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:45 smithi005 ceph-mon[31559]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-14T15:25:46.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:45 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3976289295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:25:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 -- 172.21.15.5:0/3859928605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 msgr2=0x7f986810dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 --2- 172.21.15.5:0/3859928605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f986810dc20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f985c009900 tx=0x7f985c02a410 comp rx=0 tx=0).stop 2024-05-14T15:25:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 -- 172.21.15.5:0/3859928605 shutdown_connections 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 --2- 172.21.15.5:0/3859928605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f986810e1f0 0x7f98681106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 --2- 172.21.15.5:0/3859928605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f986810dc20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 --2- 172.21.15.5:0/3859928605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9868104da0 0x7f98681051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 -- 172.21.15.5:0/3859928605 >> 172.21.15.5:0/3859928605 conn(0x7f98680782e0 msgr2=0x7f9868078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 -- 172.21.15.5:0/3859928605 shutdown_connections 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.244+0000 7f986c96c640 1 -- 172.21.15.5:0/3859928605 wait complete. 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 Processor -- start 2024-05-14T15:25:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 -- start start 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9868104da0 0x7f98681a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f98681a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986810e1f0 0x7f986819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9868113e90 con 0x7f986810b800 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9868113d10 con 0x7f986810e1f0 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986c96c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9868114010 con 0x7f9868104da0 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f98681a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f98681a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53626/0 (socket says 172.21.15.5:53626) 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 -- 172.21.15.5:0/351298862 learned_addr learned my addr 172.21.15.5:0/351298862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f986659c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9868104da0 0x7f98681a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9866d9d640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986810e1f0 0x7f986819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 -- 172.21.15.5:0/351298862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9868104da0 msgr2=0x7f98681a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9868104da0 0x7f98681a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 -- 172.21.15.5:0/351298862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986810e1f0 msgr2=0x7f986819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986810e1f0 0x7f986819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.245+0000 7f9865d9b640 1 -- 172.21.15.5:0/351298862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819aea0 con 0x7f986810b800 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.246+0000 7f9865d9b640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f98681a0b30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f985c005940 tx=0x7f985c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.246+0000 7f98577fe640 1 -- 172.21.15.5:0/351298862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c03f030 con 0x7f986810b800 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.246+0000 7f98577fe640 1 -- 172.21.15.5:0/351298862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c003cb0 con 0x7f986810b800 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.246+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986819b0d0 con 0x7f986810b800 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.246+0000 7f98577fe640 1 -- 172.21.15.5:0/351298862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c035070 con 0x7f986810b800 2024-05-14T15:25:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.246+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a7390 con 0x7f986810b800 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 Processor -- start 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f98577fe640 1 -- 172.21.15.5:0/351298862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f985c042030 con 0x7f986810b800 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 -- start start 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c6d88365d0 con 0x55c6d85072f0 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c6d87ef890 con 0x55c6d85301b0 2024-05-14T15:25:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f987381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6d8832590 con 0x55c6d850f4d0 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f986659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f986659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53632/0 (socket says 172.21.15.5:53632) 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.249+0000 7f986659c640 1 -- 172.21.15.5:0/1732609404 learned_addr learned my addr 172.21.15.5:0/1732609404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f9866d9d640 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f986659c640 1 -- 172.21.15.5:0/1732609404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 msgr2=0x55c6d88d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f986659c640 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f986659c640 1 -- 172.21.15.5:0/1732609404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 msgr2=0x55c6d88d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f98577fe640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98500b2af0 0x7f98500b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f986659c640 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f986659c640 1 -- 172.21.15.5:0/1732609404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d8548210 con 0x55c6d85072f0 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f9866d9d640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98500b2af0 0x7f98500b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f986659c640 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88d0fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9858009a70 tx=0x7f9858009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c797ad944fae4600 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f98577fe640 1 -- 172.21.15.5:0/351298862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f985c103400 con 0x7f986810b800 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f9854ff9640 1 -- 172.21.15.5:0/1732609404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858017070 con 0x55c6d85072f0 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f9854ff9640 1 -- 172.21.15.5:0/1732609404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9858013470 con 0x55c6d85072f0 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.250+0000 7f9854ff9640 1 -- 172.21.15.5:0/1732609404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580125d0 con 0x55c6d85072f0 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 -- 172.21.15.5:0/1732609404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 msgr2=0x55c6d88d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f9866d9d640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98500b2af0 0x7f98500b5000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9860005fd0 tx=0x7f9860004a40 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88d0fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9858009a70 tx=0x7f9858009f30 comp rx=0 tx=0).stop 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 -- 172.21.15.5:0/1732609404 shutdown_connections 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88d0fd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 --2- 172.21.15.5:0/1732609404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 -- 172.21.15.5:0/1732609404 >> 172.21.15.5:0/1732609404 conn(0x55c6d8524300 msgr2=0x55c6d88cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 -- 172.21.15.5:0/1732609404 shutdown_connections 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.251+0000 7f987381c740 1 -- 172.21.15.5:0/1732609404 wait complete. 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 -- start start 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 Processor -- start 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d850f4d0 0x55c6d88dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85301b0 0x55c6d88eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c6d88e2120 con 0x55c6d85072f0 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c6d87f3370 con 0x55c6d850f4d0 2024-05-14T15:25:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6d88e22a0 con 0x55c6d85301b0 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f987381c740 10 client.0 _ll_register_callbacks cb 0x55c6d8547a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f986659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f986659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88df5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53648/0 (socket says 172.21.15.5:53648) 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f986659c640 1 -- 172.21.15.5:0/1413746985 learned_addr learned my addr 172.21.15.5:0/1413746985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f9866d9d640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85301b0 0x55c6d88eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.252+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d850f4d0 0x55c6d88dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f986659c640 1 -- 172.21.15.5:0/1413746985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85301b0 msgr2=0x55c6d88eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f986659c640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85301b0 0x55c6d88eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f986659c640 1 -- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d850f4d0 msgr2=0x55c6d88dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f986659c640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d850f4d0 0x55c6d88dfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f986659c640 1 -- 172.21.15.5:0/1413746985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d8614620 con 0x55c6d85072f0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f986659c640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88df5b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9858004810 tx=0x7f9858013c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519dc7c154d51cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f984dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f984dffb640 1 -- 172.21.15.5:0/1413746985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858017050 con 0x55c6d85072f0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f984dffb640 1 -- 172.21.15.5:0/1413746985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f985801c650 con 0x55c6d85072f0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9858013470 con 0x55c6d85072f0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f984dffb640 1 -- 172.21.15.5:0/1413746985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f985802f050 con 0x55c6d85072f0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f987381c740 10 client.15528 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c6d88ee450 con 0x55c6d85072f0 2024-05-14T15:25:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.253+0000 7f984dffb640 1 -- 172.21.15.5:0/1413746985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9858013470 con 0x55c6d85072f0 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f984dffb640 1 -- 172.21.15.5:0/1413746985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9858026870 con 0x55c6d85072f0 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f987381c740 10 client.15528 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f987381c740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f987381c740 20 client.15528 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f987381c740 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84eed80 0x55c6d88f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f987381c740 4 client.15528 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d88441e0 con 0x55c6d84eed80 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.254+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84eed80 0x55c6d88f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f984dffb640 10 client.15528 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84eed80 0x55c6d88f1860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c7c0292712a7730 server_cookie=656924421df5bcf in_seq=0 out_seq=0 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f984dffb640 1 -- 172.21.15.5:0/1413746985 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6d88441e0 con 0x55c6d84eed80 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f984dffb640 10 client.15528 handle_command_reply: tid=0 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f987381c740 1 client.15528 shutdown 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 msgr2=0x55c6d88df5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.255+0000 7f987381c740 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88df5b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9858004810 tx=0x7f9858013c30 comp rx=0 tx=0).stop 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 shutdown_connections 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84eed80 0x55c6d88f1860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85301b0 0x55c6d88eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d850f4d0 0x55c6d88dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f984dffb640 0 client.15528 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 --2- 172.21.15.5:0/1413746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85072f0 0x55c6d88df5b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 >> 172.21.15.5:0/1413746985 conn(0x55c6d8524300 msgr2=0x55c6d88d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 shutdown_connections 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 1 -- 172.21.15.5:0/1413746985 wait complete. 2024-05-14T15:25:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.256+0000 7f987381c740 20 client.15528 trim_cache size 0 max 16384 2024-05-14T15:25:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 Processor -- start 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 -- start start 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85072f0 0x55c6d88cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 0x55c6d88d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98681b19c0 con 0x55c6d850f4d0 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98681af290 con 0x55c6d85301b0 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f987381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ad0e0 con 0x55c6d85072f0 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f9865d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 0x55c6d88d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f9865d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 0x55c6d88d8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53656/0 (socket says 172.21.15.5:53656) 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.271+0000 7f9865d9b640 1 -- 172.21.15.5:0/1699783265 learned_addr learned my addr 172.21.15.5:0/1699783265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f986659c640 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85072f0 0x55c6d88cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9865d9b640 1 -- 172.21.15.5:0/1699783265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85072f0 msgr2=0x55c6d88cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9866d9d640 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85072f0 0x55c6d88cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9865d9b640 1 -- 172.21.15.5:0/1699783265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 msgr2=0x55c6d88d9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9865d9b640 1 -- 172.21.15.5:0/1699783265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d88e1c40 con 0x55c6d850f4d0 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f9865d9b640 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 0x55c6d88d8e20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f985c003f80 tx=0x7f985c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c6fbf74677ebeb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f984ffff640 1 -- 172.21.15.5:0/1699783265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c049040 con 0x55c6d850f4d0 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f984ffff640 1 -- 172.21.15.5:0/1699783265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c0c9cb0 con 0x55c6d850f4d0 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.272+0000 7f984ffff640 1 -- 172.21.15.5:0/1699783265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c005e00 con 0x55c6d850f4d0 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 -- 172.21.15.5:0/1699783265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 msgr2=0x55c6d88d8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 0x55c6d88d8e20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f985c003f80 tx=0x7f985c032f60 comp rx=0 tx=0).stop 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 -- 172.21.15.5:0/1699783265 shutdown_connections 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85301b0 0x55c6d88d9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d850f4d0 0x55c6d88d8e20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 --2- 172.21.15.5:0/1699783265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d85072f0 0x55c6d88cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 -- 172.21.15.5:0/1699783265 >> 172.21.15.5:0/1699783265 conn(0x55c6d8524300 msgr2=0x55c6d88cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 -- 172.21.15.5:0/1699783265 shutdown_connections 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 -- 172.21.15.5:0/1699783265 wait complete. 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 -- start start 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.273+0000 7f987381c740 1 Processor -- start 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85072f0 0x55c6d88ef850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 0x55c6d8957970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ad750 con 0x55c6d85301b0 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98681aebc0 con 0x55c6d85072f0 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98681ade20 con 0x55c6d850f4d0 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f987381c740 10 client.0 _ll_register_callbacks cb 0x7f98580098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 0x55c6d8957970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 0x55c6d8957970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53660/0 (socket says 172.21.15.5:53660) 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 -- 172.21.15.5:0/2364035689 learned_addr learned my addr 172.21.15.5:0/2364035689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9865d9b640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f986659c640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85072f0 0x55c6d88ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 -- 172.21.15.5:0/2364035689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 msgr2=0x55c6d88efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 -- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85072f0 msgr2=0x55c6d88ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85072f0 0x55c6d88ef850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 -- 172.21.15.5:0/2364035689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d88f05e0 con 0x55c6d85301b0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f986659c640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85072f0 0x55c6d88ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f9866d9d640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 0x55c6d8957970 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f986001f9f0 tx=0x7f9860004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bf3175dd94ff54 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.274+0000 7f984dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f984dffb640 1 -- 172.21.15.5:0/2364035689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860012b00 con 0x55c6d85301b0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f984dffb640 1 -- 172.21.15.5:0/2364035689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f9860012ca0 con 0x55c6d85301b0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c0c9cb0 con 0x55c6d85301b0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f984dffb640 1 -- 172.21.15.5:0/2364035689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f986001d470 con 0x55c6d85301b0 2024-05-14T15:25:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 10 client.15534 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c6d8958f30 con 0x55c6d85301b0 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f984dffb640 1 -- 172.21.15.5:0/2364035689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c0c9cb0 con 0x55c6d85301b0 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f984dffb640 1 -- 172.21.15.5:0/2364035689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f986001b850 con 0x55c6d85301b0 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 10 client.15534 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 20 client.15534 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84c41c0 0x55c6d895b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 4 client.15534 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6d883e050 con 0x55c6d84c41c0 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.275+0000 7f986659c640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84c41c0 0x55c6d895b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.276+0000 7f984dffb640 10 client.15534 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.276+0000 7f986659c640 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84c41c0 0x55c6d895b770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b61a252404a359f server_cookie=8d6988136125acbc in_seq=0 out_seq=0 2024-05-14T15:25:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.283+0000 7f984dffb640 1 -- 172.21.15.5:0/2364035689 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6d883e050 con 0x55c6d84c41c0 2024-05-14T15:25:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.283+0000 7f984dffb640 10 client.15534 handle_command_reply: tid=0 2024-05-14T15:25:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.283+0000 7f987381c740 1 client.15534 shutdown 2024-05-14T15:25:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 msgr2=0x55c6d8957970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 0x55c6d8957970 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f986001f9f0 tx=0x7f9860004ab0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 shutdown_connections 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c6d84c41c0 0x55c6d895b770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6d850f4d0 0x55c6d88efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6d85301b0 0x55c6d8957970 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f984dffb640 0 client.15534 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 --2- 172.21.15.5:0/2364035689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c6d85072f0 0x55c6d88ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 >> 172.21.15.5:0/2364035689 conn(0x55c6d8524300 msgr2=0x55c6d88e0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 shutdown_connections 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 1 -- 172.21.15.5:0/2364035689 wait complete. 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f987381c740 20 client.15534 trim_cache size 0 max 16384 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:25:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:25:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98500b2af0 msgr2=0x7f98500b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98500b2af0 0x7f98500b5000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9860005fd0 tx=0x7f9860004a40 comp rx=0 tx=0).stop 2024-05-14T15:25:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 msgr2=0x7f98681a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f98681a0b30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f985c005940 tx=0x7f985c0042f0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 shutdown_connections 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98500b2af0 0x7f98500b5000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986810e1f0 0x7f986819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f986810b800 0x7f98681a0b30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 --2- 172.21.15.5:0/351298862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9868104da0 0x7f98681a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.284+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 >> 172.21.15.5:0/351298862 conn(0x7f98680782e0 msgr2=0x7f98681012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.285+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 shutdown_connections 2024-05-14T15:25:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:46.285+0000 7f986c96c640 1 -- 172.21.15.5:0/351298862 wait complete. 2024-05-14T15:25:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 -- 172.21.15.5:0/2131207002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 msgr2=0x7f0e601111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 --2- 172.21.15.5:0/2131207002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e601111c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b0d0 tx=0x7f0e5800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 -- 172.21.15.5:0/2131207002 shutdown_connections 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 --2- 172.21.15.5:0/2131207002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e601111c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 --2- 172.21.15.5:0/2131207002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e601045e0 0x7f0e601049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 --2- 172.21.15.5:0/2131207002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e60108f10 0x7f0e60109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 -- 172.21.15.5:0/2131207002 >> 172.21.15.5:0/2131207002 conn(0x7f0e600ffed0 msgr2=0x7f0e60102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 -- 172.21.15.5:0/2131207002 shutdown_connections 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.766+0000 7f0e684b9700 1 -- 172.21.15.5:0/2131207002 wait complete. 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 Processor -- start 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 -- start start 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e601045e0 0x7f0e60076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60108f10 0x7f0e60077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e60077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601144e0 con 0x7f0e60109860 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60114360 con 0x7f0e601045e0 2024-05-14T15:25:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.767+0000 7f0e684b9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60114660 con 0x7f0e60108f10 2024-05-14T15:25:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e60077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e60077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53688/0 (socket says 172.21.15.5:53688) 2024-05-14T15:25:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 -- 172.21.15.5:0/164155786 learned_addr learned my addr 172.21.15.5:0/164155786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66255700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e601045e0 0x7f0e60076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e65a54700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60108f10 0x7f0e60077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 -- 172.21.15.5:0/164155786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60108f10 msgr2=0x7f0e60077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60108f10 0x7f0e60077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 -- 172.21.15.5:0/164155786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e601045e0 msgr2=0x7f0e60076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e601045e0 0x7f0e60076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 -- 172.21.15.5:0/164155786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e601a91b0 con 0x7f0e60109860 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e66a56700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e60077650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0e580079a0 tx=0x7f0e580079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e577fe700 1 -- 172.21.15.5:0/164155786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5801f070 con 0x7f0e60109860 2024-05-14T15:25:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e601a9440 con 0x7f0e60109860 2024-05-14T15:25:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e577fe700 1 -- 172.21.15.5:0/164155786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e58003c00 con 0x7f0e60109860 2024-05-14T15:25:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.768+0000 7f0e577fe700 1 -- 172.21.15.5:0/164155786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5801b690 con 0x7f0e60109860 2024-05-14T15:25:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.769+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a9890 con 0x7f0e60109860 2024-05-14T15:25:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.772+0000 7f0e577fe700 1 -- 172.21.15.5:0/164155786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f0e58022070 con 0x7f0e60109860 2024-05-14T15:25:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.772+0000 7f0e577fe700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e4c0b2b10 0x7f0e4c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.772+0000 7f0e76ecdb80 1 Processor -- start 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.772+0000 7f0e66255700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e4c0b2b10 0x7f0e4c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e577fe700 1 -- 172.21.15.5:0/164155786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0e5800f020 con 0x7f0e60109860 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 -- start start 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef19afe40 0x556ef1b2d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b31f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 0x556ef1b32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ef1a4f860 con 0x556ef1b32470 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556ef19ee7c0 con 0x556ef1b2dc60 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556ef1a4b820 con 0x556ef19afe40 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66255700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e4c0b2b10 0x7f0e4c0b5020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005950 tx=0x7f0e5c00b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:47.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66a56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 0x556ef1b32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66a56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 0x556ef1b32880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53692/0 (socket says 172.21.15.5:53692) 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66a56700 1 -- 172.21.15.5:0/1594590133 learned_addr learned my addr 172.21.15.5:0/1594590133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e65a54700 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b31f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66255700 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef19afe40 0x556ef1b2d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66a56700 1 -- 172.21.15.5:0/1594590133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef19afe40 msgr2=0x556ef1b2d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.773+0000 7f0e66a56700 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef19afe40 0x556ef1b2d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e66a56700 1 -- 172.21.15.5:0/1594590133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 msgr2=0x556ef1b31f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e66a56700 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b31f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e66a56700 1 -- 172.21.15.5:0/1594590133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef199c8d0 con 0x556ef1b32470 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e65a54700 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e66a56700 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 0x556ef1b32880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e58003fd0 tx=0x7f0e580099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d02897c8f17c914 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e54d47700 1 -- 172.21.15.5:0/1594590133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e580c07e0 con 0x556ef1b32470 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e54d47700 1 -- 172.21.15.5:0/1594590133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e580b3040 con 0x556ef1b32470 2024-05-14T15:25:47.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.774+0000 7f0e54d47700 1 -- 172.21.15.5:0/1594590133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e580aee90 con 0x556ef1b32470 2024-05-14T15:25:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1594590133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 msgr2=0x556ef1b32880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 0x556ef1b32880 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e58003fd0 tx=0x7f0e580099e0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1594590133 shutdown_connections 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef1b32470 0x556ef1b32880 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef19afe40 0x556ef1b2d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1594590133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1594590133 >> 172.21.15.5:0/1594590133 conn(0x556ef19f7600 msgr2=0x556ef1b284b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1594590133 shutdown_connections 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1594590133 wait complete. 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.775+0000 7f0e76ecdb80 1 -- start start 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 Processor -- start 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b32470 0x556ef1b37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ef1a4f860 con 0x556ef19afe40 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556ef19b5c30 con 0x556ef1b2dc60 2024-05-14T15:25:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556ef1b3a1e0 con 0x556ef1b32470 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e76ecdb80 10 client.0 _ll_register_callbacks cb 0x556ef19e7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b371b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53708/0 (socket says 172.21.15.5:53708) 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 -- 172.21.15.5:0/3657755742 learned_addr learned my addr 172.21.15.5:0/3657755742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 -- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b32470 msgr2=0x556ef1b37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b32470 0x556ef1b37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 -- 172.21.15.5:0/3657755742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 msgr2=0x556ef1b376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.776+0000 7f0e66255700 1 -- 172.21.15.5:0/3657755742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef19eb690 con 0x556ef19afe40 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e66255700 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b371b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0116f0 tx=0x7f0e5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e624210f035ff98a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3657755742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c01e830 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3657755742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0e5c01e9d0 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e580b3040 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3657755742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e5c031050 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 10 client.15546 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ef1b44cb0 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3657755742 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e580b3040 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3657755742 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0e5c026020 con 0x556ef19afe40 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 10 client.15546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 20 client.15546 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1b47340 0x556ef1b497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 4 client.15546 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.777+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ef1a5d240 con 0x556ef1b47340 2024-05-14T15:25:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.778+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1b47340 0x556ef1b497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.778+0000 7f0e49ffb700 10 client.15546 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:25:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.778+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1b47340 0x556ef1b497f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9da27913b515e54 server_cookie=5d00eb4ddaae548b in_seq=0 out_seq=0 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3657755742 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ef1a5d240 con 0x556ef1b47340 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e49ffb700 10 client.15546 handle_command_reply: tid=0 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 client.15546 shutdown 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 msgr2=0x556ef1b371b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b371b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0116f0 tx=0x7f0e5c0047c0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 shutdown_connections 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b32470 0x556ef1b37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b371b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1b47340 0x556ef1b497f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3657755742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b2dc60 0x556ef1b376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 >> 172.21.15.5:0/3657755742 conn(0x556ef19f7600 msgr2=0x556ef1b2c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e49ffb700 0 client.15546 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 shutdown_connections 2024-05-14T15:25:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3657755742 wait complete. 2024-05-14T15:25:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.779+0000 7f0e76ecdb80 20 client.15546 trim_cache size 0 max 16384 2024-05-14T15:25:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.797+0000 7f0e76ecdb80 1 Processor -- start 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.797+0000 7f0e76ecdb80 1 -- start start 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b5090 con 0x556ef19afe40 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b2920 con 0x556ef1b32470 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b0770 con 0x556ef1b2dc60 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b37a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53724/0 (socket says 172.21.15.5:53724) 2024-05-14T15:25:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 -- 172.21.15.5:0/1311174064 learned_addr learned my addr 172.21.15.5:0/1311174064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e65a54700 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66a56700 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 -- 172.21.15.5:0/1311174064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 msgr2=0x556ef1b361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 -- 172.21.15.5:0/1311174064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 msgr2=0x556ef1b430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 -- 172.21.15.5:0/1311174064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef1b35720 con 0x556ef19afe40 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.798+0000 7f0e66255700 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b37a70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c01cd80 tx=0x7f0e5c00dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3951fa6b85060ac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e4bfff700 1 -- 172.21.15.5:0/1311174064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c039070 con 0x556ef19afe40 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e4bfff700 1 -- 172.21.15.5:0/1311174064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c011e40 con 0x556ef19afe40 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e4bfff700 1 -- 172.21.15.5:0/1311174064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c005020 con 0x556ef19afe40 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1311174064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 msgr2=0x556ef1b37a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b37a70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c01cd80 tx=0x7f0e5c00dc50 comp rx=0 tx=0).stop 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1311174064 shutdown_connections 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b37a70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/1311174064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.799+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1311174064 >> 172.21.15.5:0/1311174064 conn(0x556ef19f7600 msgr2=0x556ef1b35aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:47.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1311174064 shutdown_connections 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/1311174064 wait complete. 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 -- start start 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 Processor -- start 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b3cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b3d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b0de0 con 0x556ef19afe40 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b2250 con 0x556ef1b32470 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b14b0 con 0x556ef1b2dc60 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e76ecdb80 10 client.0 _ll_register_callbacks cb 0x556ef19e7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.800+0000 7f0e66255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b3f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53732/0 (socket says 172.21.15.5:53732) 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 -- 172.21.15.5:0/3802834004 learned_addr learned my addr 172.21.15.5:0/3802834004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66a56700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b3d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b3cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 -- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 msgr2=0x556ef1b3cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b3cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 -- 172.21.15.5:0/3802834004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 msgr2=0x556ef1b3d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b3d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 -- 172.21.15.5:0/3802834004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef1b3d5c0 con 0x556ef19afe40 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66a56700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b3d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b3cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e66255700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b3f790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c006010 tx=0x7f0e5c004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cc986ec1285637 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3802834004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c041070 con 0x556ef19afe40 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ef1a572e0 con 0x556ef19afe40 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3802834004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0e5c005430 con 0x556ef19afe40 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.801+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3802834004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e5c04d050 con 0x556ef19afe40 2024-05-14T15:25:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 10 client.15552 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:47.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ef1bb1880 con 0x556ef19afe40 2024-05-14T15:25:47.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3802834004 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556ef1a572e0 con 0x556ef19afe40 2024-05-14T15:25:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3802834004 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0e5c034070 con 0x556ef19afe40 2024-05-14T15:25:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 10 client.15552 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 10 client.15552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:25:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 20 client.15552 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1bb1e10 0x556ef1bb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 4 client.15552 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ef18cf250 con 0x556ef1bb1e10 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.802+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1bb1e10 0x556ef1bb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e49ffb700 10 client.15552 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e65a54700 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1bb1e10 0x556ef1bb4230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=785e023be6531d0e server_cookie=30575c32ad6f32fb in_seq=0 out_seq=0 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e49ffb700 1 -- 172.21.15.5:0/3802834004 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ef18cf250 con 0x556ef1bb1e10 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e49ffb700 10 client.15552 handle_command_reply: tid=0 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 client.15552 shutdown 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 msgr2=0x556ef1b3f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b3f790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c006010 tx=0x7f0e5c004e60 comp rx=0 tx=0).stop 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 shutdown_connections 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x556ef1bb1e10 0x556ef1bb4230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556ef1b32470 0x556ef1b3d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ef19afe40 0x556ef1b3f790 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e49ffb700 0 client.15552 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:25:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 --2- 172.21.15.5:0/3802834004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556ef1b2dc60 0x556ef1b3cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 >> 172.21.15.5:0/3802834004 conn(0x556ef19f7600 msgr2=0x556ef1b2c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 shutdown_connections 2024-05-14T15:25:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.803+0000 7f0e76ecdb80 1 -- 172.21.15.5:0/3802834004 wait complete. 2024-05-14T15:25:47.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e76ecdb80 20 client.15552 trim_cache size 0 max 16384 2024-05-14T15:25:47.810 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e4c0b2b10 msgr2=0x7f0e4c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e4c0b2b10 0x7f0e4c0b5020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005950 tx=0x7f0e5c00b410 comp rx=0 tx=0).stop 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 msgr2=0x7f0e60077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e60077650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0e580079a0 tx=0x7f0e580079d0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 shutdown_connections 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e4c0b2b10 0x7f0e4c0b5020 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e60109860 0x7f0e60077650 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e601045e0 0x7f0e60076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 --2- 172.21.15.5:0/164155786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60108f10 0x7f0e60077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:47.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 >> 172.21.15.5:0/164155786 conn(0x7f0e600ffed0 msgr2=0x7f0e60108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:47.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 shutdown_connections 2024-05-14T15:25:47.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:47.804+0000 7f0e684b9700 1 -- 172.21.15.5:0/164155786 wait complete. 2024-05-14T15:25:48.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:47 smithi144 ceph-mon[35650]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-05-14T15:25:48.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:47 smithi193 ceph-mon[36001]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-05-14T15:25:48.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:47 smithi005 ceph-mon[31559]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-05-14T15:25:48.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:25:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:50.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:49 smithi144 ceph-mon[35650]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T15:25:50.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:49 smithi193 ceph-mon[36001]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T15:25:50.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:49 smithi005 ceph-mon[31559]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 -- 172.21.15.5:0/839063390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac108740 msgr2=0x7fd5ac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 --2- 172.21.15.5:0/839063390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac108740 0x7fd5ac1100a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400b0d0 tx=0x7fd5a400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 -- 172.21.15.5:0/839063390 shutdown_connections 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 --2- 172.21.15.5:0/839063390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac108740 0x7fd5ac1100a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 --2- 172.21.15.5:0/839063390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 0x7fd5ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 --2- 172.21.15.5:0/839063390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ac107df0 0x7fd5ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 -- 172.21.15.5:0/839063390 >> 172.21.15.5:0/839063390 conn(0x7fd5ac0fedc0 msgr2=0x7fd5ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 -- 172.21.15.5:0/839063390 shutdown_connections 2024-05-14T15:25:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.704+0000 7fd5b404f700 1 -- 172.21.15.5:0/839063390 wait complete. 2024-05-14T15:25:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 Processor -- start 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 -- start start 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 0x7fd5ac19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 0x7fd5ac19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ac108740 0x7fd5ac19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac113370 con 0x7fd5ac107df0 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1131f0 con 0x7fd5ac108740 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b404f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1134f0 con 0x7fd5ac1034c0 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b15ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 0x7fd5ac19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b15ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 0x7fd5ac19c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53758/0 (socket says 172.21.15.5:53758) 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b15ea700 1 -- 172.21.15.5:0/1212323909 learned_addr learned my addr 172.21.15.5:0/1212323909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ac108740 0x7fd5ac19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.705+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 0x7fd5ac19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b15ea700 1 -- 172.21.15.5:0/1212323909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 msgr2=0x7fd5ac19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 0x7fd5ac19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b15ea700 1 -- 172.21.15.5:0/1212323909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ac108740 msgr2=0x7fd5ac19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ac108740 0x7fd5ac19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b15ea700 1 -- 172.21.15.5:0/1212323909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac1a6f70 con 0x7fd5ac107df0 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 0x7fd5ac19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 0x7fd5ac19c1d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c9e0 tx=0x7fd59c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5a2ffd700 1 -- 172.21.15.5:0/1212323909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c007aa0 con 0x7fd5ac107df0 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5a2ffd700 1 -- 172.21.15.5:0/1212323909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59c007c40 con 0x7fd5ac107df0 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5a2ffd700 1 -- 172.21.15.5:0/1212323909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c018660 con 0x7fd5ac107df0 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac1a7260 con 0x7fd5ac107df0 2024-05-14T15:25:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.706+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac1a77f0 con 0x7fd5ac107df0 2024-05-14T15:25:50.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.709+0000 7fd5a2ffd700 1 -- 172.21.15.5:0/1212323909 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fd59c018800 con 0x7fd5ac107df0 2024-05-14T15:25:50.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5a2ffd700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd5980b2b90 0x7fd5980b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5c2a63b80 1 Processor -- start 2024-05-14T15:25:50.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd5980b2b90 0x7fd5980b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5a2ffd700 1 -- 172.21.15.5:0/1212323909 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fd59c0e2b90 con 0x7fd5ac107df0 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5c2a63b80 1 -- start start 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd5980b2b90 0x7fd5980b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005950 tx=0x7fd5a80058e0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd046db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.710+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0467ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0468630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560dd0385940 con 0x560dd02e5e40 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560dd02eada0 con 0x560dd046e0b0 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560dd0381900 con 0x560dd0468220 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd046db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd046db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53768/0 (socket says 172.21.15.5:53768) 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 -- 172.21.15.5:0/2423800277 learned_addr learned my addr 172.21.15.5:0/2423800277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0468630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0467ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 -- 172.21.15.5:0/2423800277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 msgr2=0x560dd0468630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0468630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 -- 172.21.15.5:0/2423800277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 msgr2=0x560dd0467ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0467ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 -- 172.21.15.5:0/2423800277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd02d28d0 con 0x560dd02e5e40 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0468630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd046db70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800ff00 tx=0x7fd5a8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbf580077590a0b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd597fff700 1 -- 172.21.15.5:0/2423800277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8003880 con 0x560dd02e5e40 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd597fff700 1 -- 172.21.15.5:0/2423800277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a8024070 con 0x560dd02e5e40 2024-05-14T15:25:50.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.711+0000 7fd597fff700 1 -- 172.21.15.5:0/2423800277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a801fde0 con 0x560dd02e5e40 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2423800277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 msgr2=0x560dd046db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd046db70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800ff00 tx=0x7fd5a8012000 comp rx=0 tx=0).stop 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2423800277 shutdown_connections 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0468630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd046db70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2423800277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0467ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.712+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2423800277 >> 172.21.15.5:0/2423800277 conn(0x560dd032d630 msgr2=0x560dd045ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.713+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2423800277 shutdown_connections 2024-05-14T15:25:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.713+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2423800277 wait complete. 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.713+0000 7fd5c2a63b80 1 -- start start 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.713+0000 7fd5c2a63b80 1 Processor -- start 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.713+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd02e5e40 0x560dd0471900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0471e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 0x560dd0472380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560dd0474b90 con 0x560dd046e0b0 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560dd033eed0 con 0x560dd02e5e40 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560dd0474d10 con 0x560dd0468220 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5c2a63b80 10 client.0 _ll_register_callbacks cb 0x560dd031d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 0x560dd0472380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 0x560dd0472380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53784/0 (socket says 172.21.15.5:53784) 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 -- 172.21.15.5:0/2516275146 learned_addr learned my addr 172.21.15.5:0/2516275146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd02e5e40 0x560dd0471900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0471e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 -- 172.21.15.5:0/2516275146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 msgr2=0x560dd0471e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0471e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 -- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd02e5e40 msgr2=0x560dd0471900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd02e5e40 0x560dd0471900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 -- 172.21.15.5:0/2516275146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd033c760 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 0x560dd0472380 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4015000 tx=0x7fd5a4003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586d3a4674664d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.714+0000 7fd5957fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5957fa700 1 -- 172.21.15.5:0/2516275146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40077c0 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a8024070 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5957fa700 1 -- 172.21.15.5:0/2516275146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fd5a4007960 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5957fa700 1 -- 172.21.15.5:0/2516275146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5a402e050 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 10 client.15564 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560dd047f770 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5957fa700 1 -- 172.21.15.5:0/2516275146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a8024070 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5957fa700 1 -- 172.21.15.5:0/2516275146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd5a4011070 con 0x560dd046e0b0 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 10 client.15564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:25:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 20 client.15564 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd0481d00 0x560dd0484120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 4 client.15564 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.715+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd0393320 con 0x560dd0481d00 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.716+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd0481d00 0x560dd0484120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.716+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd0481d00 0x560dd0484120 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f919667800dc8a2 server_cookie=328c0e2947bc0ba5 in_seq=0 out_seq=0 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.716+0000 7fd5957fa700 10 client.15564 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5957fa700 1 -- 172.21.15.5:0/2516275146 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560dd0393320 con 0x560dd0481d00 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5957fa700 10 client.15564 handle_command_reply: tid=0 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 client.15564 shutdown 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 msgr2=0x560dd0472380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 0x560dd0472380 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4015000 tx=0x7fd5a4003a20 comp rx=0 tx=0).stop 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 shutdown_connections 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd0481d00 0x560dd0484120 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd046e0b0 0x560dd0472380 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd02e5e40 0x560dd0471900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5957fa700 0 client.15564 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2516275146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0471e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 >> 172.21.15.5:0/2516275146 conn(0x560dd032d630 msgr2=0x560dd04628c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 shutdown_connections 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2516275146 wait complete. 2024-05-14T15:25:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.717+0000 7fd5c2a63b80 20 client.15564 trim_cache size 0 max 16384 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 Processor -- start 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 -- start start 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd04721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0458c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd047dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1b0c50 con 0x560dd02e5e40 2024-05-14T15:25:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ae4e0 con 0x560dd046e0b0 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.738+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ac330 con 0x560dd0468220 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd04721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd04721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53788/0 (socket says 172.21.15.5:53788) 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 -- 172.21.15.5:0/1817221636 learned_addr learned my addr 172.21.15.5:0/1817221636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 -- 172.21.15.5:0/1817221636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 msgr2=0x560dd0458c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0458c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0458c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 -- 172.21.15.5:0/1817221636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 msgr2=0x560dd047dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd047dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 -- 172.21.15.5:0/1817221636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd046d5a0 con 0x560dd02e5e40 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0458c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd04721c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800baa0 tx=0x7fd5a801f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4420a3e7654fe240 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5977fe700 1 -- 172.21.15.5:0/1817221636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a803b070 con 0x560dd02e5e40 2024-05-14T15:25:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5977fe700 1 -- 172.21.15.5:0/1817221636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a8024040 con 0x560dd02e5e40 2024-05-14T15:25:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.739+0000 7fd5977fe700 1 -- 172.21.15.5:0/1817221636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8003970 con 0x560dd02e5e40 2024-05-14T15:25:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/1817221636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 msgr2=0x560dd04721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd04721c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800baa0 tx=0x7fd5a801f920 comp rx=0 tx=0).stop 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/1817221636 shutdown_connections 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd047dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd02e5e40 0x560dd04721c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/1817221636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd0468220 0x560dd0458c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/1817221636 >> 172.21.15.5:0/1817221636 conn(0x560dd032d630 msgr2=0x560dd0458580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/1817221636 shutdown_connections 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.740+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/1817221636 wait complete. 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 -- start start 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 Processor -- start 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd02e5e40 0x560dd047a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 0x560dd0488f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0489550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ac9a0 con 0x560dd0468220 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ade10 con 0x560dd046e0b0 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1ad070 con 0x560dd02e5e40 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.741+0000 7fd5c2a63b80 10 client.0 _ll_register_callbacks cb 0x560dd031d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 0x560dd0488f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 0x560dd0488f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53790/0 (socket says 172.21.15.5:53790) 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 -- 172.21.15.5:0/2250502510 learned_addr learned my addr 172.21.15.5:0/2250502510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd02e5e40 0x560dd047a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b25ec700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0489550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 -- 172.21.15.5:0/2250502510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd02e5e40 msgr2=0x560dd047a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd02e5e40 0x560dd047a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 -- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 msgr2=0x560dd0489550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0489550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 -- 172.21.15.5:0/2250502510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd0489d20 con 0x560dd0468220 2024-05-14T15:25:50.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5b15ea700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 0x560dd0488f50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0a6a70 tx=0x7fd59c0a9ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae9520cac71cfcc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5957fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5957fa700 1 -- 172.21.15.5:0/2250502510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0a9310 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5957fa700 1 -- 172.21.15.5:0/2250502510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fd59c00dbd0 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.742+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a8024040 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5957fa700 1 -- 172.21.15.5:0/2250502510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd59c0a9f00 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 10 client.15570 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560dd04ec4d0 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5957fa700 1 -- 172.21.15.5:0/2250502510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a8024040 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5957fa700 1 -- 172.21.15.5:0/2250502510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd59c0e0020 con 0x560dd0468220 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 10 client.15570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 20 client.15570 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd04ecb60 0x560dd04eef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 4 client.15570 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:25:50.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560dd038d3c0 con 0x560dd04ecb60 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.743+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd04ecb60 0x560dd04eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5b1deb700 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd04ecb60 0x560dd04eef80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=677541d9689907bd server_cookie=68cfa3218c80fff0 in_seq=0 out_seq=0 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5957fa700 10 client.15570 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5957fa700 1 -- 172.21.15.5:0/2250502510 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560dd038d3c0 con 0x560dd04ecb60 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5957fa700 10 client.15570 handle_command_reply: tid=0 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 client.15570 shutdown 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 msgr2=0x560dd0488f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 0x560dd0488f50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0a6a70 tx=0x7fd59c0a9ce0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 shutdown_connections 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x560dd04ecb60 0x560dd04eef80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dd046e0b0 0x560dd0489550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560dd02e5e40 0x560dd047a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5957fa700 0 client.15570 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 --2- 172.21.15.5:0/2250502510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd0468220 0x560dd0488f50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 >> 172.21.15.5:0/2250502510 conn(0x560dd032d630 msgr2=0x560dd0458d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 shutdown_connections 2024-05-14T15:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 1 -- 172.21.15.5:0/2250502510 wait complete. 2024-05-14T15:25:50.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.744+0000 7fd5c2a63b80 20 client.15570 trim_cache size 0 max 16384 2024-05-14T15:25:50.751 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd5980b2b90 msgr2=0x7fd5980b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd5980b2b90 0x7fd5980b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005950 tx=0x7fd5a80058e0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 msgr2=0x7fd5ac19c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 0x7fd5ac19c1d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c9e0 tx=0x7fd59c00ccf0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 shutdown_connections 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd5980b2b90 0x7fd5980b50a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ac108740 0x7fd5ac19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5ac1034c0 0x7fd5ac19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 --2- 172.21.15.5:0/1212323909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5ac107df0 0x7fd5ac19c1d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 >> 172.21.15.5:0/1212323909 conn(0x7fd5ac0fedc0 msgr2=0x7fd5ac107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 shutdown_connections 2024-05-14T15:25:50.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:50.745+0000 7fd5b404f700 1 -- 172.21.15.5:0/1212323909 wait complete. 2024-05-14T15:25:51.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:25:51.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:51 smithi005 ceph-mon[31559]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T15:25:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:52.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:51 smithi144 ceph-mon[35650]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T15:25:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:51 smithi193 ceph-mon[36001]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 -- 172.21.15.5:0/1428322109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 msgr2=0x7f0cf8108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1428322109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf8108210 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0099c0 tx=0x7f0cec009cd0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 -- 172.21.15.5:0/1428322109 shutdown_connections 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1428322109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf8108750 0x7f0cf81100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1428322109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8103c60 0x7f0cf8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1428322109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf8108210 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 -- 172.21.15.5:0/1428322109 >> 172.21.15.5:0/1428322109 conn(0x7f0cf80fe6f0 msgr2=0x7f0cf8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 -- 172.21.15.5:0/1428322109 shutdown_connections 2024-05-14T15:25:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.518+0000 7f0cfde90700 1 -- 172.21.15.5:0/1428322109 wait complete. 2024-05-14T15:25:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.519+0000 7f0cfde90700 1 Processor -- start 2024-05-14T15:25:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.519+0000 7f0cfde90700 1 -- start start 2024-05-14T15:25:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cfde90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf8103c60 0x7f0cf8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cfde90700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cfde90700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8108750 0x7f0cf819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cfde90700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8113360 con 0x7f0cf8107e00 2024-05-14T15:25:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cfde90700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81131e0 con 0x7f0cf8103c60 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cfde90700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81134e0 con 0x7f0cf8108750 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53812/0 (socket says 172.21.15.5:53812) 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1758748038 learned_addr learned my addr 172.21.15.5:0/1758748038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf8103c60 0x7f0cf8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8108750 0x7f0cf819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1758748038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8108750 msgr2=0x7f0cf819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8108750 0x7f0cf819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1758748038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf8103c60 msgr2=0x7f0cf8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf8103c60 0x7f0cf8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1758748038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf819ec00 con 0x7f0cf8107e00 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8108750 0x7f0cf819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.520+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf819a2d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006010 tx=0x7f0cec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.521+0000 7f0cf4ff9700 1 -- 172.21.15.5:0/1758748038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec027070 con 0x7f0cf8107e00 2024-05-14T15:25:53.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.521+0000 7f0cf4ff9700 1 -- 172.21.15.5:0/1758748038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec003f20 con 0x7f0cf8107e00 2024-05-14T15:25:53.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.521+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf819ee90 con 0x7f0cf8107e00 2024-05-14T15:25:53.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.521+0000 7f0cf4ff9700 1 -- 172.21.15.5:0/1758748038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec022cd0 con 0x7f0cf8107e00 2024-05-14T15:25:53.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.521+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf819f320 con 0x7f0cf8107e00 2024-05-14T15:25:53.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.524+0000 7f0cf4ff9700 1 -- 172.21.15.5:0/1758748038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f0cec02a070 con 0x7f0cf8107e00 2024-05-14T15:25:53.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.524+0000 7f0cf4ff9700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce00b2b10 0x7f0ce00b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce00b2b10 0x7f0ce00b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0cf4ff9700 1 -- 172.21.15.5:0/1758748038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0cec0e3b50 con 0x7f0cf8107e00 2024-05-14T15:25:53.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 Processor -- start 2024-05-14T15:25:53.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 -- start start 2024-05-14T15:25:53.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e068498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e0684a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06857f50 0x563e0685a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce00b2b10 0x7f0ce00b5020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005950 tx=0x7f0ce800b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e0676b940 con 0x563e06849e10 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e066d0da0 con 0x563e06857f50 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563e06767900 con 0x563e066cbe40 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e0684a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e0684a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53814/0 (socket says 172.21.15.5:53814) 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.525+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1934679232 learned_addr learned my addr 172.21.15.5:0/1934679232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e068498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06857f50 0x563e0685a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1934679232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 msgr2=0x563e068498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e068498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1934679232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06857f50 msgr2=0x563e0685a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06857f50 0x563e0685a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/1934679232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e066b88d0 con 0x563e06849e10 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06857f50 0x563e0685a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e0684a220 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0cec000c00 tx=0x7f0cec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db15fc8414963d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:53.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cddffb700 1 -- 172.21.15.5:0/1934679232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0aafb0 con 0x563e06849e10 2024-05-14T15:25:53.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cddffb700 1 -- 172.21.15.5:0/1934679232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec0aa8b0 con 0x563e06849e10 2024-05-14T15:25:53.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.526+0000 7f0cddffb700 1 -- 172.21.15.5:0/1934679232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0aab10 con 0x563e06849e10 2024-05-14T15:25:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1934679232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 msgr2=0x563e0684a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e0684a220 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0cec000c00 tx=0x7f0cec009310 comp rx=0 tx=0).stop 2024-05-14T15:25:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1934679232 shutdown_connections 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06857f50 0x563e0685a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e068498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1934679232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e0684a220 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1934679232 >> 172.21.15.5:0/1934679232 conn(0x563e06713630 msgr2=0x563e06844a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1934679232 shutdown_connections 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.527+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1934679232 wait complete. 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.528+0000 7f0d0c8a4b80 1 -- start start 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.528+0000 7f0d0c8a4b80 1 Processor -- start 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 0x563e06854f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06855450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e06855990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e0685a970 con 0x563e066cbe40 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e06724ed0 con 0x563e06849e10 2024-05-14T15:25:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563e0685aaf0 con 0x563e06857f50 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0d0c8a4b80 10 client.0 _ll_register_callbacks cb 0x563e06703950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 0x563e06854f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 0x563e06854f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53818/0 (socket says 172.21.15.5:53818) 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 -- 172.21.15.5:0/1848822213 learned_addr learned my addr 172.21.15.5:0/1848822213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e06855990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06855450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 -- 172.21.15.5:0/1848822213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 msgr2=0x563e06855990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e06855990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 -- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 msgr2=0x563e06855450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06855450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 -- 172.21.15.5:0/1848822213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e06722760 con 0x563e066cbe40 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 0x563e06854f10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80116f0 tx=0x7f0ce80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b58e08e62ab1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.529+0000 7f0cd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/1848822213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801e830 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/1848822213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0ce801e9d0 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cec0aa8b0 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/1848822213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ce8031050 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0d0c8a4b80 10 client.15582 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e068654e0 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/1848822213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec0aa8b0 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/1848822213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0ce8026020 con 0x563e066cbe40 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0d0c8a4b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0d0c8a4b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.530+0000 7f0d0c8a4b80 20 client.15582 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.531+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e06867b20 0x563e06869f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.531+0000 7f0d0c8a4b80 4 client.15582 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.531+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e06779320 con 0x563e06867b20 2024-05-14T15:25:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.531+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e06867b20 0x563e06869f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.531+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e06867b20 0x563e06869f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76bed29df8f13c2 server_cookie=592ee16444e4f6a1 in_seq=0 out_seq=0 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.531+0000 7f0cd6ffd700 10 client.15582 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/1848822213 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e06779320 con 0x563e06867b20 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0cd6ffd700 10 client.15582 handle_command_reply: tid=0 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 client.15582 shutdown 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 msgr2=0x563e06854f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 0x563e06854f10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80116f0 tx=0x7f0ce80047c0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 shutdown_connections 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e06867b20 0x563e06869f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e06855990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e066cbe40 0x563e06854f10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/1848822213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06855450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.532+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 >> 172.21.15.5:0/1848822213 conn(0x563e06713630 msgr2=0x563e068468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.533+0000 7f0cd6ffd700 0 client.15582 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.533+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 shutdown_connections 2024-05-14T15:25:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.533+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/1848822213 wait complete. 2024-05-14T15:25:53.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.533+0000 7f0d0c8a4b80 20 client.15582 trim_cache size 0 max 16384 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.548+0000 7f0d0c8a4b80 1 Processor -- start 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.548+0000 7f0d0c8a4b80 1 -- start start 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.548+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e066cbe40 0x563e068557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e06863930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e0686f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b0be0 con 0x563e06849e10 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ae470 con 0x563e066cbe40 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ac2c0 con 0x563e06857f50 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e06863930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e06863930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53824/0 (socket says 172.21.15.5:53824) 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2961693769 learned_addr learned my addr 172.21.15.5:0/2961693769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e066cbe40 0x563e068557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e0686f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2961693769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 msgr2=0x563e0686f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e0686f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2961693769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e066cbe40 msgr2=0x563e068557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e066cbe40 0x563e068557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2961693769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e068409e0 con 0x563e06849e10 2024-05-14T15:25:53.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.549+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e06863930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009990 tx=0x7f0cec003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2702d6a5baa967a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.550+0000 7f0cdd7fa700 1 -- 172.21.15.5:0/2961693769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0e0ca0 con 0x563e06849e10 2024-05-14T15:25:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.550+0000 7f0cdd7fa700 1 -- 172.21.15.5:0/2961693769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec0b3680 con 0x563e06849e10 2024-05-14T15:25:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.550+0000 7f0cdd7fa700 1 -- 172.21.15.5:0/2961693769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0abe40 con 0x563e06849e10 2024-05-14T15:25:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.550+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2961693769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 msgr2=0x563e06863930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.550+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e06863930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009990 tx=0x7f0cec003780 comp rx=0 tx=0).stop 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2961693769 shutdown_connections 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e06857f50 0x563e0686f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e066cbe40 0x563e068557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2961693769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06849e10 0x563e06863930 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2961693769 >> 172.21.15.5:0/2961693769 conn(0x563e06713630 msgr2=0x563e06850560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2961693769 shutdown_connections 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2961693769 wait complete. 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.551+0000 7f0d0c8a4b80 1 -- start start 2024-05-14T15:25:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 Processor -- start 2024-05-14T15:25:53.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e0685fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06860370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 0x563e0685b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ac930 con 0x563e06857f50 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81adda0 con 0x563e06849e10 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ad000 con 0x563e066cbe40 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0d0c8a4b80 10 client.0 _ll_register_callbacks cb 0x563e06703950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 0x563e0685b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06860370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06860370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:33650/0 (socket says 172.21.15.5:33650) 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 0x563e0685b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53836/0 (socket says 172.21.15.5:53836) 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2650397431 learned_addr learned my addr 172.21.15.5:0/2650397431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.552+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e0685fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2650397431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 msgr2=0x563e0685fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e0685fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2650397431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 msgr2=0x563e0685b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 0x563e0685b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf6ffd700 1 -- 172.21.15.5:0/2650397431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e0685bb30 con 0x563e06849e10 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf7fff700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 0x563e0685b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e0685fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cf6ffd700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06860370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b3840 tx=0x7f0cec0227a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac90a079e406209 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/2650397431 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0c3070 con 0x563e06849e10 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/2650397431 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f0cec0b1110 con 0x563e06849e10 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e067733c0 con 0x563e06849e10 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/2650397431 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0cec0aafb0 con 0x563e06849e10 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0d0c8a4b80 10 client.25262 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.553+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e068d1e10 con 0x563e06849e10 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/2650397431 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563e067733c0 con 0x563e06849e10 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/2650397431 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0cec0abc70 con 0x563e06849e10 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0d0c8a4b80 10 client.25262 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0d0c8a4b80 10 client.25262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0d0c8a4b80 20 client.25262 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e068d23c0 0x563e068d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0d0c8a4b80 4 client.25262 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e065eb250 con 0x563e068d23c0 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.554+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e068d23c0 0x563e068d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0cf77fe700 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e068d23c0 0x563e068d47e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396e251776d27d80 server_cookie=5b10c716e8bbb5a8 in_seq=0 out_seq=0 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0cd6ffd700 10 client.25262 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0cd6ffd700 1 -- 172.21.15.5:0/2650397431 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e065eb250 con 0x563e068d23c0 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0cd6ffd700 10 client.25262 handle_command_reply: tid=0 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 client.25262 shutdown 2024-05-14T15:25:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 msgr2=0x563e06860370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06860370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0b3840 tx=0x7f0cec0227a0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 shutdown_connections 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563e068d23c0 0x563e068d47e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e06857f50 0x563e0685b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563e066cbe40 0x563e0685fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 --2- 172.21.15.5:0/2650397431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e06849e10 0x563e06860370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 >> 172.21.15.5:0/2650397431 conn(0x563e06713630 msgr2=0x563e0683ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0cd6ffd700 0 client.25262 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 shutdown_connections 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.555+0000 7f0d0c8a4b80 1 -- 172.21.15.5:0/2650397431 wait complete. 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0d0c8a4b80 20 client.25262 trim_cache size 0 max 16384 2024-05-14T15:25:53.560 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce00b2b10 msgr2=0x7f0ce00b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce00b2b10 0x7f0ce00b5020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005950 tx=0x7f0ce800b410 comp rx=0 tx=0).stop 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 msgr2=0x7f0cf819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf819a2d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006010 tx=0x7f0cec0047b0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 shutdown_connections 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce00b2b10 0x7f0ce00b5020 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cf8108750 0x7f0cf819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf8103c60 0x7f0cf8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 --2- 172.21.15.5:0/1758748038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cf8107e00 0x7f0cf819a2d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 >> 172.21.15.5:0/1758748038 conn(0x7f0cf80fe6f0 msgr2=0x7f0cf81096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 shutdown_connections 2024-05-14T15:25:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:53.556+0000 7f0cfde90700 1 -- 172.21.15.5:0/1758748038 wait complete. 2024-05-14T15:25:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:53 smithi144 ceph-mon[35650]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-14T15:25:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:53 smithi193 ceph-mon[36001]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-14T15:25:54.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:53 smithi005 ceph-mon[31559]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-14T15:25:55.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:25:55.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:25:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:56.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:55 smithi005 ceph-mon[31559]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-14T15:25:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:55 smithi193 ceph-mon[36001]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-14T15:25:56.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:55 smithi144 ceph-mon[35650]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-14T15:25:57.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 -- 172.21.15.5:0/3905716965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d641045e0 msgr2=0x7f7d641049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 --2- 172.21.15.5:0/3905716965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d641045e0 0x7f7d641049f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400b0d0 tx=0x7f7d5400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 -- 172.21.15.5:0/3905716965 shutdown_connections 2024-05-14T15:25:57.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 --2- 172.21.15.5:0/3905716965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d64109860 0x7f7d641111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 --2- 172.21.15.5:0/3905716965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d641045e0 0x7f7d641049f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 --2- 172.21.15.5:0/3905716965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d64108f10 0x7f7d64109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.495+0000 7f7d69730700 1 -- 172.21.15.5:0/3905716965 >> 172.21.15.5:0/3905716965 conn(0x7f7d640ffe70 msgr2=0x7f7d641022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.496+0000 7f7d69730700 1 -- 172.21.15.5:0/3905716965 shutdown_connections 2024-05-14T15:25:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.496+0000 7f7d69730700 1 -- 172.21.15.5:0/3905716965 wait complete. 2024-05-14T15:25:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.496+0000 7f7d69730700 1 Processor -- start 2024-05-14T15:25:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.496+0000 7f7d69730700 1 -- start start 2024-05-14T15:25:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d69730700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d641045e0 0x7f7d6419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d69730700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d64108f10 0x7f7d6419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d69730700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 0x7f7d6419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d69730700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64114500 con 0x7f7d64109860 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d69730700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64114380 con 0x7f7d64108f10 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d69730700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64114680 con 0x7f7d641045e0 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 0x7f7d6419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 0x7f7d6419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33860/0 (socket says 172.21.15.5:33860) 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 -- 172.21.15.5:0/1019230847 learned_addr learned my addr 172.21.15.5:0/1019230847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d641045e0 0x7f7d6419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d627fc700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d64108f10 0x7f7d6419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 -- 172.21.15.5:0/1019230847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d641045e0 msgr2=0x7f7d6419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d641045e0 0x7f7d6419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 -- 172.21.15.5:0/1019230847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d64108f10 msgr2=0x7f7d6419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d64108f10 0x7f7d6419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.497+0000 7f7d637fe700 1 -- 172.21.15.5:0/1019230847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6419fd40 con 0x7f7d64109860 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.498+0000 7f7d637fe700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 0x7f7d6419f6a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00d980 tx=0x7f7d5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.498+0000 7f7d4bfff700 1 -- 172.21.15.5:0/1019230847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c017070 con 0x7f7d64109860 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.498+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d641a0030 con 0x7f7d64109860 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.498+0000 7f7d4bfff700 1 -- 172.21.15.5:0/1019230847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c005020 con 0x7f7d64109860 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.498+0000 7f7d4bfff700 1 -- 172.21.15.5:0/1019230847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00fa10 con 0x7f7d64109860 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.498+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a7950 con 0x7f7d64109860 2024-05-14T15:25:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.500+0000 7f7d4bfff700 1 -- 172.21.15.5:0/1019230847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7f7d5c00fbb0 con 0x7f7d64109860 2024-05-14T15:25:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.501+0000 7f7d4bfff700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7d4c0b2b10 0x7f7d4c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.501+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7d4c0b2b10 0x7f7d4c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.501+0000 7f7d4bfff700 1 -- 172.21.15.5:0/1019230847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f7d5c0e2ac0 con 0x7f7d64109860 2024-05-14T15:25:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.502+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7d4c0b2b10 0x7f7d4c0b5020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009900 tx=0x7f7d5400b040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.502+0000 7f7d78144b80 1 Processor -- start 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.502+0000 7f7d78144b80 1 -- start start 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.502+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b8239c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.502+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b8233d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.502+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b82346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d78144b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625b81519c0 con 0x5625b80b1ec0 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d78144b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625b80b6e20 con 0x5625b823a160 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d78144b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625b814d980 con 0x5625b82342a0 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b8239c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b8239c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33868/0 (socket says 172.21.15.5:33868) 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4072363809 learned_addr learned my addr 172.21.15.5:0/4072363809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d637fe700 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b82346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d627fc700 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b8233d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4072363809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 msgr2=0x5625b82346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b82346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4072363809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 msgr2=0x5625b8233d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b8233d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4072363809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b809e950 con 0x5625b80b1ec0 2024-05-14T15:25:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b8239c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400d010 tx=0x7f7d5401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac6b5c4cc81377 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d497fa700 1 -- 172.21.15.5:0/4072363809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5402c030 con 0x5625b80b1ec0 2024-05-14T15:25:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d497fa700 1 -- 172.21.15.5:0/4072363809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d54007a70 con 0x5625b80b1ec0 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.503+0000 7f7d497fa700 1 -- 172.21.15.5:0/4072363809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d540056c0 con 0x5625b80b1ec0 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 -- 172.21.15.5:0/4072363809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 msgr2=0x5625b8239c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b8239c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400d010 tx=0x7f7d5401d000 comp rx=0 tx=0).stop 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 -- 172.21.15.5:0/4072363809 shutdown_connections 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b82346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b8239c20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4072363809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b8233d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 -- 172.21.15.5:0/4072363809 >> 172.21.15.5:0/4072363809 conn(0x5625b80f96b0 msgr2=0x5625b822aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 -- 172.21.15.5:0/4072363809 shutdown_connections 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.504+0000 7f7d78144b80 1 -- 172.21.15.5:0/4072363809 wait complete. 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 -- start start 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 Processor -- start 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b823da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b823df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b823e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625b8240c00 con 0x5625b80b1ec0 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625b810af50 con 0x5625b823a160 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5625b8240d80 con 0x5625b82342a0 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d78144b80 10 client.0 _ll_register_callbacks cb 0x5625b80e99d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b823da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b823da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33874/0 (socket says 172.21.15.5:33874) 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.505+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4235732112 learned_addr learned my addr 172.21.15.5:0/4235732112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d637fe700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b823e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d627fc700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b823df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4235732112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 msgr2=0x5625b823df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b823df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 msgr2=0x5625b823e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b823e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d62ffd700 1 -- 172.21.15.5:0/4235732112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b81087e0 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d627fc700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b823df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d637fe700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b823e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b823da40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009e60 tx=0x7f7d54007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8740ab73312820 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d46ff5700 1 -- 172.21.15.5:0/4235732112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5402c070 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d46ff5700 1 -- 172.21.15.5:0/4235732112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f7d54005580 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d54007a70 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.506+0000 7f7d46ff5700 1 -- 172.21.15.5:0/4235732112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d54042050 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 10 client.15600 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625b824b840 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d46ff5700 1 -- 172.21.15.5:0/4235732112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d54007a70 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d46ff5700 1 -- 172.21.15.5:0/4235732112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7d54011070 con 0x5625b80b1ec0 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:57.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 20 client.15600 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b824dd10 0x5625b8250130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 4 client.15600 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625b815f3a0 con 0x5625b824dd10 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.507+0000 7f7d627fc700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b824dd10 0x5625b8250130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.508+0000 7f7d627fc700 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b824dd10 0x5625b8250130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9668b9d9118bf64 server_cookie=6eabd2766a5f3385 in_seq=0 out_seq=0 2024-05-14T15:25:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.508+0000 7f7d46ff5700 10 client.15600 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d46ff5700 1 -- 172.21.15.5:0/4235732112 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625b815f3a0 con 0x5625b824dd10 2024-05-14T15:25:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d46ff5700 10 client.15600 handle_command_reply: tid=0 2024-05-14T15:25:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 client.15600 shutdown 2024-05-14T15:25:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 msgr2=0x5625b823da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b823da40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009e60 tx=0x7f7d54007640 comp rx=0 tx=0).stop 2024-05-14T15:25:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 shutdown_connections 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b824dd10 0x5625b8250130 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b823a160 0x5625b823e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b823da40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 --2- 172.21.15.5:0/4235732112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b82342a0 0x5625b823df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d46ff5700 0 client.15600 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 >> 172.21.15.5:0/4235732112 conn(0x5625b80f96b0 msgr2=0x5625b822e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 shutdown_connections 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 1 -- 172.21.15.5:0/4235732112 wait complete. 2024-05-14T15:25:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.509+0000 7f7d78144b80 20 client.15600 trim_cache size 0 max 16384 2024-05-14T15:25:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 Processor -- start 2024-05-14T15:25:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 -- start start 2024-05-14T15:25:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b822f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b82342a0 0x5625b8249fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b823e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b0c60 con 0x5625b80b1ec0 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ae4f0 con 0x5625b82342a0 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d78144b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ac340 con 0x5625b823a160 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b822f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b822f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33886/0 (socket says 172.21.15.5:33886) 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 -- 172.21.15.5:0/2522790754 learned_addr learned my addr 172.21.15.5:0/2522790754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d637fe700 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b823e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d627fc700 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b82342a0 0x5625b8249fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 -- 172.21.15.5:0/2522790754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 msgr2=0x5625b823e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b823e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 -- 172.21.15.5:0/2522790754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b82342a0 msgr2=0x5625b8249fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b82342a0 0x5625b8249fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d62ffd700 1 -- 172.21.15.5:0/2522790754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b822b6e0 con 0x5625b80b1ec0 2024-05-14T15:25:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.532+0000 7f7d627fc700 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b82342a0 0x5625b8249fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.533+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b822f370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000f80 tx=0x7f7d54005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb308d77f47019e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.533+0000 7f7d48ff9700 1 -- 172.21.15.5:0/2522790754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54020070 con 0x5625b80b1ec0 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.533+0000 7f7d48ff9700 1 -- 172.21.15.5:0/2522790754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5402d040 con 0x5625b80b1ec0 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.533+0000 7f7d48ff9700 1 -- 172.21.15.5:0/2522790754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5402c040 con 0x5625b80b1ec0 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 -- 172.21.15.5:0/2522790754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 msgr2=0x5625b822f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b822f370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000f80 tx=0x7f7d54005e70 comp rx=0 tx=0).stop 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 -- 172.21.15.5:0/2522790754 shutdown_connections 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b823e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b80b1ec0 0x5625b822f370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2522790754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b82342a0 0x5625b8249fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 -- 172.21.15.5:0/2522790754 >> 172.21.15.5:0/2522790754 conn(0x5625b80f96b0 msgr2=0x5625b823c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 -- 172.21.15.5:0/2522790754 shutdown_connections 2024-05-14T15:25:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 -- 172.21.15.5:0/2522790754 wait complete. 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 -- start start 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.534+0000 7f7d78144b80 1 Processor -- start 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b80b1ec0 0x5625b82535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 0x5625b8251950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b8251e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ac9b0 con 0x5625b82342a0 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ade20 con 0x5625b80b1ec0 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ad080 con 0x5625b823a160 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d78144b80 10 client.0 _ll_register_callbacks cb 0x5625b80e99d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 0x5625b8251950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 0x5625b8251950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33902/0 (socket says 172.21.15.5:33902) 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 -- 172.21.15.5:0/2138056557 learned_addr learned my addr 172.21.15.5:0/2138056557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d637fe700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b8251e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b80b1ec0 0x5625b82535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 -- 172.21.15.5:0/2138056557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 msgr2=0x5625b8251e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b8251e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 -- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b80b1ec0 msgr2=0x5625b82535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b80b1ec0 0x5625b82535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 -- 172.21.15.5:0/2138056557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b8252570 con 0x5625b82342a0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d627fc700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 0x5625b8251950 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d64107d60 tx=0x7f7d5801d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fb5d77f1803385 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.535+0000 7f7d46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d46ff5700 1 -- 172.21.15.5:0/2138056557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58026670 con 0x5625b82342a0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d46ff5700 1 -- 172.21.15.5:0/2138056557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f7d58026810 con 0x5625b82342a0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5402d040 con 0x5625b82342a0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d46ff5700 1 -- 172.21.15.5:0/2138056557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d5802f050 con 0x5625b82342a0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 10 client.15606 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625b82b9160 con 0x5625b82342a0 2024-05-14T15:25:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d46ff5700 1 -- 172.21.15.5:0/2138056557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5402d040 con 0x5625b82342a0 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d46ff5700 1 -- 172.21.15.5:0/2138056557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7d5801dce0 con 0x5625b82342a0 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 20 client.15606 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b82b9850 0x5625b82bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 4 client.15606 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.536+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5625b8159440 con 0x5625b82b9850 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.537+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b82b9850 0x5625b82bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.537+0000 7f7d62ffd700 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b82b9850 0x5625b82bbc70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4c4467750e8429 server_cookie=e92d617ea005ab62 in_seq=0 out_seq=0 2024-05-14T15:25:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.537+0000 7f7d46ff5700 10 client.15606 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d46ff5700 1 -- 172.21.15.5:0/2138056557 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5625b8159440 con 0x5625b82b9850 2024-05-14T15:25:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d46ff5700 10 client.15606 handle_command_reply: tid=0 2024-05-14T15:25:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 client.15606 shutdown 2024-05-14T15:25:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 msgr2=0x5625b8251950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 0x5625b8251950 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d64107d60 tx=0x7f7d5801d420 comp rx=0 tx=0).stop 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 shutdown_connections 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5625b82b9850 0x5625b82bbc70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5625b823a160 0x5625b8251e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625b80b1ec0 0x5625b82535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 --2- 172.21.15.5:0/2138056557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b82342a0 0x5625b8251950 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 >> 172.21.15.5:0/2138056557 conn(0x5625b80f96b0 msgr2=0x5625b822f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d46ff5700 0 client.15606 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 shutdown_connections 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.538+0000 7f7d78144b80 1 -- 172.21.15.5:0/2138056557 wait complete. 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d78144b80 20 client.15606 trim_cache size 0 max 16384 2024-05-14T15:25:57.543 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:25:57.544 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:25:57.544 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "325722ca-be7f-4e4a-92b3-6e569103f8fb", 2024-05-14T15:25:57.544 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:25:57.544 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:25:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7d4c0b2b10 msgr2=0x7f7d4c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7d4c0b2b10 0x7f7d4c0b5020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009900 tx=0x7f7d5400b040 comp rx=0 tx=0).stop 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 msgr2=0x7f7d6419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 0x7f7d6419f6a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00d980 tx=0x7f7d5c00dc90 comp rx=0 tx=0).stop 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 shutdown_connections 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7d4c0b2b10 0x7f7d4c0b5020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d64109860 0x7f7d6419f6a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d641045e0 0x7f7d6419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 --2- 172.21.15.5:0/1019230847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d64108f10 0x7f7d6419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 >> 172.21.15.5:0/1019230847 conn(0x7f7d640ffe70 msgr2=0x7f7d641086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 shutdown_connections 2024-05-14T15:25:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:25:57.539+0000 7f7d69730700 1 -- 172.21.15.5:0/1019230847 wait complete. 2024-05-14T15:25:58.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:57 smithi193 ceph-mon[36001]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-14T15:25:58.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:57 smithi005 ceph-mon[31559]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-14T15:25:58.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:57 smithi144 ceph-mon[35650]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-14T15:25:58.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:25:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:25:59.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:58 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:25:59.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:58 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:59.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:58 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:25:59.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:58 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:58 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:25:59.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:58 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:59.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:58 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:25:59.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:58 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:58 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:25:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:58 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:58 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:25:59.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:58 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:25:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/1699616136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc109040 msgr2=0x7fa3dc102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/1699616136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc109040 0x7fa3dc102c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009900 tx=0x7fa3d002a410 comp rx=0 tx=0).stop 2024-05-14T15:25:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/1699616136 shutdown_connections 2024-05-14T15:25:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/1699616136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc1032f0 0x7fa3dc10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/1699616136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc109040 0x7fa3dc102c80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/1699616136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/1699616136 >> 172.21.15.5:0/1699616136 conn(0x7fa3dc0782e0 msgr2=0x7fa3dc078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/1699616136 shutdown_connections 2024-05-14T15:25:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.401+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/1699616136 wait complete. 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.402+0000 7fa3e0e6e640 1 Processor -- start 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.402+0000 7fa3e0e6e640 1 -- start start 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3e0e6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc1032f0 0x7fa3dc19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3e0e6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3e0e6e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc109040 0x7fa3dc19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3e0e6e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc112d90 con 0x7fa3dc108ba0 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3e0e6e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc112c10 con 0x7fa3dc109040 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3e0e6e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc112f10 con 0x7fa3dc1032f0 2024-05-14T15:25:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33922/0 (socket says 172.21.15.5:33922) 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3da59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc1032f0 0x7fa3dc19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/3733397893 learned_addr learned my addr 172.21.15.5:0/3733397893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc109040 0x7fa3dc19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3da59c640 1 -- 172.21.15.5:0/3733397893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc109040 msgr2=0x7fa3dc19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3da59c640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc109040 0x7fa3dc19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3da59c640 1 -- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 msgr2=0x7fa3dc19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3da59c640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3da59c640 1 -- 172.21.15.5:0/3733397893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc1a6a80 con 0x7fa3dc1032f0 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc109040 0x7fa3dc19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.403+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.404+0000 7fa3da59c640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc1032f0 0x7fa3dc19b770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00e9c0 tx=0x7fa3cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.404+0000 7fa3cb7fe640 1 -- 172.21.15.5:0/3733397893 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00ccc0 con 0x7fa3dc1032f0 2024-05-14T15:25:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.404+0000 7fa3cb7fe640 1 -- 172.21.15.5:0/3733397893 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc005020 con 0x7fa3dc1032f0 2024-05-14T15:25:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.404+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc1a6d70 con 0x7fa3dc1032f0 2024-05-14T15:25:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.404+0000 7fa3cb7fe640 1 -- 172.21.15.5:0/3733397893 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004a00 con 0x7fa3dc1032f0 2024-05-14T15:25:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.404+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc1a71b0 con 0x7fa3dc1032f0 2024-05-14T15:25:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 Processor -- start 2024-05-14T15:25:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 -- start start 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c52be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c53660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a208bb85d0 con 0x55a2088914d0 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a208b71890 con 0x55a2088b21b0 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a208bb4590 con 0x55a2088892f0 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c53660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c53660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33926/0 (socket says 172.21.15.5:33926) 2024-05-14T15:25:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1246541060 learned_addr learned my addr 172.21.15.5:0/1246541060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.407+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c52be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1246541060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 msgr2=0x55a208c53120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c53120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1246541060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 msgr2=0x55a208c52be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c52be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1246541060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2088ca210 con 0x55a2088914d0 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c53660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0006010 tx=0x7fa3d0005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f948cacc4ede51d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3c8ff9640 1 -- 172.21.15.5:0/1246541060 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0035040 con 0x55a2088914d0 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3c8ff9640 1 -- 172.21.15.5:0/1246541060 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d0031720 con 0x55a2088914d0 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3c8ff9640 1 -- 172.21.15.5:0/1246541060 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00049e0 con 0x55a2088914d0 2024-05-14T15:25:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.408+0000 7fa3cb7fe640 1 -- 172.21.15.5:0/3733397893 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fa3cc004ba0 con 0x7fa3dc1032f0 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.409+0000 7fa3cb7fe640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa3c40b2a70 0x7fa3c40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1246541060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 msgr2=0x55a208c53660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c53660 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0006010 tx=0x7fa3d0005e20 comp rx=0 tx=0).stop 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa3c40b2a70 0x7fa3c40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1246541060 shutdown_connections 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3cb7fe640 1 -- 172.21.15.5:0/3733397893 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa3cc0e2c50 con 0x7fa3dc1032f0 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c53120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c53660 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1246541060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c52be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1246541060 >> 172.21.15.5:0/1246541060 conn(0x55a2088a6300 msgr2=0x55a208c4f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1246541060 shutdown_connections 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1246541060 wait complete. 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.410+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa3c40b2a70 0x7fa3c40b4f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a208c4fd00 tx=0x7fa3d4009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 -- start start 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 Processor -- start 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 0x55a208c5d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c5d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a208c5fcb0 con 0x55a2088892f0 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a208b75370 con 0x55a2088b21b0 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a208c5fe30 con 0x55a2088914d0 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.411+0000 7fa3e7e04740 10 client.0 _ll_register_callbacks cb 0x55a2088c9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 0x55a208c5d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 0x55a208c5d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33942/0 (socket says 172.21.15.5:33942) 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 -- 172.21.15.5:0/1582709011 learned_addr learned my addr 172.21.15.5:0/1582709011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c5d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 -- 172.21.15.5:0/1582709011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 msgr2=0x55a208c5d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c5d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 -- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 msgr2=0x55a208c69160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c69160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 -- 172.21.15.5:0/1582709011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a208996620 con 0x55a2088892f0 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c5d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 0x55a208c5d1b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0e05e0 tx=0x7fa3cc0a9090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e340fe86f30c6344 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.412+0000 7fa3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1582709011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc01c040 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1582709011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa3cc0cabd0 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d0031720 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1582709011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc0aa020 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3e7e04740 10 client.15618 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a208c6c090 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1582709011 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d0031720 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.413+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1582709011 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa3cc06f130 con 0x55a2088892f0 2024-05-14T15:25:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3e7e04740 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3e7e04740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3e7e04740 20 client.15618 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a208870d80 0x55a208c6f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3e7e04740 4 client.15618 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a208bc61e0 con 0x55a208870d80 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a208870d80 0x55a208c6f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.414+0000 7fa3c1ffb640 10 client.15618 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.415+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a208870d80 0x55a208c6f560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c5d79b9c885344 server_cookie=ca88fbc3a4713140 in_seq=0 out_seq=0 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.415+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1582709011 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a208bc61e0 con 0x55a208870d80 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.415+0000 7fa3c1ffb640 10 client.15618 handle_command_reply: tid=0 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 client.15618 shutdown 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 msgr2=0x55a208c5d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 0x55a208c5d1b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0e05e0 tx=0x7fa3cc0a9090 comp rx=0 tx=0).stop 2024-05-14T15:25:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 shutdown_connections 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a208870d80 0x55a208c6f560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c5d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c69160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1582709011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088892f0 0x55a208c5d1b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 >> 172.21.15.5:0/1582709011 conn(0x55a2088a6300 msgr2=0x55a208c54900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3c1ffb640 0 client.15618 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 shutdown_connections 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1582709011 wait complete. 2024-05-14T15:25:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.416+0000 7fa3e7e04740 20 client.15618 trim_cache size 0 max 16384 2024-05-14T15:25:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.431+0000 7fa3e7e04740 1 Processor -- start 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 -- start start 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c57840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c4e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1b3c00 con 0x55a2088914d0 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1b14d0 con 0x55a2088b21b0 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1af320 con 0x55a2088892f0 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c5f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33956/0 (socket says 172.21.15.5:33956) 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1950774443 learned_addr learned my addr 172.21.15.5:0/1950774443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c57840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c4e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1950774443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 msgr2=0x55a208c57840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c57840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1950774443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 msgr2=0x55a208c4e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c4e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3d9d9b640 1 -- 172.21.15.5:0/1950774443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a208c5f840 con 0x55a2088914d0 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.432+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c57840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.433+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c5f0f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005500 tx=0x7fa3d0005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9730f79ec2e05797 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.433+0000 7fa3c3fff640 1 -- 172.21.15.5:0/1950774443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0056070 con 0x55a2088914d0 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.433+0000 7fa3c3fff640 1 -- 172.21.15.5:0/1950774443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d0004050 con 0x55a2088914d0 2024-05-14T15:25:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.433+0000 7fa3c3fff640 1 -- 172.21.15.5:0/1950774443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0035040 con 0x55a2088914d0 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1950774443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 msgr2=0x55a208c5f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c5f0f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005500 tx=0x7fa3d0005530 comp rx=0 tx=0).stop 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1950774443 shutdown_connections 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088b21b0 0x55a208c4e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088914d0 0x55a208c5f0f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1950774443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088892f0 0x55a208c57840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1950774443 >> 172.21.15.5:0/1950774443 conn(0x55a2088a6300 msgr2=0x55a208c4d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1950774443 shutdown_connections 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1950774443 wait complete. 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 -- start start 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.434+0000 7fa3e7e04740 1 Processor -- start 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088892f0 0x55a208c606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 0x55a208cd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1af990 con 0x55a2088b21b0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1b0e00 con 0x55a2088892f0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1b0060 con 0x55a2088914d0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3e7e04740 10 client.0 _ll_register_callbacks cb 0x7fa3cc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 0x55a208cd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 0x55a208cd5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33962/0 (socket says 172.21.15.5:33962) 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 -- 172.21.15.5:0/1524675373 learned_addr learned my addr 172.21.15.5:0/1524675373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088892f0 0x55a208c606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3d9d9b640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 -- 172.21.15.5:0/1524675373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 msgr2=0x55a208c60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c60be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 -- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088892f0 msgr2=0x55a208c606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088892f0 0x55a208c606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.435+0000 7fa3dad9d640 1 -- 172.21.15.5:0/1524675373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a208cd5db0 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3dad9d640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 0x55a208cd5840 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4010cf0 tx=0x7fa3d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4744a964bf63d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1524675373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d401ebb0 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d0004050 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1524675373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa3d401ed50 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1524675373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa3d4034050 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 10 client.15624 fetch_fsmap learned FSMap version 26 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a208cd70d0 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1524675373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d0004050 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1524675373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d4005720 con 0x55a2088b21b0 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 20 client.15624 populate_metadata read hostname 'smithi005' 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a2088461c0 0x55a208cd98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:25:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 4 client.15624 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:25:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.436+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a208bc0050 con 0x55a2088461c0 2024-05-14T15:25:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.437+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a2088461c0 0x55a208cd98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:25:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.437+0000 7fa3da59c640 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a2088461c0 0x55a208cd98c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157b569852f76945 server_cookie=7ca5011ff7477b59 in_seq=0 out_seq=0 2024-05-14T15:25:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.437+0000 7fa3c1ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3c1ffb640 1 -- 172.21.15.5:0/1524675373 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a208bc0050 con 0x55a2088461c0 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3c1ffb640 10 client.15624 handle_command_reply: tid=0 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 client.15624 shutdown 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 msgr2=0x55a208cd5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 0x55a208cd5840 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4010cf0 tx=0x7fa3d4004a40 comp rx=0 tx=0).stop 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 shutdown_connections 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a2088461c0 0x55a208cd98c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2088914d0 0x55a208c60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2088b21b0 0x55a208cd5840 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 --2- 172.21.15.5:0/1524675373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2088892f0 0x55a208c606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3c1ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 >> 172.21.15.5:0/1524675373 conn(0x55a2088a6300 msgr2=0x55a208c533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 shutdown_connections 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.441+0000 7fa3e7e04740 1 -- 172.21.15.5:0/1524675373 wait complete. 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e7e04740 20 client.15624 trim_cache size 0 max 16384 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:25:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:25:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa3c40b2a70 msgr2=0x7fa3c40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa3c40b2a70 0x7fa3c40b4f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a208c4fd00 tx=0x7fa3d4009500 comp rx=0 tx=0).stop 2024-05-14T15:25:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc1032f0 msgr2=0x7fa3dc19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:25:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc1032f0 0x7fa3dc19b770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00e9c0 tx=0x7fa3cc00ee80 comp rx=0 tx=0).stop 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 shutdown_connections 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa3c40b2a70 0x7fa3c40b4f80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3dc109040 0x7fa3dc19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3dc108ba0 0x7fa3dc19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 --2- 172.21.15.5:0/3733397893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc1032f0 0x7fa3dc19b770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 >> 172.21.15.5:0/3733397893 conn(0x7fa3dc0782e0 msgr2=0x7fa3dc1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 shutdown_connections 2024-05-14T15:25:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:25:59.442+0000 7fa3e0e6e640 1 -- 172.21.15.5:0/3733397893 wait complete. 2024-05-14T15:26:00.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:25:59 smithi005 ceph-mon[31559]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T15:26:00.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:25:59 smithi144 ceph-mon[35650]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T15:26:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:25:59 smithi193 ceph-mon[36001]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T15:26:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 -- 172.21.15.5:0/2889497160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54104610 msgr2=0x7fee54104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 --2- 172.21.15.5:0/2889497160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54104610 0x7fee54104a20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fee440099c0 tx=0x7fee44009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 -- 172.21.15.5:0/2889497160 shutdown_connections 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 --2- 172.21.15.5:0/2889497160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee54109890 0x7fee541111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 --2- 172.21.15.5:0/2889497160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54104610 0x7fee54104a20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 --2- 172.21.15.5:0/2889497160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee54108f40 0x7fee54109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 -- 172.21.15.5:0/2889497160 >> 172.21.15.5:0/2889497160 conn(0x7fee5407af70 msgr2=0x7fee5407b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 -- 172.21.15.5:0/2889497160 shutdown_connections 2024-05-14T15:26:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.504+0000 7fee58ac9700 1 -- 172.21.15.5:0/2889497160 wait complete. 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.505+0000 7fee58ac9700 1 Processor -- start 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.505+0000 7fee58ac9700 1 -- start start 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee58ac9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee54104610 0x7fee5419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee58ac9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee54108f40 0x7fee5419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee58ac9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 0x7fee5419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee58ac9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee541144c0 con 0x7fee54109890 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee58ac9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee54114340 con 0x7fee54108f40 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee58ac9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee54114640 con 0x7fee54104610 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 0x7fee5419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 0x7fee5419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33974/0 (socket says 172.21.15.5:33974) 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 -- 172.21.15.5:0/1815068226 learned_addr learned my addr 172.21.15.5:0/1815068226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 -- 172.21.15.5:0/1815068226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee54104610 msgr2=0x7fee5419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee54104610 0x7fee5419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 -- 172.21.15.5:0/1815068226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee54108f40 msgr2=0x7fee5419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee54108f40 0x7fee5419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 -- 172.21.15.5:0/1815068226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5419fc30 con 0x7fee54109890 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.506+0000 7fee52d9d700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 0x7fee5419f640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fee4c007ee0 tx=0x7fee4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.507+0000 7fee437fe700 1 -- 172.21.15.5:0/1815068226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00dbd0 con 0x7fee54109890 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.507+0000 7fee437fe700 1 -- 172.21.15.5:0/1815068226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee4c00dd70 con 0x7fee54109890 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.507+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5419ff20 con 0x7fee54109890 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.507+0000 7fee437fe700 1 -- 172.21.15.5:0/1815068226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c005990 con 0x7fee54109890 2024-05-14T15:26:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.507+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee541a0460 con 0x7fee54109890 2024-05-14T15:26:00.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.510+0000 7fee437fe700 1 -- 172.21.15.5:0/1815068226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 149878+0+0 (secure 0 0 0) 0x7fee4c01a020 con 0x7fee54109890 2024-05-14T15:26:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.510+0000 7fee437fe700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fee3c0b2b90 0x7fee3c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee674ddb80 1 Processor -- start 2024-05-14T15:26:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee5259c700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fee3c0b2b90 0x7fee3c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee437fe700 1 -- 172.21.15.5:0/1815068226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fee4c0e2410 con 0x7fee54109890 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee674ddb80 1 -- start start 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee5259c700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fee3c0b2b90 0x7fee3c0b50a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee44003800 tx=0x7fee44000bc0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.511+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168b6070 0x5603168ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee674ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603167d3860 con 0x560316733e40 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee674ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5603167727c0 con 0x5603168ba890 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee674ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5603167cf820 con 0x5603168b6070 2024-05-14T15:26:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33984/0 (socket says 172.21.15.5:33984) 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 -- 172.21.15.5:0/2950920064 learned_addr learned my addr 172.21.15.5:0/2950920064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee51d9b700 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168b6070 0x5603168ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 -- 172.21.15.5:0/2950920064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168b6070 msgr2=0x5603168ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168b6070 0x5603168ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee52d9d700 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 -- 172.21.15.5:0/2950920064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 msgr2=0x5603168baca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168baca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 -- 172.21.15.5:0/2950920064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603167208d0 con 0x560316733e40 2024-05-14T15:26:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.512+0000 7fee5259c700 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168b5b30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee44015000 tx=0x7fee4401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588e6451f6805723 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.513+0000 7fee40ff9700 1 -- 172.21.15.5:0/2950920064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44003df0 con 0x560316733e40 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.513+0000 7fee40ff9700 1 -- 172.21.15.5:0/2950920064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee44005500 con 0x560316733e40 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.513+0000 7fee40ff9700 1 -- 172.21.15.5:0/2950920064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440248f0 con 0x560316733e40 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 -- 172.21.15.5:0/2950920064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 msgr2=0x5603168b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168b5b30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee44015000 tx=0x7fee4401d000 comp rx=0 tx=0).stop 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 -- 172.21.15.5:0/2950920064 shutdown_connections 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168baca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168b5b30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2950920064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168b6070 0x5603168ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 -- 172.21.15.5:0/2950920064 >> 172.21.15.5:0/2950920064 conn(0x56031677b600 msgr2=0x5603168ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 -- 172.21.15.5:0/2950920064 shutdown_connections 2024-05-14T15:26:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 -- 172.21.15.5:0/2950920064 wait complete. 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 -- start start 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.514+0000 7fee674ddb80 1 Processor -- start 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168b6070 0x5603168bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168ba890 0x5603168bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603168c27e0 con 0x560316733e40 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560316739c30 con 0x5603168b6070 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5603168c2960 con 0x5603168ba890 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee674ddb80 10 client.0 _ll_register_callbacks cb 0x56031676b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168bf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33994/0 (socket says 172.21.15.5:33994) 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 -- 172.21.15.5:0/470937644 learned_addr learned my addr 172.21.15.5:0/470937644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee52d9d700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168ba890 0x5603168bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 -- 172.21.15.5:0/470937644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168ba890 msgr2=0x5603168bfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168ba890 0x5603168bfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 -- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168b6070 msgr2=0x5603168bfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee51d9b700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168b6070 0x5603168bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168b6070 0x5603168bfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 -- 172.21.15.5:0/470937644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56031676f690 con 0x560316733e40 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee51d9b700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168b6070 0x5603168bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.515+0000 7fee5259c700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168bf570 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee440098e0 tx=0x7fee44005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9210a0653ae4af36 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee39ffb700 1 -- 172.21.15.5:0/470937644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44035070 con 0x560316733e40 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee44005500 con 0x560316733e40 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee39ffb700 1 -- 172.21.15.5:0/470937644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fee440247d0 con 0x560316733e40 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee39ffb700 1 -- 172.21.15.5:0/470937644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee44041050 con 0x560316733e40 2024-05-14T15:26:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 10 client.15636 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603168c27e0 con 0x560316733e40 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee39ffb700 1 -- 172.21.15.5:0/470937644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee44005500 con 0x560316733e40 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee39ffb700 1 -- 172.21.15.5:0/470937644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fee4402e920 con 0x560316733e40 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 20 client.15636 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x5603168d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 4 client.15636 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.516+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603167e1240 con 0x5603168cf6b0 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.517+0000 7fee51d9b700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x5603168d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.517+0000 7fee39ffb700 10 client.15636 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.517+0000 7fee51d9b700 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x5603168d1ad0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e1b9d7335b6728 server_cookie=a279414c8379b976 in_seq=0 out_seq=0 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee39ffb700 1 -- 172.21.15.5:0/470937644 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603167e1240 con 0x5603168cf6b0 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee39ffb700 10 client.15636 handle_command_reply: tid=0 2024-05-14T15:26:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 client.15636 shutdown 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 msgr2=0x5603168bf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168bf570 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee440098e0 tx=0x7fee44005960 comp rx=0 tx=0).stop 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 shutdown_connections 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x5603168d1ad0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5603168ba890 0x5603168bfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560316733e40 0x5603168bf570 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee39ffb700 0 client.15636 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 --2- 172.21.15.5:0/470937644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168b6070 0x5603168bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.518+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 >> 172.21.15.5:0/470937644 conn(0x56031677b600 msgr2=0x5603168b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.519+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 shutdown_connections 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.519+0000 7fee674ddb80 1 -- 172.21.15.5:0/470937644 wait complete. 2024-05-14T15:26:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.519+0000 7fee674ddb80 20 client.15636 trim_cache size 0 max 16384 2024-05-14T15:26:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 Processor -- start 2024-05-14T15:26:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 -- start start 2024-05-14T15:26:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168cb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee541b0c70 con 0x5603168b6070 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee541ae500 con 0x5603168ba890 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee674ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee541ac350 con 0x560316733e40 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee51d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee51d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168bfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34000/0 (socket says 172.21.15.5:34000) 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee51d9b700 1 -- 172.21.15.5:0/1491981289 learned_addr learned my addr 172.21.15.5:0/1491981289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee5259c700 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee52d9d700 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.554+0000 7fee51d9b700 1 -- 172.21.15.5:0/1491981289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 msgr2=0x5603168b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee51d9b700 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee51d9b700 1 -- 172.21.15.5:0/1491981289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 msgr2=0x5603168cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee51d9b700 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee51d9b700 1 -- 172.21.15.5:0/1491981289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603168a4420 con 0x5603168b6070 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee52d9d700 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee51d9b700 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168bfe30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fee54100f80 tx=0x7fee4800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593941a19723ff4d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee3bfff700 1 -- 172.21.15.5:0/1491981289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48013d20 con 0x5603168b6070 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee3bfff700 1 -- 172.21.15.5:0/1491981289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee4801b860 con 0x5603168b6070 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.555+0000 7fee3bfff700 1 -- 172.21.15.5:0/1491981289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48011c30 con 0x5603168b6070 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 -- 172.21.15.5:0/1491981289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 msgr2=0x5603168bfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168bfe30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fee54100f80 tx=0x7fee4800a380 comp rx=0 tx=0).stop 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 -- 172.21.15.5:0/1491981289 shutdown_connections 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 --2- 172.21.15.5:0/1491981289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168bfe30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 -- 172.21.15.5:0/1491981289 >> 172.21.15.5:0/1491981289 conn(0x56031677b600 msgr2=0x5603168ac250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:00.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 -- 172.21.15.5:0/1491981289 shutdown_connections 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 -- 172.21.15.5:0/1491981289 wait complete. 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.556+0000 7fee674ddb80 1 -- start start 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 Processor -- start 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee541ac9c0 con 0x5603168b6070 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee541ade30 con 0x5603168ba890 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee541ad090 con 0x560316733e40 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee674ddb80 10 client.0 _ll_register_callbacks cb 0x56031676b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee51d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee51d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168d66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34012/0 (socket says 172.21.15.5:34012) 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee51d9b700 1 -- 172.21.15.5:0/2756305757 learned_addr learned my addr 172.21.15.5:0/2756305757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee52d9d700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.557+0000 7fee5259c700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee51d9b700 1 -- 172.21.15.5:0/2756305757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 msgr2=0x5603168c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee51d9b700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee51d9b700 1 -- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 msgr2=0x5603168d6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee51d9b700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168d6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee51d9b700 1 -- 172.21.15.5:0/2756305757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603168d73c0 con 0x5603168b6070 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee52d9d700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168d6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee51d9b700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168d66d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fee48005be0 tx=0x7fee480115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fed040803d029c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee39ffb700 1 -- 172.21.15.5:0/2756305757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48031060 con 0x5603168b6070 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee4801b860 con 0x5603168b6070 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee39ffb700 1 -- 172.21.15.5:0/2756305757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fee48013b20 con 0x5603168b6070 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee39ffb700 1 -- 172.21.15.5:0/2756305757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee48038050 con 0x5603168b6070 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee674ddb80 10 client.15642 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:00.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.558+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560316939c10 con 0x5603168b6070 2024-05-14T15:26:00.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee39ffb700 1 -- 172.21.15.5:0/2756305757 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee4801b860 con 0x5603168b6070 2024-05-14T15:26:00.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee39ffb700 1 -- 172.21.15.5:0/2756305757 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fee48004b60 con 0x5603168b6070 2024-05-14T15:26:00.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee674ddb80 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:00.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee674ddb80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:00.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee674ddb80 20 client.15642 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:00.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x56031693c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee674ddb80 4 client.15642 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603167db2e0 con 0x5603168cf6b0 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.559+0000 7fee5259c700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x56031693c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee5259c700 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x56031693c280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3327bcbc1f117a1d server_cookie=8bfa08f6e8df5786 in_seq=0 out_seq=0 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee39ffb700 10 client.15642 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee39ffb700 1 -- 172.21.15.5:0/2756305757 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5603167db2e0 con 0x5603168cf6b0 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee39ffb700 10 client.15642 handle_command_reply: tid=0 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 client.15642 shutdown 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 msgr2=0x5603168d66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168d66d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fee48005be0 tx=0x7fee480115a0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 shutdown_connections 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5603168cf6b0 0x56031693c280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603168ba890 0x5603168d6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560316733e40 0x5603168c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 --2- 172.21.15.5:0/2756305757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603168b6070 0x5603168d66d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 >> 172.21.15.5:0/2756305757 conn(0x56031677b600 msgr2=0x5603168bb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee39ffb700 0 client.15642 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 shutdown_connections 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.560+0000 7fee674ddb80 1 -- 172.21.15.5:0/2756305757 wait complete. 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee674ddb80 20 client.15642 trim_cache size 0 max 16384 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:26:00.565 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fee3c0b2b90 msgr2=0x7fee3c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fee3c0b2b90 0x7fee3c0b50a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee44003800 tx=0x7fee44000bc0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 msgr2=0x7fee5419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 0x7fee5419f640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fee4c007ee0 tx=0x7fee4c00d3b0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 shutdown_connections 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fee3c0b2b90 0x7fee3c0b50a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee54109890 0x7fee5419f640 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee54104610 0x7fee5419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 --2- 172.21.15.5:0/1815068226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee54108f40 0x7fee5419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 >> 172.21.15.5:0/1815068226 conn(0x7fee5407af70 msgr2=0x7fee541019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:00.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.561+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 shutdown_connections 2024-05-14T15:26:00.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:00.562+0000 7fee58ac9700 1 -- 172.21.15.5:0/1815068226 wait complete. 2024-05-14T15:26:01.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:00 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:26:01.065 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:26:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:26:00.882+0000 7fbbdacf9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:26:01.066 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:26:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:26:00.882+0000 7fbbdacf9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:26:01.066 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:26:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:26:00.882+0000 7fbbdacf9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:26:01.066 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:26:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:26:00.882+0000 7fbbdacf9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:26:01.066 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:26:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:26:00.882+0000 7fbbdacf9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T15:26:01.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:00 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:26:01.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:00 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:26:01.411 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:26:01.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:26:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:02.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:01 smithi005 ceph-mon[31559]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T15:26:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:01 smithi144 ceph-mon[35650]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T15:26:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:01 smithi193 ceph-mon[36001]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T15:26:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 -- 172.21.15.5:0/985317324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28108740 msgr2=0x7f9f281100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/985317324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28108740 0x7f9f281100a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000b0d0 tx=0x7f9f2000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 -- 172.21.15.5:0/985317324 shutdown_connections 2024-05-14T15:26:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/985317324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28108740 0x7f9f281100a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/985317324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f281034c0 0x7f9f281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/985317324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28107df0 0x7f9f28108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 -- 172.21.15.5:0/985317324 >> 172.21.15.5:0/985317324 conn(0x7f9f280fedc0 msgr2=0x7f9f281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 -- 172.21.15.5:0/985317324 shutdown_connections 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.495+0000 7f9f2ca85700 1 -- 172.21.15.5:0/985317324 wait complete. 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.496+0000 7f9f2ca85700 1 Processor -- start 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.496+0000 7f9f2ca85700 1 -- start start 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2ca85700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f281034c0 0x7f9f2819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2ca85700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 0x7f9f2819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2ca85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28108740 0x7f9f2819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:03.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2ca85700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28113370 con 0x7f9f28107df0 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2ca85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281131f0 con 0x7f9f28108740 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2ca85700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281134f0 con 0x7f9f281034c0 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 0x7f9f2819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 0x7f9f2819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34036/0 (socket says 172.21.15.5:34036) 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 -- 172.21.15.5:0/3193079352 learned_addr learned my addr 172.21.15.5:0/3193079352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f2659c700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f281034c0 0x7f9f2819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f26d9d700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28108740 0x7f9f2819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 -- 172.21.15.5:0/3193079352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f281034c0 msgr2=0x7f9f2819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f281034c0 0x7f9f2819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 -- 172.21.15.5:0/3193079352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28108740 msgr2=0x7f9f2819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28108740 0x7f9f2819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f25d9b700 1 -- 172.21.15.5:0/3193079352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f281a6f60 con 0x7f9f28107df0 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.497+0000 7f9f26d9d700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28108740 0x7f9f2819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.498+0000 7f9f25d9b700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 0x7f9f2819c150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d980 tx=0x7f9f1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.498+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18017070 con 0x7f9f28107df0 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.498+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f281a7250 con 0x7f9f28107df0 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.498+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f18009de0 con 0x7f9f28107df0 2024-05-14T15:26:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.498+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800f3c0 con 0x7f9f28107df0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.498+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281a77e0 con 0x7f9f28107df0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.499+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f280512c0 con 0x7f9f28107df0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.501+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7f9f18009930 con 0x7f9f28107df0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.503+0000 7f9f1f7fe700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9f140b2c20 0x7f9f140b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.503+0000 7f9f2659c700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9f140b2c20 0x7f9f140b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.503+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f9f180e2a50 con 0x7f9f28107df0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.504+0000 7f9f2659c700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9f140b2c20 0x7f9f140b5130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f10005950 tx=0x7f9f1000b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:03.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.506+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f180a9eb0 con 0x7f9f28107df0 2024-05-14T15:26:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.787+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f2806bc00 con 0x7f9f28107df0 2024-05-14T15:26:03.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.788+0000 7f9f1f7fe700 1 -- 172.21.15.5:0/3193079352 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11864 (secure 0 0 0) 0x7f9f180a97b0 con 0x7f9f28107df0 2024-05-14T15:26:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:26:03.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:24:47.806060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:26:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9f140b2c20 msgr2=0x7f9f140b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9f140b2c20 0x7f9f140b5130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f10005950 tx=0x7f9f1000b410 comp rx=0 tx=0).stop 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 msgr2=0x7f9f2819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 0x7f9f2819c150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d980 tx=0x7f9f1800dc90 comp rx=0 tx=0).stop 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 shutdown_connections 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9f140b2c20 0x7f9f140b5130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f28108740 0x7f9f2819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f281034c0 0x7f9f2819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 --2- 172.21.15.5:0/3193079352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f28107df0 0x7f9f2819c150 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.791+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 >> 172.21.15.5:0/3193079352 conn(0x7f9f280fedc0 msgr2=0x7f9f28107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.792+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 shutdown_connections 2024-05-14T15:26:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:03.792+0000 7f9f2ca85700 1 -- 172.21.15.5:0/3193079352 wait complete. 2024-05-14T15:26:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-14T15:26:04.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:03 smithi005 ceph-mon[31559]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:04.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:03 smithi005 ceph-mon[31559]: mgrmap e25: x(active, since 72s), standbys: y, z 2024-05-14T15:26:04.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:03 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3193079352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:26:04.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:03 smithi144 ceph-mon[35650]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:03 smithi144 ceph-mon[35650]: mgrmap e25: x(active, since 72s), standbys: y, z 2024-05-14T15:26:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:03 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3193079352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:26:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:03 smithi193 ceph-mon[36001]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:03 smithi193 ceph-mon[36001]: mgrmap e25: x(active, since 72s), standbys: y, z 2024-05-14T15:26:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:03 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3193079352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:26:04.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:26:04.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:06.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:05 smithi005 ceph-mon[31559]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T15:26:06.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:05 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:26:06.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:05 smithi144 ceph-mon[35650]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T15:26:06.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:05 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:26:06.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:05 smithi193 ceph-mon[36001]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-05-14T15:26:06.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:05 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4008124280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8107df0 msgr2=0x7fbad8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4008124280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8107df0 0x7fbad8108200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbac80099c0 tx=0x7fbac8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4008124280 shutdown_connections 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4008124280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbad8108740 0x7fbad81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4008124280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad81034c0 0x7fbad81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4008124280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8107df0 0x7fbad8108200 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4008124280 >> 172.21.15.5:0/4008124280 conn(0x7fbad80fedc0 msgr2=0x7fbad81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4008124280 shutdown_connections 2024-05-14T15:26:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.605+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4008124280 wait complete. 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.606+0000 7fbadf2fe700 1 Processor -- start 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.606+0000 7fbadf2fe700 1 -- start start 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.606+0000 7fbadf2fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbad81034c0 0x7fbad819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.606+0000 7fbadf2fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad8107df0 0x7fbad819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadf2fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 0x7fbad8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadf2fe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbad8113350 con 0x7fbad8108740 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadf2fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81131d0 con 0x7fbad81034c0 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadf2fe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81134d0 con 0x7fbad8107df0 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadd89b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 0x7fbad8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadd89b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 0x7fbad8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56334/0 (socket says 172.21.15.5:56334) 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadd89b700 1 -- 172.21.15.5:0/4074743752 learned_addr learned my addr 172.21.15.5:0/4074743752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadc899700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad8107df0 0x7fbad819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadd09a700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbad81034c0 0x7fbad819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadc899700 1 -- 172.21.15.5:0/4074743752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbad81034c0 msgr2=0x7fbad819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadc899700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbad81034c0 0x7fbad819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadc899700 1 -- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 msgr2=0x7fbad8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadc899700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 0x7fbad8199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadc899700 1 -- 172.21.15.5:0/4074743752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad819a260 con 0x7fbad8107df0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.607+0000 7fbadd89b700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 0x7fbad8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.608+0000 7fbadc899700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad8107df0 0x7fbad819fee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b580 tx=0x7fbac80036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.608+0000 7fbace7fc700 1 -- 172.21.15.5:0/4074743752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8005690 con 0x7fbad8107df0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.608+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad819a4f0 con 0x7fbad8107df0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.608+0000 7fbace7fc700 1 -- 172.21.15.5:0/4074743752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac8023070 con 0x7fbad8107df0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.608+0000 7fbace7fc700 1 -- 172.21.15.5:0/4074743752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac801fbd0 con 0x7fbad8107df0 2024-05-14T15:26:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.608+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad806b610 con 0x7fbad8107df0 2024-05-14T15:26:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.612+0000 7fbaedd12b80 1 Processor -- start 2024-05-14T15:26:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.612+0000 7fbaedd12b80 1 -- start start 2024-05-14T15:26:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.612+0000 7fbace7fc700 1 -- 172.21.15.5:0/4074743752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7fbac800f460 con 0x7fbad8107df0 2024-05-14T15:26:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.612+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f77b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555822e8c940 con 0x555822dece40 2024-05-14T15:26:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555822df1da0 con 0x555822f78070 2024-05-14T15:26:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555822e88900 con 0x555822f738e0 2024-05-14T15:26:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56342/0 (socket says 172.21.15.5:56342) 2024-05-14T15:26:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 -- 172.21.15.5:0/3307183815 learned_addr learned my addr 172.21.15.5:0/3307183815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd09a700 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f77b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 -- 172.21.15.5:0/3307183815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 msgr2=0x555822f77b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f77b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 -- 172.21.15.5:0/3307183815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 msgr2=0x555822f78480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.613+0000 7fbadd89b700 1 -- 172.21.15.5:0/3307183815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555822dd98d0 con 0x555822dece40 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.614+0000 7fbadd89b700 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f733a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009500 tx=0x7fbad0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf4843c3e5d0498 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.614+0000 7fbac37fe700 1 -- 172.21.15.5:0/3307183815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad001b020 con 0x555822dece40 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.614+0000 7fbac37fe700 1 -- 172.21.15.5:0/3307183815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad000edb0 con 0x555822dece40 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.614+0000 7fbace7fc700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbac40b2c20 0x7fbac40b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.614+0000 7fbac37fe700 1 -- 172.21.15.5:0/3307183815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0017d70 con 0x555822dece40 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.614+0000 7fbadd09a700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbac40b2c20 0x7fbac40b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.615+0000 7fbace7fc700 1 -- 172.21.15.5:0/4074743752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fbac80e7d80 con 0x7fbad8107df0 2024-05-14T15:26:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.615+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3307183815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 msgr2=0x555822f733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.615+0000 7fbadd09a700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbac40b2c20 0x7fbac40b5130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x555822f66f60 tx=0x7fbad4009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.615+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f733a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009500 tx=0x7fbad0009920 comp rx=0 tx=0).stop 2024-05-14T15:26:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3307183815 shutdown_connections 2024-05-14T15:26:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f78480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f733a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3307183815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f77b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3307183815 >> 172.21.15.5:0/3307183815 conn(0x555822e34630 msgr2=0x555822f65a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3307183815 shutdown_connections 2024-05-14T15:26:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3307183815 wait complete. 2024-05-14T15:26:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 -- start start 2024-05-14T15:26:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.616+0000 7fbaedd12b80 1 Processor -- start 2024-05-14T15:26:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 0x555822f7cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f78070 0x555822f7d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555822f80020 con 0x555822f738e0 2024-05-14T15:26:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbadc899700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbadd09a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 0x555822f7cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbadd09a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 0x555822f7cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49670/0 (socket says 172.21.15.5:49670) 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbadd09a700 1 -- 172.21.15.5:0/3773546095 learned_addr learned my addr 172.21.15.5:0/3773546095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbadc899700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f7d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56352/0 (socket says 172.21.15.5:56352) 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.617+0000 7fbadd89b700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f78070 0x555822f7d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555822e45ed0 con 0x555822dece40 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555822f801a0 con 0x555822f78070 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbaedd12b80 10 client.0 _ll_register_callbacks cb 0x555822e24950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbadc899700 1 -- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f78070 msgr2=0x555822f7d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbadc899700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f78070 0x555822f7d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbadc899700 1 -- 172.21.15.5:0/3773546095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 msgr2=0x555822f7cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbadc899700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 0x555822f7cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbadc899700 1 -- 172.21.15.5:0/3773546095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555822e43760 con 0x555822f738e0 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.618+0000 7fbadd09a700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 0x555822f7cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbadc899700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f7d2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbac80e4fe0 tx=0x7fbac80ae550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e1eddd5a078410 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbac0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbac0ff9700 1 -- 172.21.15.5:0/3773546095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80ae710 con 0x555822f738e0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad000edb0 con 0x555822f738e0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbac0ff9700 1 -- 172.21.15.5:0/3773546095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fbac80b7750 con 0x555822f738e0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbac0ff9700 1 -- 172.21.15.5:0/3773546095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbac80af300 con 0x555822f738e0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbaedd12b80 10 client.15666 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555822f8ac20 con 0x555822f738e0 2024-05-14T15:26:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.619+0000 7fbac0ff9700 1 -- 172.21.15.5:0/3773546095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac80aec00 con 0x555822f738e0 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbac0ff9700 1 -- 172.21.15.5:0/3773546095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fbac80e51e0 con 0x555822f738e0 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbaedd12b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbaedd12b80 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbaedd12b80 20 client.15666 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822f8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbaedd12b80 4 client.15666 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbad000edb0 con 0x555822f8d1d0 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.620+0000 7fbadd09a700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822f8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.621+0000 7fbac0ff9700 10 client.15666 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.621+0000 7fbadd09a700 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822f8f5f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2c11b32fcbfba4a server_cookie=158bba72bacd15ff in_seq=0 out_seq=0 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbac0ff9700 1 -- 172.21.15.5:0/3773546095 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbad000edb0 con 0x555822f8d1d0 2024-05-14T15:26:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbac0ff9700 10 client.15666 handle_command_reply: tid=0 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 client.15666 shutdown 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 msgr2=0x555822f7d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f7d2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbac80e4fe0 tx=0x7fbac80ae550 comp rx=0 tx=0).stop 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 shutdown_connections 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f78070 0x555822f7d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822f8f5f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbac0ff9700 0 client.15666 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822dece40 0x555822f7cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/3773546095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f7d2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 >> 172.21.15.5:0/3773546095 conn(0x555822e34630 msgr2=0x555822f698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 shutdown_connections 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.622+0000 7fbaedd12b80 1 -- 172.21.15.5:0/3773546095 wait complete. 2024-05-14T15:26:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.623+0000 7fbaedd12b80 20 client.15666 trim_cache size 0 max 16384 2024-05-14T15:26:06.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.645+0000 7fbaedd12b80 1 Processor -- start 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 -- start start 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f5fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f7d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81b0bf0 con 0x555822dece40 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81ae480 con 0x555822f78070 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbaedd12b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81ac2d0 con 0x555822f738e0 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbadd09a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f5fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbadd09a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f5fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56362/0 (socket says 172.21.15.5:56362) 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbadd09a700 1 -- 172.21.15.5:0/4048955623 learned_addr learned my addr 172.21.15.5:0/4048955623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbadd89b700 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f7d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.646+0000 7fbadc899700 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbadd09a700 1 -- 172.21.15.5:0/4048955623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 msgr2=0x555822f89110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbadd09a700 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f89110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbadd09a700 1 -- 172.21.15.5:0/4048955623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 msgr2=0x555822f7d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbadd09a700 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f7d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbadd09a700 1 -- 172.21.15.5:0/4048955623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555822f619e0 con 0x555822dece40 2024-05-14T15:26:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbadd09a700 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f5fc90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbad40095c0 tx=0x7fbad401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2677e9807b26eb67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbac2ffd700 1 -- 172.21.15.5:0/4048955623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad402a420 con 0x555822dece40 2024-05-14T15:26:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbac2ffd700 1 -- 172.21.15.5:0/4048955623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad000edb0 con 0x555822dece40 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.647+0000 7fbac2ffd700 1 -- 172.21.15.5:0/4048955623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad401ea90 con 0x555822dece40 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.648+0000 7fbaedd12b80 1 -- 172.21.15.5:0/4048955623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 msgr2=0x555822f5fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.648+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f5fc90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbad40095c0 tx=0x7fbad401b000 comp rx=0 tx=0).stop 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 -- 172.21.15.5:0/4048955623 shutdown_connections 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822f7d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822dece40 0x555822f5fc90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/4048955623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822f738e0 0x555822f89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 -- 172.21.15.5:0/4048955623 >> 172.21.15.5:0/4048955623 conn(0x555822e34630 msgr2=0x555822f5f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 -- 172.21.15.5:0/4048955623 shutdown_connections 2024-05-14T15:26:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 -- 172.21.15.5:0/4048955623 wait complete. 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 -- start start 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.649+0000 7fbaedd12b80 1 Processor -- start 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822dece40 0x555822f82840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f82d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbaedd12b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822ff5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbadc899700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f82d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbadc899700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f82d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56370/0 (socket says 172.21.15.5:56370) 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbadc899700 1 -- 172.21.15.5:0/2020369524 learned_addr learned my addr 172.21.15.5:0/2020369524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.650+0000 7fbadd09a700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822dece40 0x555822f82840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81ac940 con 0x555822f738e0 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81addb0 con 0x555822f78070 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbad81ad010 con 0x555822dece40 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadd89b700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822ff5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadd89b700 1 -- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822dece40 msgr2=0x555822f82840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadd89b700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822dece40 0x555822f82840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadd89b700 1 -- 172.21.15.5:0/2020369524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 msgr2=0x555822f82d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadd89b700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f82d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadd89b700 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555822ff5e80 con 0x555822f78070 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbadc899700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f82d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.651+0000 7fbaedd12b80 10 client.0 _ll_register_callbacks cb 0x555822e24950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad000edb0 con 0x555822f78070 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbadd89b700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822ff5940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009c10 tx=0x7fbad000e930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25126661a1b62602 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbac0ff9700 10 client.25328 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:26:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbac0ff9700 1 -- 172.21.15.5:0/2020369524 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad001b040 con 0x555822f78070 2024-05-14T15:26:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbac0ff9700 1 -- 172.21.15.5:0/2020369524 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fbad00178b0 con 0x555822f78070 2024-05-14T15:26:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbac0ff9700 1 -- 172.21.15.5:0/2020369524 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbad0017ad0 con 0x555822f78070 2024-05-14T15:26:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbaedd12b80 10 client.25328 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.652+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=26}) -- 0x555822ff6d70 con 0x555822f78070 2024-05-14T15:26:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbac0ff9700 1 -- 172.21.15.5:0/2020369524 <== mon.2 v2:172.21.15.144:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fbad0028070 con 0x555822f78070 2024-05-14T15:26:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbaedd12b80 10 client.25328 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbaedd12b80 10 client.25328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:26:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbaedd12b80 20 client.25328 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822ff92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbaedd12b80 4 client.25328 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555822e9a320 con 0x555822f8d1d0 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.653+0000 7fbadd09a700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822ff92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbadd09a700 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822ff92f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5fbfe98ce234f36 server_cookie=d8edc9a862f23b8 in_seq=0 out_seq=0 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbac0ff9700 10 client.25328 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbac0ff9700 1 -- 172.21.15.5:0/2020369524 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555822e9a320 con 0x555822f8d1d0 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbac0ff9700 10 client.25328 handle_command_reply: tid=0 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 client.25328 shutdown 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 msgr2=0x555822ff5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822ff5940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009c10 tx=0x7fbad000e930 comp rx=0 tx=0).stop 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 shutdown_connections 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x555822f8d1d0 0x555822ff92f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbac0ff9700 0 client.25328 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555822f78070 0x555822ff5940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.654+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555822dece40 0x555822f82840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.655+0000 7fbaedd12b80 1 --2- 172.21.15.5:0/2020369524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555822f738e0 0x555822f82d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.655+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 >> 172.21.15.5:0/2020369524 conn(0x555822e34630 msgr2=0x555822f65a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.655+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 shutdown_connections 2024-05-14T15:26:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.655+0000 7fbaedd12b80 1 -- 172.21.15.5:0/2020369524 wait complete. 2024-05-14T15:26:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.655+0000 7fbaedd12b80 20 client.25328 trim_cache size 0 max 16384 2024-05-14T15:26:06.658 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbac40b2c20 msgr2=0x7fbac40b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbac40b2c20 0x7fbac40b5130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x555822f66f60 tx=0x7fbad4009260 comp rx=0 tx=0).stop 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad8107df0 msgr2=0x7fbad819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad8107df0 0x7fbad819fee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b580 tx=0x7fbac80036f0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 shutdown_connections 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbac40b2c20 0x7fbac40b5130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbad8108740 0x7fbad8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbad81034c0 0x7fbad819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 --2- 172.21.15.5:0/4074743752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbad8107df0 0x7fbad819fee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 >> 172.21.15.5:0/4074743752 conn(0x7fbad80fedc0 msgr2=0x7fbad81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 shutdown_connections 2024-05-14T15:26:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:06.656+0000 7fbadf2fe700 1 -- 172.21.15.5:0/4074743752 wait complete. 2024-05-14T15:26:07.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:06 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:26:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:06 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:26:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:06 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:26:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:06 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:26:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:06 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:26:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:06 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:26:07.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:06 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:26:07.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:06 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:26:07.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:06 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:26:07.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:26:07.764 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:08.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:07 smithi005 ceph-mon[31559]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:08.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:07 smithi144 ceph-mon[35650]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:07 smithi193 ceph-mon[36001]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:09.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 -- 172.21.15.5:0/1651492089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48801034c0 msgr2=0x7f48801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 --2- 172.21.15.5:0/1651492089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48801034c0 0x7f48801038d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f48740099c0 tx=0x7f4874009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 -- 172.21.15.5:0/1651492089 shutdown_connections 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 --2- 172.21.15.5:0/1651492089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880108740 0x7f48801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 --2- 172.21.15.5:0/1651492089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48801034c0 0x7f48801038d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 --2- 172.21.15.5:0/1651492089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4880107df0 0x7f4880108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 -- 172.21.15.5:0/1651492089 >> 172.21.15.5:0/1651492089 conn(0x7f48800fedc0 msgr2=0x7f48801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 -- 172.21.15.5:0/1651492089 shutdown_connections 2024-05-14T15:26:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.516+0000 7f488716b700 1 -- 172.21.15.5:0/1651492089 wait complete. 2024-05-14T15:26:09.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.517+0000 7f488716b700 1 Processor -- start 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.517+0000 7f488716b700 1 -- start start 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f488716b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48801034c0 0x7f4880110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f488716b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880107df0 0x7f4880110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f488716b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 0x7f4880111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f488716b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4880114510 con 0x7f4880108740 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f4885708700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 0x7f4880111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f4885708700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 0x7f4880111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56386/0 (socket says 172.21.15.5:56386) 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f4885708700 1 -- 172.21.15.5:0/929686434 learned_addr learned my addr 172.21.15.5:0/929686434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f488716b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4880114390 con 0x7f48801034c0 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4880114690 con 0x7f4880107df0 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f4884f07700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48801034c0 0x7f4880110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.518+0000 7f487ffff700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880107df0 0x7f4880110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f4885708700 1 -- 172.21.15.5:0/929686434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880107df0 msgr2=0x7f4880110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f4885708700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880107df0 0x7f4880110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f4885708700 1 -- 172.21.15.5:0/929686434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48801034c0 msgr2=0x7f4880110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f4885708700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48801034c0 0x7f4880110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f4885708700 1 -- 172.21.15.5:0/929686434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801a91c0 con 0x7f4880108740 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f487ffff700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880107df0 0x7f4880110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f4885708700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 0x7f4880111500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f487800ef30 tx=0x7f487800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f487dffb700 1 -- 172.21.15.5:0/929686434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780092e0 con 0x7f4880108740 2024-05-14T15:26:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f487dffb700 1 -- 172.21.15.5:0/929686434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4878004bb0 con 0x7f4880108740 2024-05-14T15:26:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48801a9450 con 0x7f4880108740 2024-05-14T15:26:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.519+0000 7f487dffb700 1 -- 172.21.15.5:0/929686434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878004020 con 0x7f4880108740 2024-05-14T15:26:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.520+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a9940 con 0x7f4880108740 2024-05-14T15:26:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.524+0000 7f487dffb700 1 -- 172.21.15.5:0/929686434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7f487801b030 con 0x7f4880108740 2024-05-14T15:26:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.525+0000 7f487dffb700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f48700b2ba0 0x7f48700b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.525+0000 7f4895b7fb80 1 Processor -- start 2024-05-14T15:26:09.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.525+0000 7f4884f07700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f48700b2ba0 0x7f48700b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.525+0000 7f487dffb700 1 -- 172.21.15.5:0/929686434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f48780e2710 con 0x7f4880108740 2024-05-14T15:26:09.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.525+0000 7f4895b7fb80 1 -- start start 2024-05-14T15:26:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4884f07700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f48700b2ba0 0x7f48700b50b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c00 tx=0x7f487401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb823e40 0x55adeb9a7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adeb8c3940 con 0x55adeb9a1e10 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55adeb828da0 con 0x55adeb9a7d80 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55adeb8bf900 con 0x55adeb823e40 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4885708700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4885708700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56392/0 (socket says 172.21.15.5:56392) 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f4885708700 1 -- 172.21.15.5:0/3885112286 learned_addr learned my addr 172.21.15.5:0/3885112286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.526+0000 7f487ffff700 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f4884f07700 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb823e40 0x55adeb9a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f487ffff700 1 -- 172.21.15.5:0/3885112286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb823e40 msgr2=0x55adeb9a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f487ffff700 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb823e40 0x55adeb9a7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f487ffff700 1 -- 172.21.15.5:0/3885112286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 msgr2=0x55adeb9a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f487ffff700 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f487ffff700 1 -- 172.21.15.5:0/3885112286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adeb8108d0 con 0x55adeb9a7d80 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f487ffff700 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9a18d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f486c005950 tx=0x7f486c010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5694ab539aa6a00b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.527+0000 7f486affd700 1 -- 172.21.15.5:0/3885112286 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c01c480 con 0x55adeb9a7d80 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.528+0000 7f486affd700 1 -- 172.21.15.5:0/3885112286 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f486c01cad0 con 0x55adeb9a7d80 2024-05-14T15:26:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.528+0000 7f486affd700 1 -- 172.21.15.5:0/3885112286 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c011b80 con 0x55adeb9a7d80 2024-05-14T15:26:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3885112286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 msgr2=0x55adeb9a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9a18d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f486c005950 tx=0x7f486c010730 comp rx=0 tx=0).stop 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3885112286 shutdown_connections 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb823e40 0x55adeb9a7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3885112286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9a18d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.529+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3885112286 >> 172.21.15.5:0/3885112286 conn(0x55adeb86b630 msgr2=0x55adeb99ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.530+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3885112286 shutdown_connections 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.530+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3885112286 wait complete. 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.530+0000 7f4895b7fb80 1 -- start start 2024-05-14T15:26:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.531+0000 7f4895b7fb80 1 Processor -- start 2024-05-14T15:26:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a7d80 0x55adeb9abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adeb9ae870 con 0x55adeb9a1e10 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f487ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f487ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56408/0 (socket says 172.21.15.5:56408) 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4884f07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f487ffff700 1 -- 172.21.15.5:0/1153686035 learned_addr learned my addr 172.21.15.5:0/1153686035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55adeb87ced0 con 0x55adeb823e40 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4885708700 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a7d80 0x55adeb9abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55adeb9ae9f0 con 0x55adeb9a7d80 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.532+0000 7f4895b7fb80 10 client.0 _ll_register_callbacks cb 0x55adeb85b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f487ffff700 1 -- 172.21.15.5:0/1153686035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a7d80 msgr2=0x55adeb9abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f487ffff700 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a7d80 0x55adeb9abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f487ffff700 1 -- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 msgr2=0x55adeb9ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f487ffff700 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f487ffff700 1 -- 172.21.15.5:0/1153686035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adeb87a760 con 0x55adeb9a1e10 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f487ffff700 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9ab970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f486c000c00 tx=0x7f486c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef02d23408652655 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f485ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f485ffff700 1 -- 172.21.15.5:0/1153686035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c01c690 con 0x55adeb9a1e10 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f485ffff700 1 -- 172.21.15.5:0/1153686035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f486c01c830 con 0x55adeb9a1e10 2024-05-14T15:26:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.533+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486c01cad0 con 0x55adeb9a1e10 2024-05-14T15:26:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.534+0000 7f485ffff700 1 -- 172.21.15.5:0/1153686035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f486c031050 con 0x55adeb9a1e10 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.534+0000 7f4895b7fb80 10 client.15684 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.534+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55adeb9b9420 con 0x55adeb9a1e10 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.534+0000 7f485ffff700 1 -- 172.21.15.5:0/1153686035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f486c01cad0 con 0x55adeb9a1e10 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.534+0000 7f485ffff700 1 -- 172.21.15.5:0/1153686035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f486c02c030 con 0x55adeb9a1e10 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.534+0000 7f4895b7fb80 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f4895b7fb80 10 client.15684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f4895b7fb80 20 client.15684 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeb9bdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f4895b7fb80 4 client.15684 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adeb8d1320 con 0x55adeb9bb8f0 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f4884f07700 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeb9bdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.535+0000 7f485ffff700 10 client.15684 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:26:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.536+0000 7f4884f07700 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeb9bdd10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41ab7aedbb7a9cd server_cookie=ed67bf4636e1e1e3 in_seq=0 out_seq=0 2024-05-14T15:26:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.536+0000 7f485ffff700 1 -- 172.21.15.5:0/1153686035 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55adeb8d1320 con 0x55adeb9bb8f0 2024-05-14T15:26:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.536+0000 7f485ffff700 10 client.15684 handle_command_reply: tid=0 2024-05-14T15:26:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.536+0000 7f4895b7fb80 1 client.15684 shutdown 2024-05-14T15:26:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 msgr2=0x55adeb9ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9ab970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f486c000c00 tx=0x7f486c01cc80 comp rx=0 tx=0).stop 2024-05-14T15:26:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 shutdown_connections 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeb9bdd10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f485ffff700 0 client.15684 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a7d80 0x55adeb9abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/1153686035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a1e10 0x55adeb9ab970 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.537+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 >> 172.21.15.5:0/1153686035 conn(0x55adeb86b630 msgr2=0x55adeb9a08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.538+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 shutdown_connections 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.538+0000 7f4895b7fb80 1 -- 172.21.15.5:0/1153686035 wait complete. 2024-05-14T15:26:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.538+0000 7f4895b7fb80 20 client.15684 trim_cache size 0 max 16384 2024-05-14T15:26:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 Processor -- start 2024-05-14T15:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 -- start start 2024-05-14T15:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a7d80 0x55adeb9a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b5190 con 0x55adeb9a7d80 2024-05-14T15:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b2a20 con 0x55adeb823e40 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4895b7fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b0870 con 0x55adeb9a1e10 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4885708700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a7d80 0x55adeb9a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4884f07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4885708700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a7d80 0x55adeb9a6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56410/0 (socket says 172.21.15.5:56410) 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.568+0000 7f4885708700 1 -- 172.21.15.5:0/3076227507 learned_addr learned my addr 172.21.15.5:0/3076227507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f487ffff700 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f4884f07700 1 -- 172.21.15.5:0/3076227507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 msgr2=0x55adeb9b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f4884f07700 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f4884f07700 1 -- 172.21.15.5:0/3076227507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a7d80 msgr2=0x55adeb9a6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f4884f07700 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a7d80 0x55adeb9a6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f4884f07700 1 -- 172.21.15.5:0/3076227507 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adeb9a7bb0 con 0x55adeb823e40 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f487ffff700 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f4884f07700 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9a18d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f487400b580 tx=0x7f487401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=684a9727d0ea9b23 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f486a7fc700 1 -- 172.21.15.5:0/3076227507 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400b790 con 0x55adeb823e40 2024-05-14T15:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f486a7fc700 1 -- 172.21.15.5:0/3076227507 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487400bde0 con 0x55adeb823e40 2024-05-14T15:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.569+0000 7f486a7fc700 1 -- 172.21.15.5:0/3076227507 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400fb00 con 0x55adeb823e40 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.570+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3076227507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 msgr2=0x55adeb9a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.570+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9a18d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f487400b580 tx=0x7f487401f000 comp rx=0 tx=0).stop 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3076227507 shutdown_connections 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb9a7d80 0x55adeb9a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb823e40 0x55adeb9a18d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/3076227507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3076227507 >> 172.21.15.5:0/3076227507 conn(0x55adeb86b630 msgr2=0x55adeb9a9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3076227507 shutdown_connections 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 -- 172.21.15.5:0/3076227507 wait complete. 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 -- start start 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.571+0000 7f4895b7fb80 1 Processor -- start 2024-05-14T15:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4895b7fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 0x55adeb9bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4884f07700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 0x55adeb9bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4884f07700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 0x55adeb9bdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56422/0 (socket says 172.21.15.5:56422) 2024-05-14T15:26:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4884f07700 1 -- 172.21.15.5:0/2754623613 learned_addr learned my addr 172.21.15.5:0/2754623613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9be080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b0ee0 con 0x55adeb823e40 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b2350 con 0x55adeb9a7d80 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.572+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48801b15b0 con 0x55adeb9a1e10 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4895b7fb80 10 client.0 _ll_register_callbacks cb 0x55adeb85b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f487ffff700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9be080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4885708700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4884f07700 1 -- 172.21.15.5:0/2754623613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 msgr2=0x55adeb9be080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4884f07700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9be080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4884f07700 1 -- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 msgr2=0x55adeb9be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4884f07700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4884f07700 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adeb99cf40 con 0x55adeb823e40 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f487ffff700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9be080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4885708700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4884f07700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 0x55adeb9bdb40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4874005b40 tx=0x7f4874005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d1c39ec3bead23 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f485ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f485ffff700 1 -- 172.21.15.5:0/2754623613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400bba0 con 0x55adeb823e40 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f485ffff700 1 -- 172.21.15.5:0/2754623613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f487400f760 con 0x55adeb823e40 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.573+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f487400bde0 con 0x55adeb823e40 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.574+0000 7f485ffff700 1 -- 172.21.15.5:0/2754623613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4874049050 con 0x55adeb823e40 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.574+0000 7f4895b7fb80 10 client.15690 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.574+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55adeba24fd0 con 0x55adeb823e40 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.574+0000 7f485ffff700 1 -- 172.21.15.5:0/2754623613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f487400bde0 con 0x55adeb823e40 2024-05-14T15:26:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f485ffff700 1 -- 172.21.15.5:0/2754623613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f48740319f0 con 0x55adeb823e40 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f4895b7fb80 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f4895b7fb80 10 client.15690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f4895b7fb80 20 client.15690 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeba275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f4895b7fb80 4 client.15690 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adeb8cb3c0 con 0x55adeb9bb8f0 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f487ffff700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeba275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f487ffff700 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeba275b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a566b3135d66203 server_cookie=33d2db6161370724 in_seq=0 out_seq=0 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.575+0000 7f485ffff700 10 client.15690 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f485ffff700 1 -- 172.21.15.5:0/2754623613 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adeb8cb3c0 con 0x55adeb9bb8f0 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f485ffff700 10 client.15690 handle_command_reply: tid=0 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 client.15690 shutdown 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 msgr2=0x55adeb9bdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 0x55adeb9bdb40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4874005b40 tx=0x7f4874005fb0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 shutdown_connections 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55adeb9bb8f0 0x55adeba275b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55adeb9a7d80 0x55adeb9be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adeb823e40 0x55adeb9bdb40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f485ffff700 0 client.15690 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 --2- 172.21.15.5:0/2754623613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adeb9a1e10 0x55adeb9be080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 >> 172.21.15.5:0/2754623613 conn(0x55adeb86b630 msgr2=0x55adeb996d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 shutdown_connections 2024-05-14T15:26:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 1 -- 172.21.15.5:0/2754623613 wait complete. 2024-05-14T15:26:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.576+0000 7f4895b7fb80 20 client.15690 trim_cache size 0 max 16384 2024-05-14T15:26:09.581 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:26:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f48700b2ba0 msgr2=0x7f48700b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f48700b2ba0 0x7f48700b50b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c00 tx=0x7f487401a040 comp rx=0 tx=0).stop 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 msgr2=0x7f4880111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 0x7f4880111500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f487800ef30 tx=0x7f487800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 shutdown_connections 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f48700b2ba0 0x7f48700b50b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.577+0000 7f488716b700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4880108740 0x7f4880111500 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.578+0000 7f488716b700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48801034c0 0x7f4880110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.578+0000 7f488716b700 1 --2- 172.21.15.5:0/929686434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4880107df0 0x7f4880110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.578+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 >> 172.21.15.5:0/929686434 conn(0x7f48800fedc0 msgr2=0x7f48801075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.578+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 shutdown_connections 2024-05-14T15:26:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:09.578+0000 7f488716b700 1 -- 172.21.15.5:0/929686434 wait complete. 2024-05-14T15:26:10.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:09 smithi005 ceph-mon[31559]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-14T15:26:10.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:09 smithi144 ceph-mon[35650]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-14T15:26:10.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:09 smithi193 ceph-mon[36001]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-14T15:26:10.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:26:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:12.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:11 smithi005 ceph-mon[31559]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-14T15:26:12.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:11 smithi144 ceph-mon[35650]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-14T15:26:12.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:11 smithi193 ceph-mon[36001]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-14T15:26:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 -- 172.21.15.5:0/3739576867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4108f20 msgr2=0x7f52b4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 --2- 172.21.15.5:0/3739576867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4108f20 0x7f52b4109330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f529c0099c0 tx=0x7f529c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 -- 172.21.15.5:0/3739576867 shutdown_connections 2024-05-14T15:26:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 --2- 172.21.15.5:0/3739576867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52b4109870 0x7f52b41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 --2- 172.21.15.5:0/3739576867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 0x7f52b4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 --2- 172.21.15.5:0/3739576867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4108f20 0x7f52b4109330 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 -- 172.21.15.5:0/3739576867 >> 172.21.15.5:0/3739576867 conn(0x7f52b4100040 msgr2=0x7f52b4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 -- 172.21.15.5:0/3739576867 shutdown_connections 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.542+0000 7f52ba42d700 1 -- 172.21.15.5:0/3739576867 wait complete. 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.543+0000 7f52ba42d700 1 Processor -- start 2024-05-14T15:26:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 -- start start 2024-05-14T15:26:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 0x7f52b419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52b4108f20 0x7f52b419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 0x7f52b419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41144a0 con 0x7f52b4109870 2024-05-14T15:26:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4114320 con 0x7f52b41045f0 2024-05-14T15:26:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52ba42d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4114620 con 0x7f52b4108f20 2024-05-14T15:26:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.544+0000 7f52b89ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 0x7f52b419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 0x7f52b419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 0x7f52b419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56434/0 (socket says 172.21.15.5:56434) 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 -- 172.21.15.5:0/980348820 learned_addr learned my addr 172.21.15.5:0/980348820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52abfff700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52b4108f20 0x7f52b419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 -- 172.21.15.5:0/980348820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52b4108f20 msgr2=0x7f52b419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52b4108f20 0x7f52b419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 -- 172.21.15.5:0/980348820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 msgr2=0x7f52b419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 0x7f52b419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 -- 172.21.15.5:0/980348820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b419fd10 con 0x7f52b4109870 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b3fff700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 0x7f52b419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.545+0000 7f52b89ca700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 0x7f52b419f670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00ef30 tx=0x7f52ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.546+0000 7f52b1ffb700 1 -- 172.21.15.5:0/980348820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0092e0 con 0x7f52b4109870 2024-05-14T15:26:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.546+0000 7f52b1ffb700 1 -- 172.21.15.5:0/980348820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac004bb0 con 0x7f52b4109870 2024-05-14T15:26:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.546+0000 7f52b1ffb700 1 -- 172.21.15.5:0/980348820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac004020 con 0x7f52b4109870 2024-05-14T15:26:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.546+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b419ffa0 con 0x7f52b4109870 2024-05-14T15:26:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.546+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a0460 con 0x7f52b4109870 2024-05-14T15:26:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.551+0000 7f52b1ffb700 1 -- 172.21.15.5:0/980348820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7f52ac01b030 con 0x7f52b4109870 2024-05-14T15:26:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 Processor -- start 2024-05-14T15:26:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 -- start start 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52b1ffb700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52a00b2d70 0x7f52a00b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52b1ffb700 1 -- 172.21.15.5:0/980348820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f52ac0e25c0 con 0x7f52b4109870 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569ff7dd940 con 0x5569ff73de40 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5569ff742da0 con 0x5569ff8c1d80 2024-05-14T15:26:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.552+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569ff7d9900 con 0x5569ff8bbe10 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.553+0000 7f52abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.553+0000 7f52abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8bb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49800/0 (socket says 172.21.15.5:49800) 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.553+0000 7f52abfff700 1 -- 172.21.15.5:0/2203004846 learned_addr learned my addr 172.21.15.5:0/2203004846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.553+0000 7f52b3fff700 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.553+0000 7f52b89ca700 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.553+0000 7f52abfff700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52a00b2d70 0x7f52a00b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52abfff700 1 -- 172.21.15.5:0/2203004846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 msgr2=0x5569ff8bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52abfff700 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8bc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52abfff700 1 -- 172.21.15.5:0/2203004846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 msgr2=0x5569ff8c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52abfff700 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52abfff700 1 -- 172.21.15.5:0/2203004846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ff72a8d0 con 0x5569ff8c1d80 2024-05-14T15:26:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52b3fff700 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.554+0000 7f52abfff700 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8bb8d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52b406c680 tx=0x7f529c009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef58e62fada2fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.555+0000 7f52abfff700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52a00b2d70 0x7f52a00b5280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f529c025000 tx=0x7f529c027000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.555+0000 7f52aa7fc700 1 -- 172.21.15.5:0/2203004846 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c003d50 con 0x5569ff8c1d80 2024-05-14T15:26:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.555+0000 7f52aa7fc700 1 -- 172.21.15.5:0/2203004846 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c00bc40 con 0x5569ff8c1d80 2024-05-14T15:26:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.555+0000 7f52aa7fc700 1 -- 172.21.15.5:0/2203004846 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c024b00 con 0x5569ff8c1d80 2024-05-14T15:26:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2203004846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 msgr2=0x5569ff8bb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8bb8d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52b406c680 tx=0x7f529c009640 comp rx=0 tx=0).stop 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2203004846 shutdown_connections 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2203004846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8bb8d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.556+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2203004846 >> 172.21.15.5:0/2203004846 conn(0x5569ff785630 msgr2=0x5569ff8b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.557+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2203004846 shutdown_connections 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.557+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2203004846 wait complete. 2024-05-14T15:26:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.557+0000 7f52c8e41b80 1 -- start start 2024-05-14T15:26:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.558+0000 7f52c8e41b80 1 Processor -- start 2024-05-14T15:26:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.558+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff8c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56442/0 (socket says 172.21.15.5:56442) 2024-05-14T15:26:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49816/0 (socket says 172.21.15.5:49816) 2024-05-14T15:26:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52b3fff700 1 -- 172.21.15.5:0/4134802868 learned_addr learned my addr 172.21.15.5:0/4134802868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569ff8c2790 con 0x5569ff73de40 2024-05-14T15:26:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5569ff796ed0 con 0x5569ff8bbe10 2024-05-14T15:26:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569ff8c2910 con 0x5569ff8c1d80 2024-05-14T15:26:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.559+0000 7f52b89ca700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff8c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52c8e41b80 10 client.0 _ll_register_callbacks cb 0x5569ff775950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52abfff700 1 -- 172.21.15.5:0/4134802868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 msgr2=0x5569ff8c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52abfff700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff8c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52abfff700 1 -- 172.21.15.5:0/4134802868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 msgr2=0x5569ff8cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52abfff700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52abfff700 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ff794760 con 0x5569ff8bbe10 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52b3fff700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52abfff700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8cbe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f529c009350 tx=0x7f529c0059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b622e594819a086 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52937fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52937fe700 1 -- 172.21.15.5:0/4134802868 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c033040 con 0x5569ff8bbe10 2024-05-14T15:26:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.560+0000 7f52937fe700 1 -- 172.21.15.5:0/4134802868 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f529c024740 con 0x5569ff8bbe10 2024-05-14T15:26:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529c00bc40 con 0x5569ff8bbe10 2024-05-14T15:26:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52937fe700 1 -- 172.21.15.5:0/4134802868 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f529c00ba50 con 0x5569ff8bbe10 2024-05-14T15:26:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 10 client.25358 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569ff8d3320 con 0x5569ff8bbe10 2024-05-14T15:26:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52937fe700 1 -- 172.21.15.5:0/4134802868 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c00bc40 con 0x5569ff8bbe10 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52937fe700 1 -- 172.21.15.5:0/4134802868 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f529c037a00 con 0x5569ff8bbe10 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 10 client.25358 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 20 client.25358 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff8d58e0 0x5569ff8d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 4 client.25358 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.561+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ff7eb320 con 0x5569ff8d58e0 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.562+0000 7f52b3fff700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff8d58e0 0x5569ff8d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.562+0000 7f52937fe700 10 client.25358 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.563+0000 7f52b3fff700 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff8d58e0 0x5569ff8d7d70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d0e0c35f687a1b server_cookie=789a6450ddf8365e in_seq=0 out_seq=0 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.564+0000 7f52937fe700 1 -- 172.21.15.5:0/4134802868 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569ff7eb320 con 0x5569ff8d58e0 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.564+0000 7f52937fe700 10 client.25358 handle_command_reply: tid=0 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.564+0000 7f52c8e41b80 1 client.25358 shutdown 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.564+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 msgr2=0x5569ff8cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.564+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8cbe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f529c009350 tx=0x7f529c0059e0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 shutdown_connections 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff8d58e0 0x5569ff8d7d70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff8c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4134802868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8cbe90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 >> 172.21.15.5:0/4134802868 conn(0x5569ff785630 msgr2=0x5569ff8ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52937fe700 0 client.25358 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 shutdown_connections 2024-05-14T15:26:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4134802868 wait complete. 2024-05-14T15:26:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.565+0000 7f52c8e41b80 20 client.25358 trim_cache size 0 max 16384 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 -- 172.21.15.5:0/4187247693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010eac0 msgr2=0x7f5fa010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/4187247693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010eac0 0x7f5fa010eef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009900 tx=0x7f5f9002a3f0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 -- 172.21.15.5:0/4187247693 shutdown_connections 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/4187247693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fa010f580 0x7f5fa0116d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/4187247693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010eac0 0x7f5fa010eef0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/4187247693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010a530 0x7f5fa010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 -- 172.21.15.5:0/4187247693 >> 172.21.15.5:0/4187247693 conn(0x7f5fa00761d0 msgr2=0x7f5fa00765e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 -- 172.21.15.5:0/4187247693 shutdown_connections 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.582+0000 7f5fa72a8640 1 -- 172.21.15.5:0/4187247693 wait complete. 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.583+0000 7f5fa72a8640 1 Processor -- start 2024-05-14T15:26:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.583+0000 7f5fa72a8640 1 -- start start 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.583+0000 7f5fa72a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 0x7f5fa01a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.583+0000 7f5fa72a8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fa010eac0 0x7f5fa01a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa72a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010f580 0x7f5fa01a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa72a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa011a480 con 0x7f5fa010a530 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa72a8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa011a300 con 0x7f5fa010f580 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa72a8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa011a600 con 0x7f5fa010eac0 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 0x7f5fa01a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 0x7f5fa01a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56466/0 (socket says 172.21.15.5:56466) 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 -- 172.21.15.5:0/2794622562 learned_addr learned my addr 172.21.15.5:0/2794622562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5845640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010f580 0x7f5fa01a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 Processor -- start 2024-05-14T15:26:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 -- 172.21.15.5:0/2794622562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fa010eac0 msgr2=0x7f5fa01a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 -- start start 2024-05-14T15:26:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa4843640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fa010eac0 0x7f5fa01a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fa010eac0 0x7f5fa01a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 -- 172.21.15.5:0/2794622562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010f580 msgr2=0x7f5fa01a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010f580 0x7f5fa01a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5044640 1 -- 172.21.15.5:0/2794622562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa01a5cc0 con 0x7f5fa010a530 2024-05-14T15:26:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f5fa5845640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010f580 0x7f5fa01a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:12.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b0d50 con 0x5569ff73de40 2024-05-14T15:26:12.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ae5e0 con 0x5569ff8c1d80 2024-05-14T15:26:12.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.584+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ac430 con 0x5569ff8bbe10 2024-05-14T15:26:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f5fa5044640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 0x7f5fa01a0e60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00d950 tx=0x7f5f9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8bb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56472/0 (socket says 172.21.15.5:56472) 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 -- 172.21.15.5:0/4222245538 learned_addr learned my addr 172.21.15.5:0/4222245538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b89ca700 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 -- 172.21.15.5:0/4222245538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 msgr2=0x5569ff8c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 -- 172.21.15.5:0/4222245538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 msgr2=0x5569ff8b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 -- 172.21.15.5:0/4222245538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ff8b7fe0 con 0x5569ff73de40 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f52b3fff700 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8bb8d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52a4000bf0 tx=0x7f52a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1f31ae4b746e55 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.586+0000 7f52b89ca700 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f5f967fc640 1 -- 172.21.15.5:0/2794622562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c017070 con 0x7f5fa010a530 2024-05-14T15:26:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f5f967fc640 1 -- 172.21.15.5:0/2794622562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c009c30 con 0x7f5fa010a530 2024-05-14T15:26:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa01a5fb0 con 0x7f5fa010a530 2024-05-14T15:26:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.585+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa01ad930 con 0x7f5fa010a530 2024-05-14T15:26:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.586+0000 7f5f967fc640 1 -- 172.21.15.5:0/2794622562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c017070 con 0x7f5fa010a530 2024-05-14T15:26:12.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.586+0000 7f52a9ffb700 1 -- 172.21.15.5:0/4222245538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400fbc0 con 0x5569ff73de40 2024-05-14T15:26:12.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.586+0000 7f52a9ffb700 1 -- 172.21.15.5:0/4222245538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4026030 con 0x5569ff73de40 2024-05-14T15:26:12.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.587+0000 7f52a9ffb700 1 -- 172.21.15.5:0/4222245538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4005550 con 0x5569ff73de40 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4222245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 msgr2=0x5569ff8bb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8bb8d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52a4000bf0 tx=0x7f52a4009260 comp rx=0 tx=0).stop 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4222245538 shutdown_connections 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8c1d80 0x5569ff8b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8bb8d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/4222245538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8bbe10 0x5569ff8c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4222245538 >> 172.21.15.5:0/4222245538 conn(0x5569ff785630 msgr2=0x5569ff8b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.588+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4222245538 shutdown_connections 2024-05-14T15:26:12.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f52c8e41b80 1 -- 172.21.15.5:0/4222245538 wait complete. 2024-05-14T15:26:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f5faddfd740 1 Processor -- start 2024-05-14T15:26:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f52c8e41b80 1 -- start start 2024-05-14T15:26:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 Processor -- start 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f5faddfd740 1 -- start start 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac32120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.589+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac32660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac32660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac32660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56482/0 (socket says 172.21.15.5:56482) 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5faddfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56092ab975d0 con 0x56092a8704d0 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5faddfd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56092ab50890 con 0x56092a8682f0 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5faddfd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56092ab93590 con 0x56092a8911b0 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa4843640 1 -- 172.21.15.5:0/3498296485 learned_addr learned my addr 172.21.15.5:0/3498296485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa5845640 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa5845640 1 -- 172.21.15.5:0/3498296485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 msgr2=0x56092ac32120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:26:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa5845640 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac32120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff93df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41acaa0 con 0x5569ff73de40 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41adf10 con 0x5569ff8bbe10 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ad170 con 0x5569ff8c1d80 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f52c8e41b80 10 client.0 _ll_register_callbacks cb 0x5569ff775950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b89ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff93df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56488/0 (socket says 172.21.15.5:56488) 2024-05-14T15:26:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 -- 172.21.15.5:0/2915888630 learned_addr learned my addr 172.21.15.5:0/2915888630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52abfff700 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa5845640 1 -- 172.21.15.5:0/3498296485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 msgr2=0x56092ac32660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa5845640 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac32660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa5845640 1 -- 172.21.15.5:0/3498296485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092a8a9210 con 0x56092a8911b0 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.590+0000 7f5fa4843640 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac32660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f5fa5845640 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac31be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009f10 tx=0x7f5f980107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adeb8b55ce7fad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f5f8b7fe640 1 -- 172.21.15.5:0/3498296485 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9801c680 con 0x56092a8911b0 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f5f8b7fe640 1 -- 172.21.15.5:0/3498296485 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9801ccd0 con 0x56092a8911b0 2024-05-14T15:26:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f5f8b7fe640 1 -- 172.21.15.5:0/3498296485 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98011e40 con 0x56092a8911b0 2024-05-14T15:26:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 -- 172.21.15.5:0/2915888630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 msgr2=0x5569ff93df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f5f967fc640 1 -- 172.21.15.5:0/2794622562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7f5f9c010460 con 0x7f5fa010a530 2024-05-14T15:26:12.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff93df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 -- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 msgr2=0x5569ff8d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.591+0000 7f52b3fff700 1 -- 172.21.15.5:0/2915888630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ff8b6f40 con 0x5569ff73de40 2024-05-14T15:26:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.592+0000 7f52b3fff700 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8d10e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52a4004ef0 tx=0x7f52a4004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270daff64b006a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.592+0000 7f52937fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.592+0000 7f5faddfd740 1 -- 172.21.15.5:0/3498296485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 msgr2=0x56092ac31be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.592+0000 7f5faddfd740 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac31be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009f10 tx=0x7f5f980107a0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.592+0000 7f52937fe700 1 -- 172.21.15.5:0/2915888630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400fbc0 con 0x5569ff73de40 2024-05-14T15:26:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f5f967fc640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f8c0b2d00 0x7f5f8c0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f5f967fc640 1 -- 172.21.15.5:0/2794622562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5f9c0a5870 con 0x7f5fa010a530 2024-05-14T15:26:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f5fa5845640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f8c0b2d00 0x7f5f8c0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.592+0000 7f52937fe700 1 -- 172.21.15.5:0/2915888630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f52a400fd60 con 0x5569ff73de40 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a4026030 con 0x5569ff73de40 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52937fe700 1 -- 172.21.15.5:0/2915888630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52a40051d0 con 0x5569ff73de40 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 10 client.15717 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569ff93f610 con 0x5569ff73de40 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52937fe700 1 -- 172.21.15.5:0/2915888630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4026030 con 0x5569ff73de40 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52937fe700 1 -- 172.21.15.5:0/2915888630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f52a4022180 con 0x5569ff73de40 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 20 client.15717 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff93fce0 0x5569ff942100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 4 client.15717 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:26:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.593+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569ff7e53c0 con 0x5569ff93fce0 2024-05-14T15:26:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 -- 172.21.15.5:0/3498296485 shutdown_connections 2024-05-14T15:26:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac32660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac32120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 --2- 172.21.15.5:0/3498296485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac31be0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5fa5845640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f8c0b2d00 0x7f5f8c0b5210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009f10 tx=0x7f5f98009b90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 -- 172.21.15.5:0/3498296485 >> 172.21.15.5:0/3498296485 conn(0x56092a885300 msgr2=0x56092ac2e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 -- 172.21.15.5:0/3498296485 shutdown_connections 2024-05-14T15:26:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 -- 172.21.15.5:0/3498296485 wait complete. 2024-05-14T15:26:12.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f52abfff700 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff93fce0 0x5569ff942100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.594+0000 7f5faddfd740 1 -- start start 2024-05-14T15:26:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f5faddfd740 1 Processor -- start 2024-05-14T15:26:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52abfff700 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff93fce0 0x5569ff942100 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1b0b17a280426f server_cookie=d9187f974a62d778 in_seq=0 out_seq=0 2024-05-14T15:26:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52937fe700 10 client.15717 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52937fe700 1 -- 172.21.15.5:0/2915888630 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569ff7e53c0 con 0x5569ff93fce0 2024-05-14T15:26:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52937fe700 10 client.15717 handle_command_reply: tid=0 2024-05-14T15:26:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 client.15717 shutdown 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 msgr2=0x5569ff8d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8d10e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52a4004ef0 tx=0x7f52a4004f20 comp rx=0 tx=0).stop 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 shutdown_connections 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5569ff93fce0 0x5569ff942100 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569ff8c1d80 0x5569ff93df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569ff73de40 0x5569ff8d10e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 --2- 172.21.15.5:0/2915888630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5569ff8bbe10 0x5569ff8d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 >> 172.21.15.5:0/2915888630 conn(0x5569ff785630 msgr2=0x5569ff8c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.595+0000 7f52937fe700 0 client.15717 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8682f0 0x56092ac3c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 0x56092ac3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8911b0 0x56092ac481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56092ac3ed10 con 0x56092a8682f0 2024-05-14T15:26:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56092ab54370 con 0x56092a8911b0 2024-05-14T15:26:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56092ac3ee90 con 0x56092a8704d0 2024-05-14T15:26:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5faddfd740 10 client.0 _ll_register_callbacks cb 0x56092a8a8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 0x56092ac3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 0x56092ac3c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45022/0 (socket says 172.21.15.5:45022) 2024-05-14T15:26:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa4843640 1 -- 172.21.15.5:0/1368665735 learned_addr learned my addr 172.21.15.5:0/1368665735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa5044640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8682f0 0x56092ac3c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 shutdown_connections 2024-05-14T15:26:12.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f52c8e41b80 1 -- 172.21.15.5:0/2915888630 wait complete. 2024-05-14T15:26:12.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f52c8e41b80 20 client.15717 trim_cache size 0 max 16384 2024-05-14T15:26:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa5845640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8911b0 0x56092ac481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.610 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:26:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa4843640 1 -- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8911b0 msgr2=0x56092ac481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa4843640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8911b0 0x56092ac481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.596+0000 7f5fa4843640 1 -- 172.21.15.5:0/1368665735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8682f0 msgr2=0x56092ac3c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f5fa4843640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8682f0 0x56092ac3c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f5fa4843640 1 -- 172.21.15.5:0/1368665735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092a975620 con 0x56092a8704d0 2024-05-14T15:26:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f5fa5044640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8682f0 0x56092ac3c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52a00b2d70 msgr2=0x7f52a00b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52a00b2d70 0x7f52a00b5280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f529c025000 tx=0x7f529c027000 comp rx=0 tx=0).stop 2024-05-14T15:26:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 msgr2=0x7f52b419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 0x7f52b419f670 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00ef30 tx=0x7f52ac00c3a0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f5fa4843640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 0x56092ac3c6e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f90033000 tx=0x7f5f900316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1bac3cab2309fa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f5f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:26:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 shutdown_connections 2024-05-14T15:26:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52a00b2d70 0x7f52a00b5280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b4109870 0x7f52b419f670 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b41045f0 0x7f52b419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 --2- 172.21.15.5:0/980348820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52b4108f20 0x7f52b419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 >> 172.21.15.5:0/980348820 conn(0x7f52b4100040 msgr2=0x7f52b410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.597+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 shutdown_connections 2024-05-14T15:26:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f52ba42d700 1 -- 172.21.15.5:0/980348820 wait complete. 2024-05-14T15:26:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1368665735 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90051070 con 0x56092a8704d0 2024-05-14T15:26:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1368665735 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5f90004240 con 0x56092a8704d0 2024-05-14T15:26:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9801ccd0 con 0x56092a8704d0 2024-05-14T15:26:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1368665735 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f90035040 con 0x56092a8704d0 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5faddfd740 10 client.25399 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56092ac4b010 con 0x56092a8704d0 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.598+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1368665735 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9801ccd0 con 0x56092a8704d0 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1368665735 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5f9004a040 con 0x56092a8704d0 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5faddfd740 10 client.25399 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5faddfd740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5faddfd740 20 client.25399 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a84fd80 0x56092ac4e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5faddfd740 4 client.25399 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56092aba51e0 con 0x56092a84fd80 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.599+0000 7f5fa5044640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a84fd80 0x56092ac4e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.600+0000 7f5fa5044640 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a84fd80 0x56092ac4e4e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1986c35a56ec237f server_cookie=7447835fb4e7d945 in_seq=0 out_seq=0 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.600+0000 7f5f88ff9640 10 client.25399 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1368665735 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56092aba51e0 con 0x56092a84fd80 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5f88ff9640 10 client.25399 handle_command_reply: tid=0 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 client.25399 shutdown 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 msgr2=0x56092ac3c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 0x56092ac3c6e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f90033000 tx=0x7f5f900316d0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 shutdown_connections 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a84fd80 0x56092ac4e4e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8704d0 0x56092ac3c6e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5f88ff9640 0 client.25399 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8911b0 0x56092ac481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1368665735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8682f0 0x56092ac3c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 >> 172.21.15.5:0/1368665735 conn(0x56092a885300 msgr2=0x56092ac319b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 shutdown_connections 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.601+0000 7f5faddfd740 1 -- 172.21.15.5:0/1368665735 wait complete. 2024-05-14T15:26:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.602+0000 7f5faddfd740 20 client.25399 trim_cache size 0 max 16384 2024-05-14T15:26:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.632+0000 7f5faddfd740 1 Processor -- start 2024-05-14T15:26:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5faddfd740 1 -- start start 2024-05-14T15:26:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac3bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5fa5044640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac3bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5fa5044640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac3bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49878/0 (socket says 172.21.15.5:49878) 2024-05-14T15:26:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.633+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56494/0 (socket says 172.21.15.5:56494) 2024-05-14T15:26:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa5044640 1 -- 172.21.15.5:0/297093704 learned_addr learned my addr 172.21.15.5:0/297093704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01b1ab0 con 0x56092a8704d0 2024-05-14T15:26:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01af380 con 0x56092a8682f0 2024-05-14T15:26:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa006a240 con 0x56092a8911b0 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa5845640 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa4843640 1 -- 172.21.15.5:0/297093704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 msgr2=0x56092ac53c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa4843640 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac53c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa4843640 1 -- 172.21.15.5:0/297093704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 msgr2=0x56092ac3bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa4843640 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac3bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa4843640 1 -- 172.21.15.5:0/297093704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092ac3b290 con 0x56092a8704d0 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.634+0000 7f5fa4843640 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac368b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f9002a910 tx=0x7f5f90031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70c32ba2784c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.635+0000 7f5f8affd640 1 -- 172.21.15.5:0/297093704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90051030 con 0x56092a8704d0 2024-05-14T15:26:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.635+0000 7f5f8affd640 1 -- 172.21.15.5:0/297093704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f90031c00 con 0x56092a8704d0 2024-05-14T15:26:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.635+0000 7f5f8affd640 1 -- 172.21.15.5:0/297093704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9004c480 con 0x56092a8704d0 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 msgr2=0x56092ac368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac368b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f9002a910 tx=0x7f5f90031400 comp rx=0 tx=0).stop 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 shutdown_connections 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8911b0 0x56092ac53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8704d0 0x56092ac368b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 --2- 172.21.15.5:0/297093704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8682f0 0x56092ac3bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.636+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 >> 172.21.15.5:0/297093704 conn(0x56092a885300 msgr2=0x56092ac3b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.637+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 shutdown_connections 2024-05-14T15:26:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.637+0000 7f5faddfd740 1 -- 172.21.15.5:0/297093704 wait complete. 2024-05-14T15:26:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.637+0000 7f5faddfd740 1 -- start start 2024-05-14T15:26:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.637+0000 7f5faddfd740 1 Processor -- start 2024-05-14T15:26:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.638+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8682f0 0x56092ac3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.638+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8704d0 0x56092ac3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.638+0000 7f5faddfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 0x56092ac3fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5044640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8682f0 0x56092ac3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5044640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8682f0 0x56092ac3f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45048/0 (socket says 172.21.15.5:45048) 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8704d0 0x56092ac3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa4843640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8704d0 0x56092ac3f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49890/0 (socket says 172.21.15.5:49890) 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 0x56092ac3fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 0x56092ac3fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56500/0 (socket says 172.21.15.5:56500) 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5044640 1 -- 172.21.15.5:0/1299133891 learned_addr learned my addr 172.21.15.5:0/1299133891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0083ad0 con 0x56092a8911b0 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01aecb0 con 0x56092a8704d0 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01adf10 con 0x56092a8682f0 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 -- 172.21.15.5:0/1299133891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8682f0 msgr2=0x56092ac3f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8682f0 0x56092ac3f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 -- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8704d0 msgr2=0x56092ac3f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8704d0 0x56092ac3f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.639+0000 7f5fa5845640 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092acb4840 con 0x56092a8911b0 2024-05-14T15:26:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.640+0000 7f5fa5845640 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 0x56092ac3fb60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x56092ac32a90 tx=0x7f5f98009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe106733d5c1bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.640+0000 7f5f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.640+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1299133891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98037030 con 0x56092a8911b0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.640+0000 7f5faddfd740 10 client.0 _ll_register_callbacks cb 0x7f5f90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.641+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f90031c00 con 0x56092a8911b0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.641+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1299133891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5f9801cdc0 con 0x56092a8911b0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.641+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1299133891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f98044050 con 0x56092a8911b0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.641+0000 7f5faddfd740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.641+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=26}) -- 0x56092acb5790 con 0x56092a8911b0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1299133891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5f980268e0 con 0x56092a8911b0 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5faddfd740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5faddfd740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5faddfd740 20 client.15726 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a8251c0 0x56092acb7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5faddfd740 4 client.15726 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:26:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5f90031c00 con 0x56092a8251c0 2024-05-14T15:26:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.642+0000 7f5fa5044640 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a8251c0 0x56092acb7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.643+0000 7f5fa5044640 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a8251c0 0x56092acb7dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9e1e65bb446f72 server_cookie=7909c962fee3a404 in_seq=0 out_seq=0 2024-05-14T15:26:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.643+0000 7f5f88ff9640 10 client.15726 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.649+0000 7f5f88ff9640 1 -- 172.21.15.5:0/1299133891 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5f90031c00 con 0x56092a8251c0 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.649+0000 7f5f88ff9640 10 client.15726 handle_command_reply: tid=0 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.649+0000 7f5faddfd740 1 client.15726 shutdown 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.649+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 msgr2=0x56092ac3fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.649+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 0x56092ac3fb60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x56092ac32a90 tx=0x7f5f98009500 comp rx=0 tx=0).stop 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 shutdown_connections 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56092a8251c0 0x56092acb7dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56092a8911b0 0x56092ac3fb60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56092a8704d0 0x56092ac3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 --2- 172.21.15.5:0/1299133891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56092a8682f0 0x56092ac3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 >> 172.21.15.5:0/1299133891 conn(0x56092a885300 msgr2=0x56092ac31970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5f88ff9640 0 client.15726 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 shutdown_connections 2024-05-14T15:26:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 1 -- 172.21.15.5:0/1299133891 wait complete. 2024-05-14T15:26:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.650+0000 7f5faddfd740 20 client.15726 trim_cache size 0 max 16384 2024-05-14T15:26:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:26:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:26:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:26:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f8c0b2d00 msgr2=0x7f5f8c0b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f8c0b2d00 0x7f5f8c0b5210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009f10 tx=0x7f5f98009b90 comp rx=0 tx=0).stop 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 msgr2=0x7f5fa01a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 0x7f5fa01a0e60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00d950 tx=0x7f5f9c00de10 comp rx=0 tx=0).stop 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 shutdown_connections 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f8c0b2d00 0x7f5f8c0b5210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fa010eac0 0x7f5fa01a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa010f580 0x7f5fa01a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 --2- 172.21.15.5:0/2794622562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fa010a530 0x7f5fa01a0e60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 >> 172.21.15.5:0/2794622562 conn(0x7f5fa00761d0 msgr2=0x7f5fa0107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 shutdown_connections 2024-05-14T15:26:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:12.651+0000 7f5fa72a8640 1 -- 172.21.15.5:0/2794622562 wait complete. 2024-05-14T15:26:14.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:13 smithi144 ceph-mon[35650]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:14.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:13 smithi193 ceph-mon[36001]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:14.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:13 smithi005 ceph-mon[31559]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-14T15:26:14.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:26:14.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:26:14.711 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:16.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:15 smithi005 ceph-mon[31559]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T15:26:16.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:15 smithi144 ceph-mon[35650]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T15:26:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:15 smithi193 ceph-mon[36001]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T15:26:16.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 -- 172.21.15.5:0/2855052393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 msgr2=0x7f6aa8104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 --2- 172.21.15.5:0/2855052393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa8104a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6a980099c0 tx=0x7f6a98009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 -- 172.21.15.5:0/2855052393 shutdown_connections 2024-05-14T15:26:16.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 --2- 172.21.15.5:0/2855052393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 --2- 172.21.15.5:0/2855052393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa8104a00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 --2- 172.21.15.5:0/2855052393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa8108f20 0x7f6aa8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 -- 172.21.15.5:0/2855052393 >> 172.21.15.5:0/2855052393 conn(0x7f6aa8100040 msgr2=0x7f6aa8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:16.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 -- 172.21.15.5:0/2855052393 shutdown_connections 2024-05-14T15:26:16.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.663+0000 7f6aacffd700 1 -- 172.21.15.5:0/2855052393 wait complete. 2024-05-14T15:26:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.664+0000 7f6aacffd700 1 Processor -- start 2024-05-14T15:26:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.664+0000 7f6aacffd700 1 -- start start 2024-05-14T15:26:16.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aacffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aacffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa8108f20 0x7f6aa819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aacffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aacffd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81144a0 con 0x7f6aa81045f0 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aacffd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8114320 con 0x7f6aa8108f20 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aacffd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8114620 con 0x7f6aa8109870 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54038/0 (socket says 172.21.15.5:54038) 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/3833805899 learned_addr learned my addr 172.21.15.5:0/3833805899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.665+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa8108f20 0x7f6aa819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/3833805899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa8108f20 msgr2=0x7f6aa819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa8108f20 0x7f6aa819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa659c700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 msgr2=0x7f6aa819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/3833805899 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa819fd20 con 0x7f6aa8109870 2024-05-14T15:26:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa659c700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:26:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa819f680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0007ee0 tx=0x7f6aa000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6a977fe700 1 -- 172.21.15.5:0/3833805899 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000ddd0 con 0x7f6aa8109870 2024-05-14T15:26:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6a977fe700 1 -- 172.21.15.5:0/3833805899 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa0004020 con 0x7f6aa8109870 2024-05-14T15:26:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa81a0010 con 0x7f6aa8109870 2024-05-14T15:26:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6a977fe700 1 -- 172.21.15.5:0/3833805899 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0015660 con 0x7f6aa8109870 2024-05-14T15:26:16.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.666+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa81a0440 con 0x7f6aa8109870 2024-05-14T15:26:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.670+0000 7f6a977fe700 1 -- 172.21.15.5:0/3833805899 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7f6aa0015800 con 0x7f6aa8109870 2024-05-14T15:26:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.672+0000 7f6abba11b80 1 Processor -- start 2024-05-14T15:26:16.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.672+0000 7f6a977fe700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6a900b2ba0 0x7f6a900b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.672+0000 7f6abba11b80 1 -- start start 2024-05-14T15:26:16.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.672+0000 7f6a977fe700 1 -- 172.21.15.5:0/3833805899 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6aa00e26e0 con 0x7f6aa8109870 2024-05-14T15:26:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.672+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6a900b2ba0 0x7f6a900b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6a900b2ba0 0x7f6a900b50b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8103ae0 tx=0x7f6a9c0073d0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d13ffec0 0x5567d157d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 0x5567d1581fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1582900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6abba11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5567d149f8e0 con 0x5567d157dce0 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6abba11b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567d143e840 con 0x5567d13ffec0 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6abba11b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567d149b8a0 con 0x5567d15824f0 2024-05-14T15:26:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 0x5567d1581fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 0x5567d1581fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37488/0 (socket says 172.21.15.5:37488) 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/630155520 learned_addr learned my addr 172.21.15.5:0/630155520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.673+0000 7f6aa659c700 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d13ffec0 0x5567d157d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/630155520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 msgr2=0x5567d1582900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1582900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/630155520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d13ffec0 msgr2=0x5567d157d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d13ffec0 0x5567d157d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/630155520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d13ec950 con 0x5567d157dce0 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa659c700 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d13ffec0 0x5567d157d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 0x5567d1581fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c004500 tx=0x7f6a9c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e3237a16d30ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6a94ff9700 1 -- 172.21.15.5:0/630155520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c024070 con 0x5567d157dce0 2024-05-14T15:26:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6a94ff9700 1 -- 172.21.15.5:0/630155520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a9c015070 con 0x5567d157dce0 2024-05-14T15:26:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.674+0000 7f6a94ff9700 1 -- 172.21.15.5:0/630155520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c011830 con 0x5567d157dce0 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 -- 172.21.15.5:0/630155520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 msgr2=0x5567d1581fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 0x5567d1581fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c004500 tx=0x7f6a9c013000 comp rx=0 tx=0).stop 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 -- 172.21.15.5:0/630155520 shutdown_connections 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1582900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d13ffec0 0x5567d157d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 --2- 172.21.15.5:0/630155520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d157dce0 0x5567d1581fb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 -- 172.21.15.5:0/630155520 >> 172.21.15.5:0/630155520 conn(0x5567d1447680 msgr2=0x5567d1578530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 -- 172.21.15.5:0/630155520 shutdown_connections 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.676+0000 7f6abba11b80 1 -- 172.21.15.5:0/630155520 wait complete. 2024-05-14T15:26:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 -- start start 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 Processor -- start 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1587370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1587df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6aa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1587370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5567d158a4a0 con 0x5567d13ffec0 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567d1405cb0 con 0x5567d157dce0 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6abba11b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567d158a620 con 0x5567d15824f0 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6aa659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1587370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37496/0 (socket says 172.21.15.5:37496) 2024-05-14T15:26:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.677+0000 7f6aa659c700 1 -- 172.21.15.5:0/834294980 learned_addr learned my addr 172.21.15.5:0/834294980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6abba11b80 10 client.0 _ll_register_callbacks cb 0x5567d14379a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1587df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa659c700 1 -- 172.21.15.5:0/834294980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 msgr2=0x5567d1587df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa659c700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1587df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa659c700 1 -- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 msgr2=0x5567d15878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa659c700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa659c700 1 -- 172.21.15.5:0/834294980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d143b710 con 0x5567d13ffec0 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6aa659c700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1587370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009990 tx=0x7f6a98003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa6cc82a0e1ad2e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.678+0000 7f6a8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.679+0000 7f6a8dffb700 1 -- 172.21.15.5:0/834294980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98027060 con 0x5567d13ffec0 2024-05-14T15:26:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.679+0000 7f6a8dffb700 1 -- 172.21.15.5:0/834294980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6a98003da0 con 0x5567d13ffec0 2024-05-14T15:26:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.679+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9c015070 con 0x5567d13ffec0 2024-05-14T15:26:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.679+0000 7f6a8dffb700 1 -- 172.21.15.5:0/834294980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a9802e050 con 0x5567d13ffec0 2024-05-14T15:26:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.679+0000 7f6abba11b80 10 client.15735 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.679+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567d159d800 con 0x5567d13ffec0 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6a8dffb700 1 -- 172.21.15.5:0/834294980 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a9c015070 con 0x5567d13ffec0 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6a8dffb700 1 -- 172.21.15.5:0/834294980 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6a9800f8f0 con 0x5567d13ffec0 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6abba11b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6abba11b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6abba11b80 20 client.15735 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6abba11b80 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d159fe40 0x5567d15a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6abba11b80 4 client.15735 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567d14ad2c0 con 0x5567d159fe40 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.680+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d159fe40 0x5567d15a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.681+0000 7f6a8dffb700 10 client.15735 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.681+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d159fe40 0x5567d15a2280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1695e4bcbb612e74 server_cookie=cf9f6f801a282075 in_seq=0 out_seq=0 2024-05-14T15:26:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6a8dffb700 1 -- 172.21.15.5:0/834294980 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567d14ad2c0 con 0x5567d159fe40 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6a8dffb700 10 client.15735 handle_command_reply: tid=0 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 client.15735 shutdown 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 msgr2=0x5567d1587370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1587370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009990 tx=0x7f6a98003730 comp rx=0 tx=0).stop 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 shutdown_connections 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d159fe40 0x5567d15a2280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d15824f0 0x5567d1587df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6a8dffb700 0 client.15735 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1587370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 --2- 172.21.15.5:0/834294980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 >> 172.21.15.5:0/834294980 conn(0x5567d1447680 msgr2=0x5567d157c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 shutdown_connections 2024-05-14T15:26:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.682+0000 7f6abba11b80 1 -- 172.21.15.5:0/834294980 wait complete. 2024-05-14T15:26:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.683+0000 7f6abba11b80 20 client.15735 trim_cache size 0 max 16384 2024-05-14T15:26:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.724+0000 7f6abba11b80 1 Processor -- start 2024-05-14T15:26:16.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.724+0000 7f6abba11b80 1 -- start start 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1578690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d157dce0 0x5567d1572870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 0x5567d1587c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6abba11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81b0dd0 con 0x5567d13ffec0 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6abba11b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81ae660 con 0x5567d15824f0 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6abba11b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81ac4b0 con 0x5567d157dce0 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 0x5567d1587c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 0x5567d1587c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50534/0 (socket says 172.21.15.5:50534) 2024-05-14T15:26:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/1363580978 learned_addr learned my addr 172.21.15.5:0/1363580978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.725+0000 7f6aa659c700 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1578690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d157dce0 0x5567d1572870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/1363580978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d157dce0 msgr2=0x5567d1572870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d157dce0 0x5567d1572870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/1363580978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 msgr2=0x5567d1578690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1578690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa6d9d700 1 -- 172.21.15.5:0/1363580978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d1589e60 con 0x5567d15824f0 2024-05-14T15:26:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa659c700 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1578690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.726+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 0x5567d1587c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00df920 tx=0x7f6aa00a8fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca35f69d90cf41ef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.727+0000 7f6a8ffff700 1 -- 172.21.15.5:0/1363580978 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00ad770 con 0x5567d15824f0 2024-05-14T15:26:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.727+0000 7f6a8ffff700 1 -- 172.21.15.5:0/1363580978 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa00b2220 con 0x5567d15824f0 2024-05-14T15:26:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.727+0000 7f6a8ffff700 1 -- 172.21.15.5:0/1363580978 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0004e30 con 0x5567d15824f0 2024-05-14T15:26:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 -- 172.21.15.5:0/1363580978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 msgr2=0x5567d1587c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 0x5567d1587c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00df920 tx=0x7f6aa00a8fa0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 -- 172.21.15.5:0/1363580978 shutdown_connections 2024-05-14T15:26:16.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d15824f0 0x5567d1587c30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13ffec0 0x5567d1578690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 --2- 172.21.15.5:0/1363580978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d157dce0 0x5567d1572870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 -- 172.21.15.5:0/1363580978 >> 172.21.15.5:0/1363580978 conn(0x5567d1447680 msgr2=0x5567d157c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.728+0000 7f6abba11b80 1 -- 172.21.15.5:0/1363580978 shutdown_connections 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.729+0000 7f6abba11b80 1 -- 172.21.15.5:0/1363580978 wait complete. 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.729+0000 7f6abba11b80 1 -- start start 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.729+0000 7f6abba11b80 1 Processor -- start 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d13ffec0 0x5567d1598450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6abba11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 0x5567d1596d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6aa659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d13ffec0 0x5567d1598450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6aa659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d13ffec0 0x5567d1598450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54086/0 (socket says 172.21.15.5:54086) 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 0x5567d1596d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6aa6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 0x5567d1596d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37520/0 (socket says 172.21.15.5:37520) 2024-05-14T15:26:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6aa5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6aa659c700 1 -- 172.21.15.5:0/3574820023 learned_addr learned my addr 172.21.15.5:0/3574820023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81acb20 con 0x5567d15824f0 2024-05-14T15:26:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81adf90 con 0x5567d157dce0 2024-05-14T15:26:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.730+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa81ad1f0 con 0x5567d13ffec0 2024-05-14T15:26:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/3574820023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d13ffec0 msgr2=0x5567d1598450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d13ffec0 0x5567d1598450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/3574820023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 msgr2=0x5567d1596d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 0x5567d1596d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa5d9b700 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d1597260 con 0x5567d157dce0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa6d9d700 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 0x5567d1596d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6abba11b80 10 client.0 _ll_register_callbacks cb 0x5567d14379a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6aa5d9b700 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15967e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5567d1572150 tx=0x7f6a9c010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254712b5a51ef280 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa00b2220 con 0x5567d157dce0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6a8dffb700 10 client.25394 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3574820023 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c035060 con 0x5567d157dce0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.731+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3574820023 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6a9c005240 con 0x5567d157dce0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3574820023 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c024040 con 0x5567d157dce0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6abba11b80 10 client.25394 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5567d160b070 con 0x5567d157dce0 2024-05-14T15:26:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3574820023 <== mon.2 v2:172.21.15.144:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6a9c02a040 con 0x5567d157dce0 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6abba11b80 10 client.25394 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6abba11b80 10 client.25394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.732+0000 7f6abba11b80 20 client.25394 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.733+0000 7f6abba11b80 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d160b2a0 0x5567d160d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.733+0000 7f6abba11b80 4 client.25394 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.733+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f6aa00b2220 con 0x5567d160b2a0 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.733+0000 7f6aa659c700 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d160b2a0 0x5567d160d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.733+0000 7f6a8dffb700 10 client.25394 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.733+0000 7f6aa659c700 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d160b2a0 0x5567d160d6e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7dccd0da2d7f8d server_cookie=82c10c7695308f6b in_seq=0 out_seq=0 2024-05-14T15:26:16.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.734+0000 7f6a8dffb700 1 -- 172.21.15.5:0/3574820023 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f6aa00b2220 con 0x5567d160b2a0 2024-05-14T15:26:16.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.734+0000 7f6a8dffb700 10 client.25394 handle_command_reply: tid=0 2024-05-14T15:26:16.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.734+0000 7f6abba11b80 1 client.25394 shutdown 2024-05-14T15:26:16.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 msgr2=0x5567d15967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15967e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x5567d1572150 tx=0x7f6a9c010f00 comp rx=0 tx=0).stop 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 shutdown_connections 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5567d160b2a0 0x5567d160d6e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d15824f0 0x5567d1596d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567d13ffec0 0x5567d1598450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 --2- 172.21.15.5:0/3574820023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567d157dce0 0x5567d15967e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 >> 172.21.15.5:0/3574820023 conn(0x5567d1447680 msgr2=0x5567d15783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.735+0000 7f6a8dffb700 0 client.25394 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.736+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 shutdown_connections 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.736+0000 7f6abba11b80 1 -- 172.21.15.5:0/3574820023 wait complete. 2024-05-14T15:26:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.736+0000 7f6abba11b80 20 client.25394 trim_cache size 0 max 16384 2024-05-14T15:26:16.739 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:26:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:26:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "0ac09f76-21f0-40f6-a400-f1f7fe79bac5", 2024-05-14T15:26:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:26:16.739 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6a900b2ba0 msgr2=0x7f6a900b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6a900b2ba0 0x7f6a900b50b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8103ae0 tx=0x7f6a9c0073d0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 msgr2=0x7f6aa819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa819f680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0007ee0 tx=0x7f6aa000d3b0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 shutdown_connections 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6a900b2ba0 0x7f6a900b50b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aa8109870 0x7f6aa819f680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aa81045f0 0x7f6aa819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 --2- 172.21.15.5:0/3833805899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6aa8108f20 0x7f6aa819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.737+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 >> 172.21.15.5:0/3833805899 conn(0x7f6aa8100040 msgr2=0x7f6aa81086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.738+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 shutdown_connections 2024-05-14T15:26:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:16.738+0000 7f6aacffd700 1 -- 172.21.15.5:0/3833805899 wait complete. 2024-05-14T15:26:17.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:26:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:18.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:17 smithi005 ceph-mon[31559]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:26:18.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:17 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:26:18.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:17 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:26:18.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:17 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:26:18.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:17 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:26:18.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:17 smithi144 ceph-mon[35650]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:26:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:17 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:26:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:17 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:26:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:17 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:26:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:17 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:26:18.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:17 smithi193 ceph-mon[36001]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-14T15:26:18.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:17 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:26:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:17 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:26:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:17 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:26:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:17 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:26:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 -- 172.21.15.5:0/1060591152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 msgr2=0x7f177410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1060591152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177410f890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f176c00b0a0 tx=0x7f176c00b3b0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 -- 172.21.15.5:0/1060591152 shutdown_connections 2024-05-14T15:26:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1060591152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177410f890 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1060591152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17741089f0 0x7f1774108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1060591152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17741029c0 0x7f1774102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.838+0000 7f177c0bb700 1 -- 172.21.15.5:0/1060591152 >> 172.21.15.5:0/1060591152 conn(0x7f17740fe6f0 msgr2=0x7f1774100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.839+0000 7f177c0bb700 1 -- 172.21.15.5:0/1060591152 shutdown_connections 2024-05-14T15:26:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.839+0000 7f177c0bb700 1 -- 172.21.15.5:0/1060591152 wait complete. 2024-05-14T15:26:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 Processor -- start 2024-05-14T15:26:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 -- start start 2024-05-14T15:26:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17741029c0 0x7f177419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17741089f0 0x7f177419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1774112b20 con 0x7f1774103310 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17741129a0 con 0x7f17741029c0 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f177c0bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1774112ca0 con 0x7f17741089f0 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.840+0000 7f1779656700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37536/0 (socket says 172.21.15.5:37536) 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 -- 172.21.15.5:0/1792986089 learned_addr learned my addr 172.21.15.5:0/1792986089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779e57700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17741029c0 0x7f177419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f177a658700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17741089f0 0x7f177419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 -- 172.21.15.5:0/1792986089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17741089f0 msgr2=0x7f177419c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17741089f0 0x7f177419c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 -- 172.21.15.5:0/1792986089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17741029c0 msgr2=0x7f177419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17741029c0 0x7f177419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 -- 172.21.15.5:0/1792986089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17741a6ff0 con 0x7f1774103310 2024-05-14T15:26:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.841+0000 7f1779656700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177419c1e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f176400e970 tx=0x7f176400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.842+0000 7f176affd700 1 -- 172.21.15.5:0/1792986089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176400cbd0 con 0x7f1774103310 2024-05-14T15:26:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.842+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17741a72e0 con 0x7f1774103310 2024-05-14T15:26:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.842+0000 7f176affd700 1 -- 172.21.15.5:0/1792986089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176400cd70 con 0x7f1774103310 2024-05-14T15:26:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.842+0000 7f176affd700 1 -- 172.21.15.5:0/1792986089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764010430 con 0x7f1774103310 2024-05-14T15:26:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.842+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17741a7870 con 0x7f1774103310 2024-05-14T15:26:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.846+0000 7f176affd700 1 -- 172.21.15.5:0/1792986089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 149935+0+0 (secure 0 0 0) 0x7f17640105d0 con 0x7f1774103310 2024-05-14T15:26:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.846+0000 7f178aacfb80 1 Processor -- start 2024-05-14T15:26:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 -- start start 2024-05-14T15:26:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 0x55a455355720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a455359f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a45535a470 0x55a45535a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a455277860 con 0x55a455355c60 2024-05-14T15:26:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a4552167c0 con 0x55a4551d7e40 2024-05-14T15:26:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f178aacfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a455273820 con 0x55a45535a470 2024-05-14T15:26:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f177a658700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 0x55a455355720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f177a658700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 0x55a455355720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50578/0 (socket says 172.21.15.5:50578) 2024-05-14T15:26:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f177a658700 1 -- 172.21.15.5:0/3503720980 learned_addr learned my addr 172.21.15.5:0/3503720980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.847+0000 7f1779e57700 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a455359f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f1779656700 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a45535a470 0x55a45535a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f177a658700 1 -- 172.21.15.5:0/3503720980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a45535a470 msgr2=0x55a45535a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f177a658700 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a45535a470 0x55a45535a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f177a658700 1 -- 172.21.15.5:0/3503720980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 msgr2=0x55a455359f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f177a658700 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a455359f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f177a658700 1 -- 172.21.15.5:0/3503720980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4551c48d0 con 0x55a4551d7e40 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f177a658700 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 0x55a455355720 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f176c00b0a0 tx=0x7f176c009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1234224bf3282f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.848+0000 7f176affd700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17600b2ba0 0x7f17600b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.849+0000 7f1779e57700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17600b2ba0 0x7f17600b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.849+0000 7f175ffff700 1 -- 172.21.15.5:0/3503720980 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c00bab0 con 0x55a4551d7e40 2024-05-14T15:26:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.849+0000 7f175ffff700 1 -- 172.21.15.5:0/3503720980 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c00bc50 con 0x55a4551d7e40 2024-05-14T15:26:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.849+0000 7f175ffff700 1 -- 172.21.15.5:0/3503720980 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c007d30 con 0x55a4551d7e40 2024-05-14T15:26:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.849+0000 7f176affd700 1 -- 172.21.15.5:0/1792986089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f17640e2fd0 con 0x7f1774103310 2024-05-14T15:26:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.849+0000 7f1779e57700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17600b2ba0 0x7f17600b50b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55a455351b70 tx=0x7f1770009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.850+0000 7f178aacfb80 1 -- 172.21.15.5:0/3503720980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 msgr2=0x55a455355720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.850+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 0x55a455355720 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f176c00b0a0 tx=0x7f176c009d90 comp rx=0 tx=0).stop 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 -- 172.21.15.5:0/3503720980 shutdown_connections 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a45535a470 0x55a45535a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a4551d7e40 0x55a455355720 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3503720980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a455359f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 -- 172.21.15.5:0/3503720980 >> 172.21.15.5:0/3503720980 conn(0x55a45521f600 msgr2=0x55a4553504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 -- 172.21.15.5:0/3503720980 shutdown_connections 2024-05-14T15:26:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 -- 172.21.15.5:0/3503720980 wait complete. 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.851+0000 7f178aacfb80 1 -- start start 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 Processor -- start 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455365e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a455355c60 0x55a45535f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 0x55a45535f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4553667d0 con 0x55a45535a470 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a4551ddc30 con 0x55a455355c60 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a455366950 con 0x55a4551d7e40 2024-05-14T15:26:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f178aacfb80 10 client.0 _ll_register_callbacks cb 0x55a45520f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f177a658700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 0x55a45535f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f177a658700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 0x55a45535f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37556/0 (socket says 172.21.15.5:37556) 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f177a658700 1 -- 172.21.15.5:0/3972814323 learned_addr learned my addr 172.21.15.5:0/3972814323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.852+0000 7f1779656700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a455355c60 0x55a45535f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f1779e57700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455365e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f177a658700 1 -- 172.21.15.5:0/3972814323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 msgr2=0x55a455365e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f177a658700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455365e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f177a658700 1 -- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a455355c60 msgr2=0x55a45535f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f177a658700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a455355c60 0x55a45535f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f177a658700 1 -- 172.21.15.5:0/3972814323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a455213690 con 0x55a45535a470 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f177a658700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 0x55a45535f6f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d010 tx=0x7f176c0077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5999cfd79f462d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f175d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f175d7fa700 1 -- 172.21.15.5:0/3972814323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c007930 con 0x55a45535a470 2024-05-14T15:26:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f175d7fa700 1 -- 172.21.15.5:0/3972814323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f176c007ad0 con 0x55a45535a470 2024-05-14T15:26:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.853+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c00bc50 con 0x55a45535a470 2024-05-14T15:26:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f175d7fa700 1 -- 172.21.15.5:0/3972814323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f176c039050 con 0x55a45535a470 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 10 client.15753 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a455364560 con 0x55a45535a470 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f175d7fa700 1 -- 172.21.15.5:0/3972814323 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c00bc50 con 0x55a45535a470 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f175d7fa700 1 -- 172.21.15.5:0/3972814323 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f176c011070 con 0x55a45535a470 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 20 client.15753 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a455377ce0 0x55a45537a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 4 client.15753 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.854+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a455285240 con 0x55a455377ce0 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.855+0000 7f1779e57700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a455377ce0 0x55a45537a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.855+0000 7f175d7fa700 10 client.15753 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.855+0000 7f1779e57700 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a455377ce0 0x55a45537a100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59cc475a7d91525e server_cookie=f6f6e3e93a022d75 in_seq=0 out_seq=0 2024-05-14T15:26:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f175d7fa700 1 -- 172.21.15.5:0/3972814323 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a455285240 con 0x55a455377ce0 2024-05-14T15:26:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f175d7fa700 10 client.15753 handle_command_reply: tid=0 2024-05-14T15:26:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 client.15753 shutdown 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 msgr2=0x55a45535f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 0x55a45535f6f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d010 tx=0x7f176c0077f0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 shutdown_connections 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a455377ce0 0x55a45537a100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a45535a470 0x55a45535f6f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455365e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 --2- 172.21.15.5:0/3972814323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a455355c60 0x55a45535f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f175d7fa700 0 client.15753 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.856+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 >> 172.21.15.5:0/3972814323 conn(0x55a45521f600 msgr2=0x55a455354450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.857+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 shutdown_connections 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.857+0000 7f178aacfb80 1 -- 172.21.15.5:0/3972814323 wait complete. 2024-05-14T15:26:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.857+0000 7f178aacfb80 20 client.15753 trim_cache size 0 max 16384 2024-05-14T15:26:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 Processor -- start 2024-05-14T15:26:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 -- start start 2024-05-14T15:26:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455359ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a45534fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a455350120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17741b0c50 con 0x55a455355c60 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17741ae4e0 con 0x55a45535a470 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f178aacfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17741ac330 con 0x55a4551d7e40 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f1779e57700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455359ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f1779e57700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455359ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54142/0 (socket says 172.21.15.5:54142) 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f1779656700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a45534fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f1779e57700 1 -- 172.21.15.5:0/2758747347 learned_addr learned my addr 172.21.15.5:0/2758747347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.897+0000 7f177a658700 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a455350120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f1779e57700 1 -- 172.21.15.5:0/2758747347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 msgr2=0x55a455350120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f1779e57700 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a455350120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f1779e57700 1 -- 172.21.15.5:0/2758747347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 msgr2=0x55a45534fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f1779e57700 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a45534fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f1779e57700 1 -- 172.21.15.5:0/2758747347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a455351320 con 0x55a4551d7e40 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f1779e57700 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455359ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1770003ac0 tx=0x7f1770019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3103ee672eed6e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:19.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f175f7fe700 1 -- 172.21.15.5:0/2758747347 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177001e8b0 con 0x55a4551d7e40 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f175f7fe700 1 -- 172.21.15.5:0/2758747347 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1770040070 con 0x55a4551d7e40 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.898+0000 7f175f7fe700 1 -- 172.21.15.5:0/2758747347 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177001d670 con 0x55a4551d7e40 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 -- 172.21.15.5:0/2758747347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 msgr2=0x55a455359ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455359ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1770003ac0 tx=0x7f1770019000 comp rx=0 tx=0).stop 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 -- 172.21.15.5:0/2758747347 shutdown_connections 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a455350120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4551d7e40 0x55a455359ec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 --2- 172.21.15.5:0/2758747347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a455355c60 0x55a45534fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 -- 172.21.15.5:0/2758747347 >> 172.21.15.5:0/2758747347 conn(0x55a45521f600 msgr2=0x55a4553597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 -- 172.21.15.5:0/2758747347 shutdown_connections 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 -- 172.21.15.5:0/2758747347 wait complete. 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 -- start start 2024-05-14T15:26:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.900+0000 7f178aacfb80 1 Processor -- start 2024-05-14T15:26:19.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 0x55a455373e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a455355c60 0x55a4553743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a4553da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17741ac9a0 con 0x55a4551d7e40 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17741ade10 con 0x55a45535a470 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17741ad070 con 0x55a455355c60 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f1779e57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 0x55a455373e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f1779e57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 0x55a455373e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37574/0 (socket says 172.21.15.5:37574) 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f1779e57700 1 -- 172.21.15.5:0/4165657768 learned_addr learned my addr 172.21.15.5:0/4165657768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f178aacfb80 10 client.0 _ll_register_callbacks cb 0x55a45520f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f1779656700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a455355c60 0x55a4553743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.901+0000 7f177a658700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a4553da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f1779e57700 1 -- 172.21.15.5:0/4165657768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a455355c60 msgr2=0x55a4553743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f1779e57700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a455355c60 0x55a4553743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f1779e57700 1 -- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 msgr2=0x55a4553da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f1779e57700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a4553da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f1779e57700 1 -- 172.21.15.5:0/4165657768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4553da8c0 con 0x55a4551d7e40 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f1779e57700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 0x55a455373e70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1770005760 tx=0x7f1770005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ffdcb8d1e98955 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f175d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f175d7fa700 1 -- 172.21.15.5:0/4165657768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177002aab0 con 0x55a4551d7e40 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1770040070 con 0x55a4551d7e40 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f175d7fa700 1 -- 172.21.15.5:0/4165657768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f177001e8b0 con 0x55a4551d7e40 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.902+0000 7f175d7fa700 1 -- 172.21.15.5:0/4165657768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f177001eaf0 con 0x55a4551d7e40 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 10 client.15759 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a4553e1b90 con 0x55a4551d7e40 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f175d7fa700 1 -- 172.21.15.5:0/4165657768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1770040070 con 0x55a4551d7e40 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f175d7fa700 1 -- 172.21.15.5:0/4165657768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1770011440 con 0x55a4551d7e40 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 20 client.15759 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a4553e2230 0x55a4553e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:19.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: "0ac09f76-21f0-40f6-a400-f1f7fe79bac5": { 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0ac09f76-21f0-40f6-a400-f1f7fe79bac5", 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:26:19.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 4 client.15759 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a45527f2e0 con 0x55a4553e2230 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.903+0000 7f1779656700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a4553e2230 0x55a4553e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f175d7fa700 10 client.15759 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f1779656700 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a4553e2230 0x55a4553e4650 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cdb44956b7cafbe server_cookie=2e05703a160361d in_seq=0 out_seq=0 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f175d7fa700 1 -- 172.21.15.5:0/4165657768 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a45527f2e0 con 0x55a4553e2230 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f175d7fa700 10 client.15759 handle_command_reply: tid=0 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f178aacfb80 1 client.15759 shutdown 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 msgr2=0x55a455373e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.904+0000 7f178aacfb80 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 0x55a455373e70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1770005760 tx=0x7f1770005790 comp rx=0 tx=0).stop 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 shutdown_connections 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a4553e2230 0x55a4553e4650 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a45535a470 0x55a4553da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4551d7e40 0x55a455373e70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 --2- 172.21.15.5:0/4165657768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a455355c60 0x55a4553743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 >> 172.21.15.5:0/4165657768 conn(0x55a45521f600 msgr2=0x55a45534fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f175d7fa700 0 client.15759 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 shutdown_connections 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 1 -- 172.21.15.5:0/4165657768 wait complete. 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.905+0000 7f178aacfb80 20 client.15759 trim_cache size 0 max 16384 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17600b2ba0 msgr2=0x7f17600b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17600b2ba0 0x7f17600b50b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55a455351b70 tx=0x7f1770009260 comp rx=0 tx=0).stop 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 msgr2=0x7f177419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177419c1e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f176400e970 tx=0x7f176400ed30 comp rx=0 tx=0).stop 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 shutdown_connections 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17600b2ba0 0x7f17600b50b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17741089f0 0x7f177419c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17741029c0 0x7f177419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 --2- 172.21.15.5:0/1792986089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1774103310 0x7f177419c1e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 >> 172.21.15.5:0/1792986089 conn(0x7f17740fe6f0 msgr2=0x7f1774100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 shutdown_connections 2024-05-14T15:26:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:19.906+0000 7f177c0bb700 1 -- 172.21.15.5:0/1792986089 wait complete. 2024-05-14T15:26:19.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:19 smithi005 ceph-mon[31559]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-14T15:26:20.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:19 smithi144 ceph-mon[35650]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-14T15:26:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:19 smithi193 ceph-mon[36001]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-14T15:26:20.739 INFO:tasks.cephfs.filesystem:scrub status for tag:0ac09f76-21f0-40f6-a400-f1f7fe79bac5 - {'path': '/', 'tag': '0ac09f76-21f0-40f6-a400-f1f7fe79bac5', 'options': 'recursive,force'} 2024-05-14T15:26:21.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:20 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:26:21.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:20 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:26:21.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:20 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:26:22.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:21 smithi144 ceph-mon[35650]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-14T15:26:22.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:21 smithi193 ceph-mon[36001]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-14T15:26:22.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:21 smithi005 ceph-mon[31559]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-14T15:26:24.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:23 smithi144 ceph-mon[35650]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-14T15:26:24.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:23 smithi144 ceph-mon[35650]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-14T15:26:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:23 smithi193 ceph-mon[36001]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-14T15:26:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:23 smithi193 ceph-mon[36001]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-14T15:26:24.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:23 smithi005 ceph-mon[31559]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-14T15:26:24.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:23 smithi005 ceph-mon[31559]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-14T15:26:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 -- 172.21.15.5:0/1671620189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064108ba0 msgr2=0x7fa064102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 --2- 172.21.15.5:0/1671620189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064108ba0 0x7fa064102740 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa058009900 tx=0x7fa05802a410 comp rx=0 tx=0).stop 2024-05-14T15:26:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 -- 172.21.15.5:0/1671620189 shutdown_connections 2024-05-14T15:26:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 --2- 172.21.15.5:0/1671620189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa064109040 0x7fa064102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 --2- 172.21.15.5:0/1671620189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa06410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 --2- 172.21.15.5:0/1671620189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064108ba0 0x7fa064102740 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 -- 172.21.15.5:0/1671620189 >> 172.21.15.5:0/1671620189 conn(0x7fa0640782e0 msgr2=0x7fa064078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 -- 172.21.15.5:0/1671620189 shutdown_connections 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.809+0000 7fa06afb2640 1 -- 172.21.15.5:0/1671620189 wait complete. 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.810+0000 7fa06afb2640 1 Processor -- start 2024-05-14T15:26:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.810+0000 7fa06afb2640 1 -- start start 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06afb2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa0641a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06afb2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa064108ba0 0x7fa0641a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06afb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064109040 0x7fa06419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06afb2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa064113eb0 con 0x7fa064109040 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa068d4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa0641a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064109040 0x7fa06419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa068d4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa0641a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55836/0 (socket says 172.21.15.5:55836) 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064109040 0x7fa06419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60822/0 (socket says 172.21.15.5:60822) 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa068d4e640 1 -- 172.21.15.5:0/2068961792 learned_addr learned my addr 172.21.15.5:0/2068961792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa064113d30 con 0x7fa0641032f0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa064114030 con 0x7fa064108ba0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.811+0000 7fa063fff640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa064108ba0 0x7fa0641a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa068d4e640 1 -- 172.21.15.5:0/2068961792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa064108ba0 msgr2=0x7fa0641a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa068d4e640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa064108ba0 0x7fa0641a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa068d4e640 1 -- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064109040 msgr2=0x7fa06419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa068d4e640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064109040 0x7fa06419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa068d4e640 1 -- 172.21.15.5:0/2068961792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419ae80 con 0x7fa0641032f0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa068d4e640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa0641a05d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa058032fd0 tx=0x7fa058031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa061ffb640 1 -- 172.21.15.5:0/2068961792 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05803f030 con 0x7fa0641032f0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa061ffb640 1 -- 172.21.15.5:0/2068961792 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058004490 con 0x7fa0641032f0 2024-05-14T15:26:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419b0b0 con 0x7fa0641032f0 2024-05-14T15:26:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.812+0000 7fa061ffb640 1 -- 172.21.15.5:0/2068961792 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058035070 con 0x7fa0641032f0 2024-05-14T15:26:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.813+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06419b5a0 con 0x7fa0641032f0 2024-05-14T15:26:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.816+0000 7fa071e67740 1 Processor -- start 2024-05-14T15:26:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.816+0000 7fa071e67740 1 -- start start 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa071e67740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 0x564a84da0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa071e67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa071e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84da1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa071e67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a84d065d0 con 0x564a849d72f0 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa071e67740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564a84cbf890 con 0x564a84a001b0 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa071e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a84d02590 con 0x564a849df4d0 2024-05-14T15:26:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa061ffb640 1 -- 172.21.15.5:0/2068961792 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa058042030 con 0x7fa0641032f0 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa06954f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 0x564a84da0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa06954f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 0x564a84da0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55842/0 (socket says 172.21.15.5:55842) 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa06954f640 1 -- 172.21.15.5:0/165455111 learned_addr learned my addr 172.21.15.5:0/165455111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa068d4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.817+0000 7fa063fff640 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84da1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa06954f640 1 -- 172.21.15.5:0/165455111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 msgr2=0x564a84da1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa06954f640 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84da1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa06954f640 1 -- 172.21.15.5:0/165455111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 msgr2=0x564a84da1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa06954f640 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa06954f640 1 -- 172.21.15.5:0/165455111 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a84a18210 con 0x564a84a001b0 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa068d4e640 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa06954f640 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 0x564a84da0be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009fd0 tx=0x7fa05c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa3c437a0c0d42c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa04effd640 1 -- 172.21.15.5:0/165455111 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c004d20 con 0x564a84a001b0 2024-05-14T15:26:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.818+0000 7fa04effd640 1 -- 172.21.15.5:0/165455111 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c004050 con 0x564a84a001b0 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.819+0000 7fa04effd640 1 -- 172.21.15.5:0/165455111 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c005280 con 0x564a84a001b0 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 -- 172.21.15.5:0/165455111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 msgr2=0x564a84da0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 0x564a84da0be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009fd0 tx=0x7fa05c0118d0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa061ffb640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0540b2950 0x7fa0540b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 -- 172.21.15.5:0/165455111 shutdown_connections 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84da1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 --2- 172.21.15.5:0/165455111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a84a001b0 0x564a84da0be0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa071e67740 1 -- 172.21.15.5:0/165455111 >> 172.21.15.5:0/165455111 conn(0x564a849f4300 msgr2=0x564a84d9d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa068d4e640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0540b2950 0x7fa0540b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.820+0000 7fa061ffb640 1 -- 172.21.15.5:0/2068961792 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa058102a00 con 0x7fa0641032f0 2024-05-14T15:26:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.821+0000 7fa068d4e640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0540b2950 0x7fa0540b4e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0580ffe40 tx=0x7fa058009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.821+0000 7fa071e67740 1 -- 172.21.15.5:0/165455111 shutdown_connections 2024-05-14T15:26:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.821+0000 7fa071e67740 1 -- 172.21.15.5:0/165455111 wait complete. 2024-05-14T15:26:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.821+0000 7fa071e67740 1 -- start start 2024-05-14T15:26:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 Processor -- start 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849d72f0 0x564a84dab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84dab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a84a001b0 0x564a84db7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a84daddb0 con 0x564a84a001b0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564a84cc3370 con 0x564a849d72f0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.822+0000 7fa071e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a84dadf30 con 0x564a849df4d0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a84a001b0 0x564a84db7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a84a001b0 0x564a84db7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60828/0 (socket says 172.21.15.5:60828) 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa06954f640 1 -- 172.21.15.5:0/3507057273 learned_addr learned my addr 172.21.15.5:0/3507057273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa071e67740 10 client.0 _ll_register_callbacks cb 0x564a84a17a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84dab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa06954f640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849d72f0 0x564a84dab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 -- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849d72f0 msgr2=0x564a84dab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849d72f0 0x564a84dab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 -- 172.21.15.5:0/3507057273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a84a001b0 msgr2=0x564a84db7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a84a001b0 0x564a84db7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 -- 172.21.15.5:0/3507057273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a84ae4620 con 0x564a849df4d0 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa063fff640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84dab6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa06419bf80 tx=0x7fa050012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a75472c4990d48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.823+0000 7fa043fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.824+0000 7fa043fff640 1 -- 172.21.15.5:0/3507057273 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050004300 con 0x564a849df4d0 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.824+0000 7fa043fff640 1 -- 172.21.15.5:0/3507057273 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa050011970 con 0x564a849df4d0 2024-05-14T15:26:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.824+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c004050 con 0x564a849df4d0 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.824+0000 7fa043fff640 1 -- 172.21.15.5:0/3507057273 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa050025050 con 0x564a849df4d0 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.824+0000 7fa071e67740 10 client.25444 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.824+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564a84dba100 con 0x564a849df4d0 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa043fff640 1 -- 172.21.15.5:0/3507057273 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c004050 con 0x564a849df4d0 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa043fff640 1 -- 172.21.15.5:0/3507057273 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa050003c70 con 0x564a849df4d0 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa071e67740 10 client.25444 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa071e67740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa071e67740 20 client.25444 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa071e67740 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849bed80 0x564a84dbd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa071e67740 4 client.25444 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a84d141e0 con 0x564a849bed80 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.825+0000 7fa06954f640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849bed80 0x564a84dbd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.826+0000 7fa06954f640 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849bed80 0x564a84dbd5d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0a8ca777071ec7 server_cookie=e06d2b1bbca140ca in_seq=0 out_seq=0 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.826+0000 7fa043fff640 10 client.25444 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa043fff640 1 -- 172.21.15.5:0/3507057273 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a84d141e0 con 0x564a849bed80 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa043fff640 10 client.25444 handle_command_reply: tid=0 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 client.25444 shutdown 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 msgr2=0x564a84dab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84dab6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa06419bf80 tx=0x7fa050012b60 comp rx=0 tx=0).stop 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 shutdown_connections 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849bed80 0x564a84dbd5d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a849df4d0 0x564a84dab6d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a84a001b0 0x564a84db7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 --2- 172.21.15.5:0/3507057273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849d72f0 0x564a84dab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 >> 172.21.15.5:0/3507057273 conn(0x564a849f4300 msgr2=0x564a84da0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa043fff640 0 client.25444 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 shutdown_connections 2024-05-14T15:26:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.827+0000 7fa071e67740 1 -- 172.21.15.5:0/3507057273 wait complete. 2024-05-14T15:26:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.828+0000 7fa071e67740 20 client.25444 trim_cache size 0 max 16384 2024-05-14T15:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 Processor -- start 2024-05-14T15:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 -- start start 2024-05-14T15:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84da1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84da15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641b3c90 con 0x564a849d72f0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641b1560 con 0x564a849df4d0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa071e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641af3b0 con 0x564a84a001b0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.858+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60844/0 (socket says 172.21.15.5:60844) 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa06954f640 1 -- 172.21.15.5:0/1357025955 learned_addr learned my addr 172.21.15.5:0/1357025955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84da1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa06954f640 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84da15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 -- 172.21.15.5:0/1357025955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 msgr2=0x564a84da15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84da15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 -- 172.21.15.5:0/1357025955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 msgr2=0x564a84da58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 -- 172.21.15.5:0/1357025955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a84d9c980 con 0x564a849df4d0 2024-05-14T15:26:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa06954f640 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa063fff640 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84da1080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa06419bf80 tx=0x7fa05000f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9d34356b292b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.859+0000 7fa04e7fc640 1 -- 172.21.15.5:0/1357025955 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05002d070 con 0x564a849df4d0 2024-05-14T15:26:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.860+0000 7fa04e7fc640 1 -- 172.21.15.5:0/1357025955 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa050005370 con 0x564a849df4d0 2024-05-14T15:26:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.860+0000 7fa04e7fc640 1 -- 172.21.15.5:0/1357025955 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050022040 con 0x564a849df4d0 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 -- 172.21.15.5:0/1357025955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 msgr2=0x564a84da1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84da1080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa06419bf80 tx=0x7fa05000f8b0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 -- 172.21.15.5:0/1357025955 shutdown_connections 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84da15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84da1080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 --2- 172.21.15.5:0/1357025955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84da58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 -- 172.21.15.5:0/1357025955 >> 172.21.15.5:0/1357025955 conn(0x564a849f4300 msgr2=0x564a84da0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 -- 172.21.15.5:0/1357025955 shutdown_connections 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 -- 172.21.15.5:0/1357025955 wait complete. 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.861+0000 7fa071e67740 1 -- start start 2024-05-14T15:26:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 Processor -- start 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84dae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84dae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84daec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641afa20 con 0x564a849d72f0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641b0e90 con 0x564a849df4d0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641b00f0 con 0x564a84a001b0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84dae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa06954f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84dae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60860/0 (socket says 172.21.15.5:60860) 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa06954f640 1 -- 172.21.15.5:0/856297169 learned_addr learned my addr 172.21.15.5:0/856297169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa071e67740 10 client.0 _ll_register_callbacks cb 0x7fa050009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.862+0000 7fa063fff640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84dae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84daec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 -- 172.21.15.5:0/856297169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 msgr2=0x564a84daec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84daec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 -- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 msgr2=0x564a84dae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84dae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 -- 172.21.15.5:0/856297169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a84e23950 con 0x564a849d72f0 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa06954f640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84dae180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x564a84daa7e0 tx=0x7fa05c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa281ed7b76c62f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.863+0000 7fa043fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.864+0000 7fa043fff640 1 -- 172.21.15.5:0/856297169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c005310 con 0x564a849d72f0 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.864+0000 7fa043fff640 1 -- 172.21.15.5:0/856297169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fa05c0054b0 con 0x564a849d72f0 2024-05-14T15:26:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.864+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa050005370 con 0x564a849d72f0 2024-05-14T15:26:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.869+0000 7fa043fff640 1 -- 172.21.15.5:0/856297169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa05c0056b0 con 0x564a849d72f0 2024-05-14T15:26:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.869+0000 7fa071e67740 10 client.15777 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.869+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564a84e24b20 con 0x564a849d72f0 2024-05-14T15:26:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.869+0000 7fa043fff640 1 -- 172.21.15.5:0/856297169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa050005370 con 0x564a849d72f0 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.869+0000 7fa043fff640 1 -- 172.21.15.5:0/856297169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa05c028450 con 0x564a849d72f0 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa071e67740 10 client.15777 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa071e67740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa071e67740 20 client.15777 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa071e67740 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849941c0 0x564a84e27120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa071e67740 4 client.15777 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a84da6b60 con 0x564a849941c0 2024-05-14T15:26:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa063fff640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849941c0 0x564a84e27120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.870+0000 7fa043fff640 10 client.15777 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.871+0000 7fa063fff640 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849941c0 0x564a84e27120 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c31606a0ac030d server_cookie=7e26a8333f40c308 in_seq=0 out_seq=0 2024-05-14T15:26:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa043fff640 1 -- 172.21.15.5:0/856297169 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a84da6b60 con 0x564a849941c0 2024-05-14T15:26:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa043fff640 10 client.15777 handle_command_reply: tid=0 2024-05-14T15:26:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 client.15777 shutdown 2024-05-14T15:26:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 msgr2=0x564a84dae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84dae180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x564a84daa7e0 tx=0x7fa05c000f40 comp rx=0 tx=0).stop 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 shutdown_connections 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564a849941c0 0x564a84e27120 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a84a001b0 0x564a84daec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa043fff640 0 client.15777 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a849df4d0 0x564a84dae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 --2- 172.21.15.5:0/856297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a849d72f0 0x564a84dae180 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 >> 172.21.15.5:0/856297169 conn(0x564a849f4300 msgr2=0x564a84d9b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 shutdown_connections 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.877+0000 7fa071e67740 1 -- 172.21.15.5:0/856297169 wait complete. 2024-05-14T15:26:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.878+0000 7fa071e67740 20 client.15777 trim_cache size 0 max 16384 2024-05-14T15:26:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:26:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:26:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:26:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.878+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0540b2950 msgr2=0x7fa0540b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.878+0000 7fa06afb2640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0540b2950 0x7fa0540b4e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa0580ffe40 tx=0x7fa058009260 comp rx=0 tx=0).stop 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.878+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 msgr2=0x7fa0641a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.878+0000 7fa06afb2640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa0641a05d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa058032fd0 tx=0x7fa058031a20 comp rx=0 tx=0).stop 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 shutdown_connections 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0540b2950 0x7fa0540b4e60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa064108ba0 0x7fa0641a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064109040 0x7fa06419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 --2- 172.21.15.5:0/2068961792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0641032f0 0x7fa0641a05d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 >> 172.21.15.5:0/2068961792 conn(0x7fa0640782e0 msgr2=0x7fa0640ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 shutdown_connections 2024-05-14T15:26:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:25.879+0000 7fa06afb2640 1 -- 172.21.15.5:0/2068961792 wait complete. 2024-05-14T15:26:26.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:25 smithi144 ceph-mon[35650]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-14T15:26:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:25 smithi193 ceph-mon[36001]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-14T15:26:26.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:25 smithi005 ceph-mon[31559]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-14T15:26:28.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:27 smithi144 ceph-mon[35650]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-14T15:26:28.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:27 smithi193 ceph-mon[36001]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-14T15:26:28.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:27 smithi005 ceph-mon[31559]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-05-14T15:26:30.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:29 smithi144 ceph-mon[35650]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-14T15:26:30.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:29 smithi193 ceph-mon[36001]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-14T15:26:30.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:29 smithi005 ceph-mon[31559]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-14T15:26:32.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:31 smithi144 ceph-mon[35650]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-14T15:26:32.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:31 smithi193 ceph-mon[36001]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-14T15:26:32.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:31 smithi005 ceph-mon[31559]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s 2024-05-14T15:26:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:33 smithi144 ceph-mon[35650]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-14T15:26:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:33 smithi193 ceph-mon[36001]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-14T15:26:34.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:33 smithi005 ceph-mon[31559]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-14T15:26:36.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:35 smithi144 ceph-mon[35650]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T15:26:36.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:35 smithi193 ceph-mon[36001]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T15:26:36.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:35 smithi005 ceph-mon[31559]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T15:26:37.365 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 94% (11577/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to 'e2b755a64f0d72176b57299d02e880cf0620d780'. 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.022 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-05-14T15:26:38.023 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at e2b755a64f0 Merge PR #53755 into wip-vshankar-testing-20240509.053109-debug 2024-05-14T15:26:38.029 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-14T15:26:38.051 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-14T15:26:38.054 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-14T15:26:38.054 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-14T15:26:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:37 smithi144 ceph-mon[35650]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T15:26:38.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:37 smithi193 ceph-mon[36001]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T15:26:38.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:37 smithi005 ceph-mon[31559]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-14T15:26:38.756 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-14T15:26:38.814 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-14T15:26:38.856 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-14T15:26:38.858 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-14T15:26:38.858 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-14T15:26:38.899 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-14T15:26:38.903 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:26:38.903 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-14T15:26:38.960 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-14T15:26:38.961 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-14T15:26:38.962 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e2b755a64f0d72176b57299d02e880cf0620d780 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-14T15:26:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 -- 172.21.15.5:0/1700279964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204107ac0 msgr2=0x7f7204102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 --2- 172.21.15.5:0/1700279964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204107ac0 0x7f7204102ef0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7200009900 tx=0x7f720002a410 comp rx=0 tx=0).stop 2024-05-14T15:26:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 -- 172.21.15.5:0/1700279964 shutdown_connections 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 --2- 172.21.15.5:0/1700279964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204103430 0x7f7204103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 --2- 172.21.15.5:0/1700279964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 0x7f720410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 --2- 172.21.15.5:0/1700279964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204107ac0 0x7f7204102ef0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 -- 172.21.15.5:0/1700279964 >> 172.21.15.5:0/1700279964 conn(0x7f72040fd9d0 msgr2=0x7f72040ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 -- 172.21.15.5:0/1700279964 shutdown_connections 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 -- 172.21.15.5:0/1700279964 wait complete. 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 Processor -- start 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.984+0000 7f720aef9640 1 -- start start 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f720aef9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 0x7f72041a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f720aef9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204107ac0 0x7f72041a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f720aef9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 0x7f720419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f720aef9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7204112d90 con 0x7f7204103430 2024-05-14T15:26:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f720aef9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7204112c10 con 0x7f720410d5b0 2024-05-14T15:26:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f720aef9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7204112f10 con 0x7f7204107ac0 2024-05-14T15:26:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 0x7f72041a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 0x7f72041a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42724/0 (socket says 172.21.15.5:42724) 2024-05-14T15:26:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 -- 172.21.15.5:0/109390909 learned_addr learned my addr 172.21.15.5:0/109390909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f71fbfff640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204107ac0 0x7f72041a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7209496640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 0x7f720419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 -- 172.21.15.5:0/109390909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204107ac0 msgr2=0x7f72041a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204107ac0 0x7f72041a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 -- 172.21.15.5:0/109390909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 msgr2=0x7f720419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 0x7f720419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 -- 172.21.15.5:0/109390909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419ada0 con 0x7f7204103430 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f71fbfff640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204107ac0 0x7f72041a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7209496640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 0x7f720419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.985+0000 7f7208c95640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 0x7f72041a05e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7200005940 tx=0x7f72000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.986+0000 7f71f9ffb640 1 -- 172.21.15.5:0/109390909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720003f030 con 0x7f7204103430 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.986+0000 7f71f9ffb640 1 -- 172.21.15.5:0/109390909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7200003cb0 con 0x7f7204103430 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.986+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720419b030 con 0x7f7204103430 2024-05-14T15:26:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.986+0000 7f71f9ffb640 1 -- 172.21.15.5:0/109390909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200035070 con 0x7f7204103430 2024-05-14T15:26:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.986+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720419b570 con 0x7f7204103430 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.989+0000 7f7211da9740 1 Processor -- start 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.989+0000 7f7211da9740 1 -- start start 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.989+0000 7f71f9ffb640 1 -- 172.21.15.5:0/109390909 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7200042030 con 0x7f7204103430 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.989+0000 7f7211da9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e46fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7211da9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e46fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7211da9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e46fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7211da9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2e46635d0 con 0x55e2e433c4d0 2024-05-14T15:26:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7211da9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2e461c890 con 0x55e2e43342f0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7211da9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2e465f590 con 0x55e2e435d1b0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e46fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e46fe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42736/0 (socket says 172.21.15.5:42736) 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 -- 172.21.15.5:0/1734880499 learned_addr learned my addr 172.21.15.5:0/1734880499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7209496640 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e46fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7208c95640 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e46fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 -- 172.21.15.5:0/1734880499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 msgr2=0x55e2e46fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e46fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 -- 172.21.15.5:0/1734880499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 msgr2=0x55e2e46fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e46fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 -- 172.21.15.5:0/1734880499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2e4375210 con 0x55e2e433c4d0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f7208c95640 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e46fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71fbfff640 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e46fe660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f71f400bcd0 tx=0x7f71f400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f115ba35d11cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.990+0000 7f71f9ffb640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f71ec0b29d0 0x7f71ec0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.991+0000 7f7209496640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f71ec0b29d0 0x7f71ec0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.991+0000 7f71eaffd640 1 -- 172.21.15.5:0/1734880499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4015070 con 0x55e2e433c4d0 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.991+0000 7f71eaffd640 1 -- 172.21.15.5:0/1734880499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f401a910 con 0x55e2e433c4d0 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.991+0000 7f71f9ffb640 1 -- 172.21.15.5:0/109390909 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f7200103300 con 0x7f7204103430 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.991+0000 7f71eaffd640 1 -- 172.21.15.5:0/1734880499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4011980 con 0x55e2e433c4d0 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 -- 172.21.15.5:0/1734880499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 msgr2=0x55e2e46fe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e46fe660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f71f400bcd0 tx=0x7f71f400f730 comp rx=0 tx=0).stop 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7209496640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f71ec0b29d0 0x7f71ec0b4ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55e2e46fad00 tx=0x7f71fc008620 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 -- 172.21.15.5:0/1734880499 shutdown_connections 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e46fe660 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e46fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 --2- 172.21.15.5:0/1734880499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e46fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 -- 172.21.15.5:0/1734880499 >> 172.21.15.5:0/1734880499 conn(0x55e2e4351300 msgr2=0x55e2e46fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 -- 172.21.15.5:0/1734880499 shutdown_connections 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 -- 172.21.15.5:0/1734880499 wait complete. 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 -- start start 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.992+0000 7f7211da9740 1 Processor -- start 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e47081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4708700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e47141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2e470ad10 con 0x55e2e433c4d0 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2e4620370 con 0x55e2e43342f0 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2e470ae90 con 0x55e2e435d1b0 2024-05-14T15:26:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7211da9740 10 client.0 _ll_register_callbacks cb 0x55e2e4374a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4708700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4708700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42746/0 (socket says 172.21.15.5:42746) 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 -- 172.21.15.5:0/177586573 learned_addr learned my addr 172.21.15.5:0/177586573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7209496640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e47141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f7208c95640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e47081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 -- 172.21.15.5:0/177586573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 msgr2=0x55e2e47141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e47141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 -- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 msgr2=0x55e2e47081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e47081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 -- 172.21.15.5:0/177586573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2e4441620 con 0x55e2e433c4d0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71fbfff640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4708700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ba10 tx=0x7f71f400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc258cc0d9e4e518 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.993+0000 7f71e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f71e3fff640 1 -- 172.21.15.5:0/177586573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4028060 con 0x55e2e433c4d0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f71e3fff640 1 -- 172.21.15.5:0/177586573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f71f401acc0 con 0x55e2e433c4d0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f401a910 con 0x55e2e433c4d0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f71e3fff640 1 -- 172.21.15.5:0/177586573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71f402f050 con 0x55e2e433c4d0 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 10 client.15789 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2e4717040 con 0x55e2e433c4d0 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f71e3fff640 1 -- 172.21.15.5:0/177586573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f401a910 con 0x55e2e433c4d0 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f71e3fff640 1 -- 172.21.15.5:0/177586573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f71f4022030 con 0x55e2e433c4d0 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 10 client.15789 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 20 client.15789 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e431bd80 0x55e2e471a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 4 client.15789 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.994+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2e46711e0 con 0x55e2e431bd80 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.995+0000 7f7208c95640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e431bd80 0x55e2e471a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.995+0000 7f71e3fff640 10 client.15789 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.995+0000 7f7208c95640 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e431bd80 0x55e2e471a400 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11461769dc3e5603 server_cookie=45b4e18b35b4a147 in_seq=0 out_seq=0 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f71e3fff640 1 -- 172.21.15.5:0/177586573 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2e46711e0 con 0x55e2e431bd80 2024-05-14T15:26:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f71e3fff640 10 client.15789 handle_command_reply: tid=0 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 client.15789 shutdown 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 msgr2=0x55e2e4708700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4708700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f71f400ba10 tx=0x7f71f400bfd0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 shutdown_connections 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e431bd80 0x55e2e471a400 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e435d1b0 0x55e2e47141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4708700 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f71e3fff640 0 client.15789 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 --2- 172.21.15.5:0/177586573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e47081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 >> 172.21.15.5:0/177586573 conn(0x55e2e4351300 msgr2=0x55e2e46ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 shutdown_connections 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 1 -- 172.21.15.5:0/177586573 wait complete. 2024-05-14T15:26:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:38.996+0000 7f7211da9740 20 client.15789 trim_cache size 0 max 16384 2024-05-14T15:26:39.009 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 1 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 Processor -- start 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 -- start start 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 0x55e2e4709de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e4714130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e433c4d0 0x55e2e4714670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b3b50 con 0x55e2e431bd80 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b1420 con 0x55e2e43342f0 2024-05-14T15:26:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7211da9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72041af270 con 0x55e2e433c4d0 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 0x55e2e4709de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 0x55e2e4709de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42760/0 (socket says 172.21.15.5:42760) 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 -- 172.21.15.5:0/328830357 learned_addr learned my addr 172.21.15.5:0/328830357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f71fbfff640 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e4714130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7209496640 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e433c4d0 0x55e2e4714670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 -- 172.21.15.5:0/328830357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e433c4d0 msgr2=0x55e2e4714670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e433c4d0 0x55e2e4714670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 -- 172.21.15.5:0/328830357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 msgr2=0x55e2e4714130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e4714130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f7208c95640 1 -- 172.21.15.5:0/328830357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2e46fad30 con 0x55e2e431bd80 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.012+0000 7f71fbfff640 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e4714130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.013+0000 7f7208c95640 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 0x55e2e4709de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7200005e70 tx=0x7f720003b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fa3e1bbe21c375 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.013+0000 7f71ea7fc640 1 -- 172.21.15.5:0/328830357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200049040 con 0x55e2e431bd80 2024-05-14T15:26:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.013+0000 7f71ea7fc640 1 -- 172.21.15.5:0/328830357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720003b460 con 0x55e2e431bd80 2024-05-14T15:26:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.013+0000 7f71ea7fc640 1 -- 172.21.15.5:0/328830357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000ce5d0 con 0x55e2e431bd80 2024-05-14T15:26:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 -- 172.21.15.5:0/328830357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 msgr2=0x55e2e4709de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 0x55e2e4709de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7200005e70 tx=0x7f720003b640 comp rx=0 tx=0).stop 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 -- 172.21.15.5:0/328830357 shutdown_connections 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e433c4d0 0x55e2e4714670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e43342f0 0x55e2e4714130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 --2- 172.21.15.5:0/328830357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e431bd80 0x55e2e4709de0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 -- 172.21.15.5:0/328830357 >> 172.21.15.5:0/328830357 conn(0x55e2e4351300 msgr2=0x55e2e47096d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 -- 172.21.15.5:0/328830357 shutdown_connections 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 -- 172.21.15.5:0/328830357 wait complete. 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 -- start start 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.014+0000 7f7211da9740 1 Processor -- start 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e431bd80 0x55e2e471a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e43342f0 0x55e2e471a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4780520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72041af8e0 con 0x55e2e433c4d0 2024-05-14T15:26:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72041b0d50 con 0x55e2e431bd80 2024-05-14T15:26:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72041affb0 con 0x55e2e43342f0 2024-05-14T15:26:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 10 client.0 _ll_register_callbacks cb 0x7f71fc0084f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4780520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4780520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42762/0 (socket says 172.21.15.5:42762) 2024-05-14T15:26:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 -- 172.21.15.5:0/724307918 learned_addr learned my addr 172.21.15.5:0/724307918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f71fbfff640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e43342f0 0x55e2e471a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7208c95640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e431bd80 0x55e2e471a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 -- 172.21.15.5:0/724307918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e43342f0 msgr2=0x55e2e471a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e43342f0 0x55e2e471a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 -- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e431bd80 msgr2=0x55e2e471a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e431bd80 0x55e2e471a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 -- 172.21.15.5:0/724307918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2e471b1f0 con 0x55e2e433c4d0 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f71fbfff640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e43342f0 0x55e2e471a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7208c95640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e431bd80 0x55e2e471a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7209496640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4780520 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7204103ed0 tx=0x7f71fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273ffa119575e91e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f71e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f71e3fff640 1 -- 172.21.15.5:0/724307918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc029060 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f71e3fff640 1 -- 172.21.15.5:0/724307918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f71fc0245f0 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.015+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2e466b050 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f71e3fff640 1 -- 172.21.15.5:0/724307918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71fc030050 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 10 client.15795 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2e4781ac0 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f71e3fff640 1 -- 172.21.15.5:0/724307918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e2e466b050 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f71e3fff640 1 -- 172.21.15.5:0/724307918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f71fc017040 con 0x55e2e433c4d0 2024-05-14T15:26:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 10 client.15795 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 20 client.15795 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e435d1b0 0x55e2e4784280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 4 client.15795 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:26:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2e45e65f0 con 0x55e2e435d1b0 2024-05-14T15:26:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.016+0000 7f7208c95640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e435d1b0 0x55e2e4784280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.017+0000 7f7208c95640 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e435d1b0 0x55e2e4784280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0781786f6c3a43a server_cookie=1ec30664b0a5e967 in_seq=0 out_seq=0 2024-05-14T15:26:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.017+0000 7f71e3fff640 10 client.15795 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:39.027 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-14T15:26:39.027 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-05-14T15:26:39.027 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-14T15:26:39.027 INFO:tasks.workunit.client.0.smithi005.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.026+0000 7f71e3fff640 1 -- 172.21.15.5:0/724307918 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2e45e65f0 con 0x55e2e435d1b0 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.026+0000 7f71e3fff640 10 client.15795 handle_command_reply: tid=0 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 client.15795 shutdown 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 msgr2=0x55e2e4780520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4780520 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7204103ed0 tx=0x7f71fc004a40 comp rx=0 tx=0).stop 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 shutdown_connections 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e2e435d1b0 0x55e2e4784280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2e43342f0 0x55e2e471a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f71e3fff640 0 client.15795 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2e433c4d0 0x55e2e4780520 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 --2- 172.21.15.5:0/724307918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2e431bd80 0x55e2e471a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 >> 172.21.15.5:0/724307918 conn(0x55e2e4351300 msgr2=0x55e2e46ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 shutdown_connections 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 1 -- 172.21.15.5:0/724307918 wait complete. 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.027+0000 7f7211da9740 20 client.15795 trim_cache size 0 max 16384 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:26:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f71ec0b29d0 msgr2=0x7f71ec0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f71ec0b29d0 0x7f71ec0b4ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55e2e46fad00 tx=0x7f71fc008620 comp rx=0 tx=0).stop 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 msgr2=0x7f72041a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 0x7f72041a05e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7200005940 tx=0x7f72000042f0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 shutdown_connections 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f71ec0b29d0 0x7f71ec0b4ee0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7204107ac0 0x7f72041a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f720410d5b0 0x7f720419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 --2- 172.21.15.5:0/109390909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204103430 0x7f72041a05e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 >> 172.21.15.5:0/109390909 conn(0x7f72040fd9d0 msgr2=0x7f72040ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 shutdown_connections 2024-05-14T15:26:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:39.028+0000 7f720aef9640 1 -- 172.21.15.5:0/109390909 wait complete. 2024-05-14T15:26:39.124 INFO:tasks.workunit.client.0.smithi005.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-14T15:26:39.124 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-05-14T15:26:40.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 2973 33.51 MB/sec warmup 1 sec latency 9.102 ms 2024-05-14T15:26:40.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:39 smithi144 ceph-mon[35650]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:26:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:39 smithi193 ceph-mon[36001]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:26:40.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:39 smithi005 ceph-mon[31559]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:26:41.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 5335 22.70 MB/sec warmup 2 sec latency 13.353 ms 2024-05-14T15:26:42.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 7911 19.90 MB/sec warmup 3 sec latency 13.186 ms 2024-05-14T15:26:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:41 smithi144 ceph-mon[35650]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:26:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:41 smithi193 ceph-mon[36001]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:26:42.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:41 smithi005 ceph-mon[31559]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T15:26:43.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11017 19.58 MB/sec warmup 4 sec latency 20.241 ms 2024-05-14T15:26:44.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 13839 18.32 MB/sec warmup 5 sec latency 23.889 ms 2024-05-14T15:26:44.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:43 smithi144 ceph-mon[35650]: pgmap v59: 129 pgs: 129 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 346 KiB/s wr, 49 op/s 2024-05-14T15:26:44.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:43 smithi193 ceph-mon[36001]: pgmap v59: 129 pgs: 129 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 346 KiB/s wr, 49 op/s 2024-05-14T15:26:44.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:43 smithi005 ceph-mon[31559]: pgmap v59: 129 pgs: 129 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 346 KiB/s wr, 49 op/s 2024-05-14T15:26:45.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 16572 17.41 MB/sec warmup 6 sec latency 12.950 ms 2024-05-14T15:26:46.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18802 16.93 MB/sec warmup 7 sec latency 15.794 ms 2024-05-14T15:26:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:45 smithi144 ceph-mon[35650]: pgmap v60: 129 pgs: 129 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-05-14T15:26:46.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:45 smithi193 ceph-mon[36001]: pgmap v60: 129 pgs: 129 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-05-14T15:26:46.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:45 smithi005 ceph-mon[31559]: pgmap v60: 129 pgs: 129 active+clean; 4.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-05-14T15:26:47.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21782 16.96 MB/sec warmup 8 sec latency 20.306 ms 2024-05-14T15:26:48.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 24854 17.02 MB/sec warmup 9 sec latency 12.977 ms 2024-05-14T15:26:48.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:47 smithi144 ceph-mon[35650]: pgmap v61: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-14T15:26:48.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:47 smithi193 ceph-mon[36001]: pgmap v61: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-14T15:26:48.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:47 smithi005 ceph-mon[31559]: pgmap v61: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-14T15:26:49.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 27322 16.20 MB/sec warmup 10 sec latency 19.430 ms 2024-05-14T15:26:50.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29520 16.00 MB/sec warmup 11 sec latency 12.728 ms 2024-05-14T15:26:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:49 smithi144 ceph-mon[35650]: pgmap v62: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-05-14T15:26:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:49 smithi193 ceph-mon[36001]: pgmap v62: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-05-14T15:26:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:49 smithi005 ceph-mon[31559]: pgmap v62: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-05-14T15:26:50.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:26:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:51.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 32448 16.09 MB/sec warmup 12 sec latency 9.604 ms 2024-05-14T15:26:52.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 35453 16.19 MB/sec warmup 13 sec latency 17.215 ms 2024-05-14T15:26:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 -- 172.21.15.5:0/2844556383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810e6d0 msgr2=0x7efe48110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 --2- 172.21.15.5:0/2844556383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810e6d0 0x7efe48110af0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efe40009ca0 tx=0x7efe4002a8c0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 -- 172.21.15.5:0/2844556383 shutdown_connections 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 --2- 172.21.15.5:0/2844556383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe4810e6d0 0x7efe48110af0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 --2- 172.21.15.5:0/2844556383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481021d0 0x7efe481025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 --2- 172.21.15.5:0/2844556383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe48109d20 0x7efe48101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 -- 172.21.15.5:0/2844556383 >> 172.21.15.5:0/2844556383 conn(0x7efe480779e0 msgr2=0x7efe48077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 -- 172.21.15.5:0/2844556383 shutdown_connections 2024-05-14T15:26:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.134+0000 7efe4db84640 1 -- 172.21.15.5:0/2844556383 wait complete. 2024-05-14T15:26:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 Processor -- start 2024-05-14T15:26:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 -- start start 2024-05-14T15:26:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 0x7efe48102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe48109d20 0x7efe481033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4810e6d0 0x7efe481038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe48113ee0 con 0x7efe481021d0 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe48113d60 con 0x7efe48109d20 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.135+0000 7efe4db84640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe48114060 con 0x7efe4810e6d0 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 0x7efe48102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 0x7efe48102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51048/0 (socket says 172.21.15.5:51048) 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 -- 172.21.15.5:0/1599921292 learned_addr learned my addr 172.21.15.5:0/1599921292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe3ffff640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe48109d20 0x7efe481033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe47fff640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4810e6d0 0x7efe481038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 -- 172.21.15.5:0/1599921292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4810e6d0 msgr2=0x7efe481038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4810e6d0 0x7efe481038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 -- 172.21.15.5:0/1599921292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe48109d20 msgr2=0x7efe481033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe48109d20 0x7efe481033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 -- 172.21.15.5:0/1599921292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe481a8b80 con 0x7efe481021d0 2024-05-14T15:26:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe477fe640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 0x7efe48102e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe3800e9f0 tx=0x7efe3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe457fa640 1 -- 172.21.15.5:0/1599921292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3800ccc0 con 0x7efe481021d0 2024-05-14T15:26:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe457fa640 1 -- 172.21.15.5:0/1599921292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe380035f0 con 0x7efe481021d0 2024-05-14T15:26:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe481a8e70 con 0x7efe481021d0 2024-05-14T15:26:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe457fa640 1 -- 172.21.15.5:0/1599921292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3800ce60 con 0x7efe481021d0 2024-05-14T15:26:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.136+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe481a92e0 con 0x7efe481021d0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 Processor -- start 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 -- start start 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe457fa640 1 -- 172.21.15.5:0/1599921292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7efe38003aa0 con 0x7efe481021d0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb8455be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb8456120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb8456660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55deb83bb5d0 con 0x55deb80944d0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55deb8374890 con 0x55deb808c2f0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.139+0000 7efe54a39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55deb83b7590 con 0x55deb80b51b0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb8456660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb8456120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb8456660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51052/0 (socket says 172.21.15.5:51052) 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 -- 172.21.15.5:0/3063761466 learned_addr learned my addr 172.21.15.5:0/3063761466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe47fff640 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb8455be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 -- 172.21.15.5:0/3063761466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 msgr2=0x55deb8455be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb8455be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 -- 172.21.15.5:0/3063761466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 msgr2=0x55deb8456120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb8456120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 -- 172.21.15.5:0/3063761466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb80cd210 con 0x55deb80944d0 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3ffff640 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb8456660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efe3000bc90 tx=0x7efe300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a2af76fed45e29 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe457fa640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe340b2950 0x7efe340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3dffb640 1 -- 172.21.15.5:0/3063761466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3001c480 con 0x55deb80944d0 2024-05-14T15:26:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe47fff640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe340b2950 0x7efe340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3dffb640 1 -- 172.21.15.5:0/3063761466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe3001cad0 con 0x55deb80944d0 2024-05-14T15:26:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe3dffb640 1 -- 172.21.15.5:0/3063761466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30011b60 con 0x55deb80944d0 2024-05-14T15:26:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.140+0000 7efe457fa640 1 -- 172.21.15.5:0/1599921292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7efe380e2d80 con 0x7efe481021d0 2024-05-14T15:26:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 -- 172.21.15.5:0/3063761466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 msgr2=0x55deb8456660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe47fff640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe340b2950 0x7efe340b4e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe40005e30 tx=0x7efe40005da0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb8456660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efe3000bc90 tx=0x7efe300107a0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 -- 172.21.15.5:0/3063761466 shutdown_connections 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb8456660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb8456120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 --2- 172.21.15.5:0/3063761466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb8455be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 -- 172.21.15.5:0/3063761466 >> 172.21.15.5:0/3063761466 conn(0x55deb80a9300 msgr2=0x55deb8452340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 -- 172.21.15.5:0/3063761466 shutdown_connections 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.141+0000 7efe54a39740 1 -- 172.21.15.5:0/3063761466 wait complete. 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 -- start start 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 Processor -- start 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 0x55deb84601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb80944d0 0x55deb8460700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb846c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55deb8462d10 con 0x55deb808c2f0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55deb8378370 con 0x55deb80944d0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55deb8462e90 con 0x55deb80b51b0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.142+0000 7efe54a39740 10 client.0 _ll_register_callbacks cb 0x55deb80cca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 0x55deb84601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb80944d0 0x55deb8460700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 0x55deb84601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51068/0 (socket says 172.21.15.5:51068) 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe477fe640 1 -- 172.21.15.5:0/813555664 learned_addr learned my addr 172.21.15.5:0/813555664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe47fff640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb846c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 -- 172.21.15.5:0/813555664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 msgr2=0x55deb846c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb846c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 -- 172.21.15.5:0/813555664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 msgr2=0x55deb84601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 0x55deb84601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 -- 172.21.15.5:0/813555664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb8199620 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe47fff640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb846c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe477fe640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 0x55deb84601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe3ffff640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb80944d0 0x55deb8460700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe3000bc60 tx=0x7efe300058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b4ba4af9948b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.143+0000 7efe26ffd640 1 -- 172.21.15.5:0/813555664 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3001c690 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe26ffd640 1 -- 172.21.15.5:0/813555664 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7efe3001c830 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe3001cad0 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe26ffd640 1 -- 172.21.15.5:0/813555664 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe30004020 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 10 client.25451 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55deb846f040 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe26ffd640 1 -- 172.21.15.5:0/813555664 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe3001cad0 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe26ffd640 1 -- 172.21.15.5:0/813555664 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efe30011980 con 0x55deb80944d0 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 10 client.25451 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 20 client.25451 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb8073d80 0x55deb8472400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 4 client.25451 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb83c91e0 con 0x55deb8073d80 2024-05-14T15:26:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.144+0000 7efe477fe640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb8073d80 0x55deb8472400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.145+0000 7efe26ffd640 10 client.25451 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.145+0000 7efe477fe640 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb8073d80 0x55deb8472400 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff11357d8987183 server_cookie=9569872e30b043bf in_seq=0 out_seq=0 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe26ffd640 1 -- 172.21.15.5:0/813555664 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55deb83c91e0 con 0x55deb8073d80 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe26ffd640 10 client.25451 handle_command_reply: tid=0 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 client.25451 shutdown 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb80944d0 msgr2=0x55deb8460700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb80944d0 0x55deb8460700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe3000bc60 tx=0x7efe300058c0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 shutdown_connections 2024-05-14T15:26:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb8073d80 0x55deb8472400 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80b51b0 0x55deb846c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb80944d0 0x55deb8460700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe26ffd640 0 client.25451 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 --2- 172.21.15.5:0/813555664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb808c2f0 0x55deb84601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.147+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 >> 172.21.15.5:0/813555664 conn(0x55deb80a9300 msgr2=0x55deb8457900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.148+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 shutdown_connections 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.148+0000 7efe54a39740 1 -- 172.21.15.5:0/813555664 wait complete. 2024-05-14T15:26:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.148+0000 7efe54a39740 20 client.25451 trim_cache size 0 max 16384 2024-05-14T15:26:52.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:51 smithi005 ceph-mon[31559]: pgmap v63: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-05-14T15:26:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 Processor -- start 2024-05-14T15:26:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 -- start start 2024-05-14T15:26:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 0x55deb8462100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb845f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80944d0 0x55deb8456080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe481b5e70 con 0x55deb8073d80 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe481b3740 con 0x55deb808c2f0 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.163+0000 7efe54a39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe481b1590 con 0x55deb80944d0 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 0x55deb8462100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 0x55deb8462100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51074/0 (socket says 172.21.15.5:51074) 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 -- 172.21.15.5:0/2237333698 learned_addr learned my addr 172.21.15.5:0/2237333698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe3ffff640 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb845f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe47fff640 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80944d0 0x55deb8456080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 -- 172.21.15.5:0/2237333698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80944d0 msgr2=0x55deb8456080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80944d0 0x55deb8456080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 -- 172.21.15.5:0/2237333698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 msgr2=0x55deb845f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb845f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 -- 172.21.15.5:0/2237333698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb84533c0 con 0x55deb8073d80 2024-05-14T15:26:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe477fe640 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 0x55deb8462100 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe380e01c0 tx=0x7efe380eb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d027b78c7e81264 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe3d7fa640 1 -- 172.21.15.5:0/2237333698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380a9a70 con 0x55deb8073d80 2024-05-14T15:26:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe3d7fa640 1 -- 172.21.15.5:0/2237333698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe380b2440 con 0x55deb8073d80 2024-05-14T15:26:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.164+0000 7efe3d7fa640 1 -- 172.21.15.5:0/2237333698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380a5430 con 0x55deb8073d80 2024-05-14T15:26:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 -- 172.21.15.5:0/2237333698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 msgr2=0x55deb8462100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 0x55deb8462100 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe380e01c0 tx=0x7efe380eb040 comp rx=0 tx=0).stop 2024-05-14T15:26:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 -- 172.21.15.5:0/2237333698 shutdown_connections 2024-05-14T15:26:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb80944d0 0x55deb8456080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb845f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 --2- 172.21.15.5:0/2237333698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb8073d80 0x55deb8462100 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 -- 172.21.15.5:0/2237333698 >> 172.21.15.5:0/2237333698 conn(0x55deb80a9300 msgr2=0x55deb8455970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 -- 172.21.15.5:0/2237333698 shutdown_connections 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.165+0000 7efe54a39740 1 -- 172.21.15.5:0/2237333698 wait complete. 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 -- start start 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 Processor -- start 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb8073d80 0x55deb8468080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb84685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb84634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe481b1c00 con 0x55deb80944d0 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe481b3070 con 0x55deb808c2f0 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe481b22d0 con 0x55deb8073d80 2024-05-14T15:26:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe54a39740 10 client.0 _ll_register_callbacks cb 0x7efe380e67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb84634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb84685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb84634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51080/0 (socket says 172.21.15.5:51080) 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe47fff640 1 -- 172.21.15.5:0/336340308 learned_addr learned my addr 172.21.15.5:0/336340308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.166+0000 7efe477fe640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb8073d80 0x55deb8468080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe3ffff640 1 -- 172.21.15.5:0/336340308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb8073d80 msgr2=0x55deb8468080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe3ffff640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb8073d80 0x55deb8468080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe3ffff640 1 -- 172.21.15.5:0/336340308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 msgr2=0x55deb84634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe3ffff640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb84634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe3ffff640 1 -- 172.21.15.5:0/336340308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb8463be0 con 0x55deb808c2f0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe477fe640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb8073d80 0x55deb8468080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe3ffff640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb84685c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe30011da0 tx=0x7efe300043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f020862f8e14e0ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:26:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe26ffd640 1 -- 172.21.15.5:0/336340308 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3003a060 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe26ffd640 1 -- 172.21.15.5:0/336340308 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7efe3001c8a0 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55deb83c3050 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe26ffd640 1 -- 172.21.15.5:0/336340308 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe3001cac0 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe54a39740 10 client.25457 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.167+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55deb84d9f30 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe26ffd640 1 -- 172.21.15.5:0/336340308 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55deb83c3050 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe26ffd640 1 -- 172.21.15.5:0/336340308 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efe300039d0 con 0x55deb808c2f0 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe54a39740 10 client.25457 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe54a39740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe54a39740 20 client.25457 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe54a39740 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb80b51b0 0x55deb84dc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe54a39740 4 client.25457 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:26:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55deb833e5f0 con 0x55deb80b51b0 2024-05-14T15:26:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe477fe640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb80b51b0 0x55deb84dc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.168+0000 7efe26ffd640 10 client.25457 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.169+0000 7efe477fe640 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb80b51b0 0x55deb84dc630 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95c989a16de498d server_cookie=22ecbd69bd5c641e in_seq=0 out_seq=0 2024-05-14T15:26:52.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:51 smithi144 ceph-mon[35650]: pgmap v63: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-05-14T15:26:52.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:51 smithi193 ceph-mon[36001]: pgmap v63: 129 pgs: 129 active+clean; 11 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-05-14T15:26:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe26ffd640 1 -- 172.21.15.5:0/336340308 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55deb833e5f0 con 0x55deb80b51b0 2024-05-14T15:26:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe26ffd640 10 client.25457 handle_command_reply: tid=0 2024-05-14T15:26:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 client.25457 shutdown 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 msgr2=0x55deb84685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb84685c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe30011da0 tx=0x7efe300043f0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 shutdown_connections 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55deb80b51b0 0x55deb84dc630 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb80944d0 0x55deb84634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55deb808c2f0 0x55deb84685c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 --2- 172.21.15.5:0/336340308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55deb8073d80 0x55deb8468080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe26ffd640 0 client.25457 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 >> 172.21.15.5:0/336340308 conn(0x55deb80a9300 msgr2=0x55deb84563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 shutdown_connections 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 1 -- 172.21.15.5:0/336340308 wait complete. 2024-05-14T15:26:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.422+0000 7efe54a39740 20 client.25457 trim_cache size 0 max 16384 2024-05-14T15:26:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:26:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:26:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:26:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe340b2950 msgr2=0x7efe340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe340b2950 0x7efe340b4e60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe40005e30 tx=0x7efe40005da0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 msgr2=0x7efe48102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 0x7efe48102e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe3800e9f0 tx=0x7efe3800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 shutdown_connections 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe340b2950 0x7efe340b4e60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4810e6d0 0x7efe481038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe48109d20 0x7efe481033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 --2- 172.21.15.5:0/1599921292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe481021d0 0x7efe48102e70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 >> 172.21.15.5:0/1599921292 conn(0x7efe480779e0 msgr2=0x7efe4810a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 shutdown_connections 2024-05-14T15:26:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:26:52.423+0000 7efe4db84640 1 -- 172.21.15.5:0/1599921292 wait complete. 2024-05-14T15:26:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 -- 172.21.15.5:0/1651872619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd141034c0 msgr2=0x7fdd141038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/1651872619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd141034c0 0x7fdd141038d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdd100099c0 tx=0x7fdd10009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 -- 172.21.15.5:0/1651872619 shutdown_connections 2024-05-14T15:26:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/1651872619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd14108740 0x7fdd141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/1651872619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd141034c0 0x7fdd141038d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/1651872619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd14107df0 0x7fdd14108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 -- 172.21.15.5:0/1651872619 >> 172.21.15.5:0/1651872619 conn(0x7fdd140fedc0 msgr2=0x7fdd141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 -- 172.21.15.5:0/1651872619 shutdown_connections 2024-05-14T15:26:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.885+0000 7fdd1b55a700 1 -- 172.21.15.5:0/1651872619 wait complete. 2024-05-14T15:26:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 Processor -- start 2024-05-14T15:26:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 -- start start 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141034c0 0x7fdd1419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd14107df0 0x7fdd1419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 0x7fdd1419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14113370 con 0x7fdd14108740 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141131f0 con 0x7fdd141034c0 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd1b55a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141134f0 con 0x7fdd14107df0 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 0x7fdd1419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 0x7fdd1419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51100/0 (socket says 172.21.15.5:51100) 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 -- 172.21.15.5:0/462591847 learned_addr learned my addr 172.21.15.5:0/462591847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd192f6700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141034c0 0x7fdd1419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd18af5700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd14107df0 0x7fdd1419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 -- 172.21.15.5:0/462591847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd14107df0 msgr2=0x7fdd1419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd14107df0 0x7fdd1419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 -- 172.21.15.5:0/462591847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141034c0 msgr2=0x7fdd1419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141034c0 0x7fdd1419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.886+0000 7fdd19af7700 1 -- 172.21.15.5:0/462591847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1419fca0 con 0x7fdd14108740 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.887+0000 7fdd19af7700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 0x7fdd1419f600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00ef30 tx=0x7fdd0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.887+0000 7fdd0a7fc700 1 -- 172.21.15.5:0/462591847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0092e0 con 0x7fdd14108740 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.887+0000 7fdd0a7fc700 1 -- 172.21.15.5:0/462591847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd0c004bb0 con 0x7fdd14108740 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.887+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1419ff30 con 0x7fdd14108740 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.887+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a0420 con 0x7fdd14108740 2024-05-14T15:26:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.887+0000 7fdd0a7fc700 1 -- 172.21.15.5:0/462591847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c004280 con 0x7fdd14108740 2024-05-14T15:26:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.890+0000 7fdd0a7fc700 1 -- 172.21.15.5:0/462591847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fdd0c01b030 con 0x7fdd14108740 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.891+0000 7fdd29f6eb80 1 Processor -- start 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.891+0000 7fdd0a7fc700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdd000b29f0 0x7fdd000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.891+0000 7fdd29f6eb80 1 -- start start 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.891+0000 7fdd18af5700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdd000b29f0 0x7fdd000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd0a7fc700 1 -- 172.21.15.5:0/462591847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fdd0c0e24f0 con 0x7fdd14108740 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a44fff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a45003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a45004470 0x563a45004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563a44f21860 con 0x563a44fffc60 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a44ec07c0 con 0x563a45004470 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a44f1d820 con 0x563a44e81e40 2024-05-14T15:26:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a44fff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a44fff720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:40728/0 (socket says 172.21.15.5:40728) 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 -- 172.21.15.5:0/3216955393 learned_addr learned my addr 172.21.15.5:0/3216955393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd19af7700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a45004470 0x563a45004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd18af5700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a45003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 -- 172.21.15.5:0/3216955393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a45004470 msgr2=0x563a45004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a45004470 0x563a45004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 -- 172.21.15.5:0/3216955393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 msgr2=0x563a45003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a45003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 -- 172.21.15.5:0/3216955393 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a44e6e8d0 con 0x563a44e81e40 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd19af7700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a45004470 0x563a45004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd18af5700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a45003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd18af5700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdd000b29f0 0x7fdd000b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd141079b0 tx=0x7fdd040113e0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.892+0000 7fdd192f6700 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a44fff720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd10015000 tx=0x7fdd10005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337f0e81e7276fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.893+0000 7fdcff7fe700 1 -- 172.21.15.5:0/3216955393 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10027070 con 0x563a44e81e40 2024-05-14T15:26:52.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.893+0000 7fdcff7fe700 1 -- 172.21.15.5:0/3216955393 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd100037c0 con 0x563a44e81e40 2024-05-14T15:26:52.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.893+0000 7fdcff7fe700 1 -- 172.21.15.5:0/3216955393 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000b9f0 con 0x563a44e81e40 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/3216955393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 msgr2=0x563a44fff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a44fff720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd10015000 tx=0x7fdd10005040 comp rx=0 tx=0).stop 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/3216955393 shutdown_connections 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a45004470 0x563a45004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a44fff720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/3216955393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a45003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/3216955393 >> 172.21.15.5:0/3216955393 conn(0x563a44ec9600 msgr2=0x563a44ffa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/3216955393 shutdown_connections 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/3216955393 wait complete. 2024-05-14T15:26:52.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 -- start start 2024-05-14T15:26:52.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.894+0000 7fdd29f6eb80 1 Processor -- start 2024-05-14T15:26:52.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a45009150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a45009690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a45004470 0x563a45009bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563a4500c3c0 con 0x563a45004470 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a44e87c30 con 0x563a44fffc60 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a4500c540 con 0x563a44e81e40 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd29f6eb80 10 client.0 _ll_register_callbacks cb 0x563a44eb9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a45009150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd19af7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a45004470 0x563a45009bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a45009150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:40732/0 (socket says 172.21.15.5:40732) 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd18af5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a45009690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 -- 172.21.15.5:0/4135146163 learned_addr learned my addr 172.21.15.5:0/4135146163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd19af7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a45004470 0x563a45009bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51116/0 (socket says 172.21.15.5:51116) 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 -- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 msgr2=0x563a45009690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a45009690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 -- 172.21.15.5:0/4135146163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a45004470 msgr2=0x563a45009bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a45004470 0x563a45009bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 -- 172.21.15.5:0/4135146163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a44ebd690 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdd192f6700 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a45009150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005f50 tx=0x7fdd10005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f77177049a6d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.895+0000 7fdcfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdcfcff9700 1 -- 172.21.15.5:0/4135146163 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10027050 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdcfcff9700 1 -- 172.21.15.5:0/4135146163 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fdd1000b990 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd100037c0 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdcfcff9700 1 -- 172.21.15.5:0/4135146163 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd1003b050 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdd29f6eb80 10 client.25495 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a4500c3c0 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.896+0000 7fdcfcff9700 1 -- 172.21.15.5:0/4135146163 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd100037c0 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdcfcff9700 1 -- 172.21.15.5:0/4135146163 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdd10036030 con 0x563a44e81e40 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd29f6eb80 10 client.25495 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd29f6eb80 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd29f6eb80 20 client.25495 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a45019590 0x563a4501ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd29f6eb80 4 client.25495 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a44f2f240 con 0x563a45019590 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdd18af5700 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a45019590 0x563a4501ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.897+0000 7fdcfcff9700 10 client.25495 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.898+0000 7fdd18af5700 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a45019590 0x563a4501ba20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8796370d0fc0a05 server_cookie=1b7548f5f43d56ed in_seq=0 out_seq=0 2024-05-14T15:26:52.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdcfcff9700 1 -- 172.21.15.5:0/4135146163 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a44f2f240 con 0x563a45019590 2024-05-14T15:26:52.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdcfcff9700 10 client.25495 handle_command_reply: tid=0 2024-05-14T15:26:52.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 client.25495 shutdown 2024-05-14T15:26:52.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 msgr2=0x563a45009150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a45009150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005f50 tx=0x7fdd10005360 comp rx=0 tx=0).stop 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 shutdown_connections 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a45019590 0x563a4501ba20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a45004470 0x563a45009bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a44e81e40 0x563a45009150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdcfcff9700 0 client.25495 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/4135146163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a45009690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.899+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 >> 172.21.15.5:0/4135146163 conn(0x563a44ec9600 msgr2=0x563a44ffe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.900+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 shutdown_connections 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.900+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/4135146163 wait complete. 2024-05-14T15:26:52.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.900+0000 7fdd29f6eb80 20 client.25495 trim_cache size 0 max 16384 2024-05-14T15:26:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 Processor -- start 2024-05-14T15:26:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 -- start start 2024-05-14T15:26:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44e81e40 0x563a45015810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a44ffa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a44ffeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141b0c70 con 0x563a44fffc60 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ae500 con 0x563a44e81e40 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ac350 con 0x563a45004470 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd18af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a44ffa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44e81e40 0x563a45015810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd18af5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a44ffa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51124/0 (socket says 172.21.15.5:51124) 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd18af5700 1 -- 172.21.15.5:0/1658561343 learned_addr learned my addr 172.21.15.5:0/1658561343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.923+0000 7fdd19af7700 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a44ffeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdd18af5700 1 -- 172.21.15.5:0/1658561343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 msgr2=0x563a44ffeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdd18af5700 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a44ffeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdd18af5700 1 -- 172.21.15.5:0/1658561343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44e81e40 msgr2=0x563a45015810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdd18af5700 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44e81e40 0x563a45015810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdd18af5700 1 -- 172.21.15.5:0/1658561343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a450076c0 con 0x563a44fffc60 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdd18af5700 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a44ffa610 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ab0 tx=0x7fdd04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5257b650afe27ca server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdcfeffd700 1 -- 172.21.15.5:0/1658561343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04012de0 con 0x563a44fffc60 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdcfeffd700 1 -- 172.21.15.5:0/1658561343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd040044d0 con 0x563a44fffc60 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.924+0000 7fdcfeffd700 1 -- 172.21.15.5:0/1658561343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003970 con 0x563a44fffc60 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/1658561343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 msgr2=0x563a44ffa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a44ffa610 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ab0 tx=0x7fdd04016000 comp rx=0 tx=0).stop 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/1658561343 shutdown_connections 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a44ffeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44e81e40 0x563a45015810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/1658561343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44fffc60 0x563a44ffa610 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/1658561343 >> 172.21.15.5:0/1658561343 conn(0x563a44ec9600 msgr2=0x563a44ffe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/1658561343 shutdown_connections 2024-05-14T15:26:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.925+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/1658561343 wait complete. 2024-05-14T15:26:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.926+0000 7fdd29f6eb80 1 -- start start 2024-05-14T15:26:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.926+0000 7fdd29f6eb80 1 Processor -- start 2024-05-14T15:26:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.926+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 0x563a45021470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.926+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a4501e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd29f6eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a4501ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ac9c0 con 0x563a44e81e40 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ade30 con 0x563a44fffc60 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd29f6eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ad090 con 0x563a45004470 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd29f6eb80 10 client.0 _ll_register_callbacks cb 0x563a44eb9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 0x563a45021470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 0x563a45021470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51140/0 (socket says 172.21.15.5:51140) 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 -- 172.21.15.5:0/958918831 learned_addr learned my addr 172.21.15.5:0/958918831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd18af5700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a4501e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd19af7700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a4501ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 -- 172.21.15.5:0/958918831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 msgr2=0x563a4501ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a4501ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 -- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 msgr2=0x563a4501e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a4501e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 -- 172.21.15.5:0/958918831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4501f370 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdd192f6700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 0x563a45021470 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdd100098e0 tx=0x7fdd10005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea16b40348ccc48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdcfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.927+0000 7fdcfcff9700 1 -- 172.21.15.5:0/958918831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10027040 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdcfcff9700 1 -- 172.21.15.5:0/958918831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7fdd1000b790 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd040044d0 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdcfcff9700 1 -- 172.21.15.5:0/958918831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd10048050 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 10 client.15831 fetch_fsmap learned FSMap version 26 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a45083a00 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdcfcff9700 1 -- 172.21.15.5:0/958918831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd040044d0 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdcfcff9700 1 -- 172.21.15.5:0/958918831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdd10036030 con 0x563a44e81e40 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 10 client.15831 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 20 client.15831 populate_metadata read hostname 'smithi005' 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a450840b0 0x563a450864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 4 client.15831 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:26:52.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.928+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a44f292e0 con 0x563a450840b0 2024-05-14T15:26:52.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.929+0000 7fdd18af5700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a450840b0 0x563a450864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:52.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.929+0000 7fdcfcff9700 10 client.15831 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.929+0000 7fdd18af5700 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a450840b0 0x563a450864d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60378946bb4e6cd1 server_cookie=4065546bf77eb239 in_seq=0 out_seq=0 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdcfcff9700 1 -- 172.21.15.5:0/958918831 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a44f292e0 con 0x563a450840b0 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdcfcff9700 10 client.15831 handle_command_reply: tid=0 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 client.15831 shutdown 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 msgr2=0x563a45021470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 0x563a45021470 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdd100098e0 tx=0x7fdd10005ee0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 shutdown_connections 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563a450840b0 0x563a450864d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a45004470 0x563a4501ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a44e81e40 0x563a45021470 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 --2- 172.21.15.5:0/958918831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a44fffc60 0x563a4501e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 >> 172.21.15.5:0/958918831 conn(0x563a44ec9600 msgr2=0x563a44ffa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.930+0000 7fdcfcff9700 0 client.15831 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 shutdown_connections 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd29f6eb80 1 -- 172.21.15.5:0/958918831 wait complete. 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd29f6eb80 20 client.15831 trim_cache size 0 max 16384 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:26:52.935 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:26:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdd000b29f0 msgr2=0x7fdd000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdd000b29f0 0x7fdd000b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd141079b0 tx=0x7fdd040113e0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 msgr2=0x7fdd1419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 0x7fdd1419f600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00ef30 tx=0x7fdd0c00c3a0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 shutdown_connections 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdd000b29f0 0x7fdd000b4f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd14108740 0x7fdd1419f600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd141034c0 0x7fdd1419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 --2- 172.21.15.5:0/462591847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd14107df0 0x7fdd1419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 >> 172.21.15.5:0/462591847 conn(0x7fdd140fedc0 msgr2=0x7fdd141063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 shutdown_connections 2024-05-14T15:26:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:52.931+0000 7fdd1b55a700 1 -- 172.21.15.5:0/462591847 wait complete. 2024-05-14T15:26:53.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 36634 15.30 MB/sec warmup 14 sec latency 163.097 ms 2024-05-14T15:26:53.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:26:53.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:26:54.060 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:54.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:53 smithi005 ceph-mon[31559]: pgmap v64: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-05-14T15:26:54.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 39598 15.42 MB/sec warmup 15 sec latency 8.022 ms 2024-05-14T15:26:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:53 smithi144 ceph-mon[35650]: pgmap v64: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-05-14T15:26:54.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:53 smithi193 ceph-mon[36001]: pgmap v64: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-05-14T15:26:55.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 42589 15.55 MB/sec warmup 16 sec latency 35.210 ms 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 -- 172.21.15.5:0/1852047311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb81034c0 msgr2=0x7ffbb81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/1852047311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb81034c0 0x7ffbb81038d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffba80099c0 tx=0x7ffba8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 -- 172.21.15.5:0/1852047311 shutdown_connections 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/1852047311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb8108740 0x7ffbb81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/1852047311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb81034c0 0x7ffbb81038d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/1852047311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb8107df0 0x7ffbb8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.042+0000 7ffbbdd46700 1 -- 172.21.15.5:0/1852047311 >> 172.21.15.5:0/1852047311 conn(0x7ffbb80fedc0 msgr2=0x7ffbb81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.043+0000 7ffbbdd46700 1 -- 172.21.15.5:0/1852047311 shutdown_connections 2024-05-14T15:26:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.043+0000 7ffbbdd46700 1 -- 172.21.15.5:0/1852047311 wait complete. 2024-05-14T15:26:56.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:55 smithi005 ceph-mon[31559]: pgmap v65: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-14T15:26:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.043+0000 7ffbbdd46700 1 Processor -- start 2024-05-14T15:26:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 -- start start 2024-05-14T15:26:56.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 0x7ffbb819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb8107df0 0x7ffbb819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb8108740 0x7ffbb81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113370 con 0x7ffbb8108740 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81131f0 con 0x7ffbb8107df0 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbbdd46700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81134f0 con 0x7ffbb81034c0 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 0x7ffbb819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbb7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb8108740 0x7ffbb81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 0x7ffbb819cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35850/0 (socket says 172.21.15.5:35850) 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbb77fe700 1 -- 172.21.15.5:0/3516266669 learned_addr learned my addr 172.21.15.5:0/3516266669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.044+0000 7ffbb6ffd700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb8107df0 0x7ffbb819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb77fe700 1 -- 172.21.15.5:0/3516266669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb8107df0 msgr2=0x7ffbb819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb77fe700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb8107df0 0x7ffbb819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb77fe700 1 -- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb8108740 msgr2=0x7ffbb81a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb77fe700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb8108740 0x7ffbb81a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb77fe700 1 -- 172.21.15.5:0/3516266669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb81a1ec0 con 0x7ffbb81034c0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb77fe700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 0x7ffbb819cff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006010 tx=0x7ffba80055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8026070 con 0x7ffbb81034c0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffba800f460 con 0x7ffbb81034c0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8009e60 con 0x7ffbb81034c0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb81a2150 con 0x7ffbb81034c0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.045+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb81a2630 con 0x7ffbb81034c0 2024-05-14T15:26:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.047+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb80512c0 con 0x7ffbb81034c0 2024-05-14T15:26:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.048+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ffba8029070 con 0x7ffbb81034c0 2024-05-14T15:26:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.049+0000 7ffbb4ff9700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ffba00b29f0 0x7ffba00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:26:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.049+0000 7ffbb6ffd700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ffba00b29f0 0x7ffba00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:26:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.049+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7ffba80e34b0 con 0x7ffbb81034c0 2024-05-14T15:26:56.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.050+0000 7ffbb6ffd700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ffba00b29f0 0x7ffba00b4f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffbb806c5b0 tx=0x7ffbac009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:26:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.053+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffba8013070 con 0x7ffbb81034c0 2024-05-14T15:26:56.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 45027 15.15 MB/sec warmup 17 sec latency 21.924 ms 2024-05-14T15:26:56.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:55 smithi144 ceph-mon[35650]: pgmap v65: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-14T15:26:56.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:55 smithi193 ceph-mon[36001]: pgmap v65: 129 pgs: 129 active+clean; 17 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-05-14T15:26:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.342+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb802e5f0 con 0x7ffbb81034c0 2024-05-14T15:26:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.343+0000 7ffbb4ff9700 1 -- 172.21.15.5:0/3516266669 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11864 (secure 0 0 0) 0x7ffba80aab00 con 0x7ffbb81034c0 2024-05-14T15:26:56.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:26:56.344 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:24:47.806060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3352720235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3352720235},{"type":"v1","addr":"172.21.15.144:6835","nonce":3352720235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ffba00b29f0 msgr2=0x7ffba00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ffba00b29f0 0x7ffba00b4f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffbb806c5b0 tx=0x7ffbac009260 comp rx=0 tx=0).stop 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 msgr2=0x7ffbb819cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 0x7ffbb819cff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006010 tx=0x7ffba80055f0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 shutdown_connections 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ffba00b29f0 0x7ffba00b4f00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb8108740 0x7ffbb81a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb81034c0 0x7ffbb819cff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 --2- 172.21.15.5:0/3516266669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb8107df0 0x7ffbb819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:26:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.345+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 >> 172.21.15.5:0/3516266669 conn(0x7ffbb80fedc0 msgr2=0x7ffbb8106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:26:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.346+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 shutdown_connections 2024-05-14T15:26:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:26:56.346+0000 7ffbbdd46700 1 -- 172.21.15.5:0/3516266669 wait complete. 2024-05-14T15:26:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-14T15:26:57.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 47330 15.09 MB/sec warmup 18 sec latency 17.095 ms 2024-05-14T15:26:57.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:56 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3516266669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:26:57.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:56 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3516266669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:26:57.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:56 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3516266669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:26:57.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:26:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:26:58.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 50255 15.19 MB/sec warmup 19 sec latency 9.982 ms 2024-05-14T15:26:58.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:57 smithi005 ceph-mon[31559]: pgmap v66: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.9 MiB/s wr, 411 op/s 2024-05-14T15:26:58.198 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:57 smithi144 ceph-mon[35650]: pgmap v66: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.9 MiB/s wr, 411 op/s 2024-05-14T15:26:58.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:57 smithi193 ceph-mon[36001]: pgmap v66: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 1.9 MiB/s wr, 411 op/s 2024-05-14T15:26:59.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 53242 15.31 MB/sec warmup 20 sec latency 33.900 ms 2024-05-14T15:27:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 -- 172.21.15.5:0/549151787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac107df0 msgr2=0x7f6cac108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/549151787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac107df0 0x7f6cac108200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00099c0 tx=0x7f6ca0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:26:59 smithi005 ceph-mon[31559]: pgmap v67: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 329 op/s 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 -- 172.21.15.5:0/549151787 shutdown_connections 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/549151787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac108740 0x7f6cac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/549151787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 0x7f6cac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/549151787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac107df0 0x7f6cac108200 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 -- 172.21.15.5:0/549151787 >> 172.21.15.5:0/549151787 conn(0x7f6cac0fedc0 msgr2=0x7f6cac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 -- 172.21.15.5:0/549151787 shutdown_connections 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.025+0000 7f6cb1db5700 1 -- 172.21.15.5:0/549151787 wait complete. 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.026+0000 7f6cb1db5700 1 Processor -- start 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.026+0000 7f6cb1db5700 1 -- start start 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cb1db5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 0x7f6cac0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cb1db5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac107df0 0x7f6cac0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cb1db5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 0x7f6cac0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cb1db5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac113350 con 0x7f6cac108740 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cb1db5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1131d0 con 0x7f6cac107df0 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cb1db5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1134d0 con 0x7f6cac1034c0 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 0x7f6cac0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 0x7f6cac0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 0x7f6cac0ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50222/0 (socket says 172.21.15.5:50222) 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6cabfff700 1 -- 172.21.15.5:0/273254574 learned_addr learned my addr 172.21.15.5:0/273254574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.027+0000 7f6caaffd700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac107df0 0x7f6cac0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cabfff700 1 -- 172.21.15.5:0/273254574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 msgr2=0x7f6cac0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cabfff700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 0x7f6cac0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cabfff700 1 -- 172.21.15.5:0/273254574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac107df0 msgr2=0x7f6cac0ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cabfff700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac107df0 0x7f6cac0ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cabfff700 1 -- 172.21.15.5:0/273254574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac1a9170 con 0x7f6cac108740 2024-05-14T15:27:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 0x7f6cac0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cabfff700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 0x7f6cac0ffd20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400ef30 tx=0x7f6ca400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6ca8ff9700 1 -- 172.21.15.5:0/273254574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40092e0 con 0x7f6cac108740 2024-05-14T15:27:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac1a9400 con 0x7f6cac108740 2024-05-14T15:27:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6ca8ff9700 1 -- 172.21.15.5:0/273254574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca4004bb0 con 0x7f6cac108740 2024-05-14T15:27:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac1a9910 con 0x7f6cac108740 2024-05-14T15:27:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.028+0000 7f6ca8ff9700 1 -- 172.21.15.5:0/273254574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400cea0 con 0x7f6cac108740 2024-05-14T15:27:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.032+0000 7f6ca8ff9700 1 -- 172.21.15.5:0/273254574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6ca401b030 con 0x7f6cac108740 2024-05-14T15:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.034+0000 7f6ca8ff9700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c940b29f0 0x7f6c940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.034+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c940b29f0 0x7f6c940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.034+0000 7f6ca8ff9700 1 -- 172.21.15.5:0/273254574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6ca40e2400 con 0x7f6cac108740 2024-05-14T15:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.034+0000 7f6cc07c9b80 1 Processor -- start 2024-05-14T15:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.034+0000 7f6cc07c9b80 1 -- start start 2024-05-14T15:27:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.034+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c940b29f0 0x7f6c940b4f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00afd0 tx=0x7f6c9c00a380 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 0x55afb778d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7791f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb7792470 0x55afb7792880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55afb76af860 con 0x55afb760fe40 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afb764e7c0 con 0x55afb7792470 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55afb76ab820 con 0x55afb778dc60 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 0x55afb778d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 0x55afb778d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50224/0 (socket says 172.21.15.5:50224) 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cabfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb7792470 0x55afb7792880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3408718305 learned_addr learned my addr 172.21.15.5:0/3408718305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6caaffd700 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7791f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3408718305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 msgr2=0x55afb7791f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7791f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3408718305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb7792470 msgr2=0x55afb7792880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb7792470 0x55afb7792880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3408718305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb75fc8d0 con 0x55afb760fe40 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.035+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 0x55afb778d720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c010780 tx=0x7f6c9c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6bcdc7f57941db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.036+0000 7f6c91ffb700 1 -- 172.21.15.5:0/3408718305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c016070 con 0x55afb760fe40 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.036+0000 7f6c91ffb700 1 -- 172.21.15.5:0/3408718305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c00fe50 con 0x55afb760fe40 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.036+0000 7f6c91ffb700 1 -- 172.21.15.5:0/3408718305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c01bad0 con 0x55afb760fe40 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.036+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3408718305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 msgr2=0x55afb778d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.036+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 0x55afb778d720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c010780 tx=0x7f6c9c012000 comp rx=0 tx=0).stop 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3408718305 shutdown_connections 2024-05-14T15:27:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb7792470 0x55afb7792880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb760fe40 0x55afb778d720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3408718305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7791f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3408718305 >> 172.21.15.5:0/3408718305 conn(0x55afb7657600 msgr2=0x55afb77884b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3408718305 shutdown_connections 2024-05-14T15:27:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3408718305 wait complete. 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 -- start start 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.037+0000 7f6cc07c9b80 1 Processor -- start 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7797780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7792470 0x55afb7797cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55afb779a420 con 0x55afb7792470 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afb7615c30 con 0x55afb760fe40 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55afb779a5a0 con 0x55afb778dc60 2024-05-14T15:27:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cc07c9b80 10 client.0 _ll_register_callbacks cb 0x55afb7647920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:40882/0 (socket says 172.21.15.5:40882) 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3983188265 learned_addr learned my addr 172.21.15.5:0/3983188265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cabfff700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7792470 0x55afb7797cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6caaffd700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7797780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 msgr2=0x55afb7797780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7797780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3983188265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7792470 msgr2=0x55afb7797cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7792470 0x55afb7797cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 -- 172.21.15.5:0/3983188265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb764b690 con 0x55afb760fe40 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00add0 tx=0x7f6c9c005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed22d4ed5f5b61 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.038+0000 7f6c8affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6c8affd700 1 -- 172.21.15.5:0/3983188265 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c01b420 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c00fe50 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6c8affd700 1 -- 172.21.15.5:0/3983188265 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6c9c004020 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6c8affd700 1 -- 172.21.15.5:0/3983188265 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c9c03a050 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 10 client.25493 fetch_fsmap learned FSMap version 26 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afb77a5100 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6c8affd700 1 -- 172.21.15.5:0/3983188265 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c00fe50 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6c8affd700 1 -- 172.21.15.5:0/3983188265 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6c9c003680 con 0x55afb760fe40 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 10 client.25493 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 10 client.25493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 20 client.25493 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb77a75d0 0x55afb77a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 4 client.25493 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.039+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afb76bd240 con 0x55afb77a75d0 2024-05-14T15:27:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.040+0000 7f6caaffd700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb77a75d0 0x55afb77a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.040+0000 7f6c8affd700 10 client.25493 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.040+0000 7f6caaffd700 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb77a75d0 0x55afb77a99f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4f9a3b92ef45903 server_cookie=2f22fd078798ae51 in_seq=0 out_seq=0 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6c8affd700 1 -- 172.21.15.5:0/3983188265 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55afb76bd240 con 0x55afb77a75d0 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6c8affd700 10 client.25493 handle_command_reply: tid=0 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 client.25493 shutdown 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 msgr2=0x55afb7797240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00add0 tx=0x7f6c9c005e50 comp rx=0 tx=0).stop 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 shutdown_connections 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7792470 0x55afb7797cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb77a75d0 0x55afb77a99f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/3983188265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7797780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6c8affd700 0 client.25493 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 >> 172.21.15.5:0/3983188265 conn(0x55afb7657600 msgr2=0x55afb778c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 shutdown_connections 2024-05-14T15:27:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/3983188265 wait complete. 2024-05-14T15:27:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.041+0000 7f6cc07c9b80 20 client.25493 trim_cache size 0 max 16384 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 Processor -- start 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 -- start start 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7793890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c98011690 con 0x55afb7657600 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9800ef20 con 0x55afb760fe40 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9800cd70 con 0x55afb778dc60 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50242/0 (socket says 172.21.15.5:50242) 2024-05-14T15:27:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 -- 172.21.15.5:0/1777235463 learned_addr learned my addr 172.21.15.5:0/1777235463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6caaffd700 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cabfff700 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7793890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 -- 172.21.15.5:0/1777235463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 msgr2=0x55afb7793890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7793890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 -- 172.21.15.5:0/1777235463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 msgr2=0x55afb7797b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6cab7fe700 1 -- 172.21.15.5:0/1777235463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb7789ea0 con 0x55afb7657600 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.058+0000 7f6caaffd700 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.059+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77961a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c000c00 tx=0x7f6c9c010700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e25fc32a7424e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.059+0000 7f6c917fa700 1 -- 172.21.15.5:0/1777235463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c016070 con 0x55afb7657600 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.059+0000 7f6c917fa700 1 -- 172.21.15.5:0/1777235463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c01b420 con 0x55afb7657600 2024-05-14T15:27:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.059+0000 7f6c917fa700 1 -- 172.21.15.5:0/1777235463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003d60 con 0x55afb7657600 2024-05-14T15:27:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/1777235463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 msgr2=0x55afb77961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77961a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c000c00 tx=0x7f6c9c010700 comp rx=0 tx=0).stop 2024-05-14T15:27:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/1777235463 shutdown_connections 2024-05-14T15:27:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb7793890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77961a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/1777235463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb7797b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/1777235463 >> 172.21.15.5:0/1777235463 conn(0x55afb778e3e0 msgr2=0x55afb7787be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/1777235463 shutdown_connections 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/1777235463 wait complete. 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.060+0000 7f6cc07c9b80 1 -- start start 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 Processor -- start 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb779fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb77ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9800d3e0 con 0x55afb7657600 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9800e850 con 0x55afb760fe40 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9800dab0 con 0x55afb778dc60 2024-05-14T15:27:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cc07c9b80 10 client.0 _ll_register_callbacks cb 0x55afb7647920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50252/0 (socket says 172.21.15.5:50252) 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 -- 172.21.15.5:0/124724870 learned_addr learned my addr 172.21.15.5:0/124724870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb779fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6cabfff700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb77ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 -- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 msgr2=0x55afb77ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb77ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 -- 172.21.15.5:0/124724870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 msgr2=0x55afb779fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb779fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.061+0000 7f6caaffd700 1 -- 172.21.15.5:0/124724870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb77a0230 con 0x55afb7657600 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6caaffd700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77ae050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009350 tx=0x7f6ca00054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298db16504bca41e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6c8affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6c8affd700 1 -- 172.21.15.5:0/124724870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca002c070 con 0x55afb7657600 2024-05-14T15:27:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9c01b420 con 0x55afb7657600 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6c8affd700 1 -- 172.21.15.5:0/124724870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f6ca0003bd0 con 0x55afb7657600 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6c8affd700 1 -- 172.21.15.5:0/124724870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ca0038050 con 0x55afb7657600 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6cc07c9b80 10 client.15855 fetch_fsmap learned FSMap version 26 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afb78115e0 con 0x55afb7657600 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.062+0000 7f6c8affd700 1 -- 172.21.15.5:0/124724870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c01b420 con 0x55afb7657600 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6c8affd700 1 -- 172.21.15.5:0/124724870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6ca0028270 con 0x55afb7657600 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cc07c9b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cc07c9b80 10 client.15855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cc07c9b80 20 client.15855 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb7792470 0x55afb7813ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cc07c9b80 4 client.15855 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afb76b72e0 con 0x55afb7792470 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb7792470 0x55afb7813ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.063+0000 7f6c8affd700 10 client.15855 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cab7fe700 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb7792470 0x55afb7813ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8686657828c81238 server_cookie=ab5b93a26507916f in_seq=0 out_seq=0 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6c8affd700 1 -- 172.21.15.5:0/124724870 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afb76b72e0 con 0x55afb7792470 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6c8affd700 10 client.15855 handle_command_reply: tid=0 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 client.15855 shutdown 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 msgr2=0x55afb77ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77ae050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009350 tx=0x7f6ca00054b0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 shutdown_connections 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55afb778dc60 0x55afb77ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55afb7792470 0x55afb7813ce0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afb760fe40 0x55afb779fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 --2- 172.21.15.5:0/124724870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afb7657600 0x55afb77ae050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6c8affd700 0 client.15855 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 >> 172.21.15.5:0/124724870 conn(0x55afb778e3e0 msgr2=0x55afb778c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 shutdown_connections 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 1 -- 172.21.15.5:0/124724870 wait complete. 2024-05-14T15:27:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.064+0000 7f6cc07c9b80 20 client.15855 trim_cache size 0 max 16384 2024-05-14T15:27:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c940b29f0 msgr2=0x7f6c940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c940b29f0 0x7f6c940b4f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00afd0 tx=0x7f6c9c00a380 comp rx=0 tx=0).stop 2024-05-14T15:27:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 msgr2=0x7f6cac0ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 0x7f6cac0ffd20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400ef30 tx=0x7f6ca400c3a0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 shutdown_connections 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c940b29f0 0x7f6c940b4f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac108740 0x7f6cac0ffd20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1034c0 0x7f6cac0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 --2- 172.21.15.5:0/273254574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac107df0 0x7f6cac0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 >> 172.21.15.5:0/273254574 conn(0x7f6cac0fedc0 msgr2=0x7f6cac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 shutdown_connections 2024-05-14T15:27:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:00.065+0000 7f6cb1db5700 1 -- 172.21.15.5:0/273254574 wait complete. 2024-05-14T15:27:00.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 55638 14.99 MB/sec warmup 21 sec latency 26.822 ms 2024-05-14T15:27:00.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:26:59 smithi144 ceph-mon[35650]: pgmap v67: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 329 op/s 2024-05-14T15:27:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:26:59 smithi193 ceph-mon[36001]: pgmap v67: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 329 op/s 2024-05-14T15:27:00.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:27:01.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57694 14.89 MB/sec warmup 22 sec latency 22.217 ms 2024-05-14T15:27:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:02.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:01 smithi005 ceph-mon[31559]: pgmap v68: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 329 op/s 2024-05-14T15:27:02.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 59221 14.52 MB/sec warmup 23 sec latency 47.648 ms 2024-05-14T15:27:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:01 smithi144 ceph-mon[35650]: pgmap v68: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 329 op/s 2024-05-14T15:27:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:01 smithi193 ceph-mon[36001]: pgmap v68: 129 pgs: 129 active+clean; 15 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.3 MiB/s wr, 329 op/s 2024-05-14T15:27:03.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 61483 14.50 MB/sec warmup 24 sec latency 13.196 ms 2024-05-14T15:27:04.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 64337 14.61 MB/sec warmup 25 sec latency 41.168 ms 2024-05-14T15:27:04.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:03 smithi144 ceph-mon[35650]: pgmap v69: 129 pgs: 129 active+clean; 21 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.0 MiB/s wr, 435 op/s 2024-05-14T15:27:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:03 smithi193 ceph-mon[36001]: pgmap v69: 129 pgs: 129 active+clean; 21 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.0 MiB/s wr, 435 op/s 2024-05-14T15:27:04.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:03 smithi005 ceph-mon[31559]: pgmap v69: 129 pgs: 129 active+clean; 21 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.0 MiB/s wr, 435 op/s 2024-05-14T15:27:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.933+0000 7f5bc478b700 1 -- 172.21.15.5:0/2014558720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb40987c0 msgr2=0x7f5bb4098bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.933+0000 7f5bc478b700 1 --2- 172.21.15.5:0/2014558720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb40987c0 0x7f5bb4098bd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.934+0000 7f5bc478b700 1 -- 172.21.15.5:0/2014558720 shutdown_connections 2024-05-14T15:27:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.934+0000 7f5bc478b700 1 --2- 172.21.15.5:0/2014558720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bb4099110 0x7f5bb40a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.934+0000 7f5bc478b700 1 --2- 172.21.15.5:0/2014558720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb409e7f0 0x7f5bb409ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.934+0000 7f5bc478b700 1 --2- 172.21.15.5:0/2014558720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb40987c0 0x7f5bb4098bd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.934+0000 7f5bc478b700 1 -- 172.21.15.5:0/2014558720 >> 172.21.15.5:0/2014558720 conn(0x7f5bb40942e0 msgr2=0x7f5bb4096730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:04.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.935+0000 7f5bc478b700 1 -- 172.21.15.5:0/2014558720 shutdown_connections 2024-05-14T15:27:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.936+0000 7f5bc478b700 1 -- 172.21.15.5:0/2014558720 wait complete. 2024-05-14T15:27:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.937+0000 7f5bc478b700 1 Processor -- start 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 -- start start 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bb40987c0 0x7f5bb40aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 0x7f5bb40a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb409e7f0 0x7f5bb40a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb400aac0 con 0x7f5bb40987c0 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb400a940 con 0x7f5bb409e7f0 2024-05-14T15:27:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc478b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb400ac40 con 0x7f5bb4099110 2024-05-14T15:27:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.938+0000 7f5bc2d28700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb409e7f0 0x7f5bb40a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.939+0000 7f5bc1d26700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 0x7f5bb40a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.939+0000 7f5bc1d26700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 0x7f5bb40a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35930/0 (socket says 172.21.15.5:35930) 2024-05-14T15:27:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.939+0000 7f5bc1d26700 1 -- 172.21.15.5:0/371046094 learned_addr learned my addr 172.21.15.5:0/371046094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.941+0000 7f5bc2527700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bb40987c0 0x7f5bb40aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.942+0000 7f5bc1d26700 1 -- 172.21.15.5:0/371046094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb409e7f0 msgr2=0x7f5bb40a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.942+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb409e7f0 0x7f5bb40a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.942+0000 7f5bc1d26700 1 -- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bb40987c0 msgr2=0x7f5bb40aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.942+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bb40987c0 0x7f5bb40aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.942+0000 7f5bc1d26700 1 -- 172.21.15.5:0/371046094 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb40a54e0 con 0x7f5bb4099110 2024-05-14T15:27:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.942+0000 7f5bc2d28700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb409e7f0 0x7f5bb40a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:04.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.945+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 0x7f5bb40a4a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006010 tx=0x7f5bb800ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:04.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.946+0000 7f5bb37fe700 1 -- 172.21.15.5:0/371046094 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800bda0 con 0x7f5bb4099110 2024-05-14T15:27:04.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.946+0000 7f5bb37fe700 1 -- 172.21.15.5:0/371046094 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb8018d10 con 0x7f5bb4099110 2024-05-14T15:27:04.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.946+0000 7f5bb37fe700 1 -- 172.21.15.5:0/371046094 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8004580 con 0x7f5bb4099110 2024-05-14T15:27:04.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.947+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb40a5710 con 0x7f5bb4099110 2024-05-14T15:27:04.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.948+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb40aaaf0 con 0x7f5bb4099110 2024-05-14T15:27:04.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.954+0000 7f5bd319fb80 1 Processor -- start 2024-05-14T15:27:04.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.955+0000 7f5bd319fb80 1 -- start start 2024-05-14T15:27:04.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c3848e40 0x5571c39c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c39c7850 0x5571c39d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571c38e8940 con 0x5571c3848e40 2024-05-14T15:27:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5571c384dda0 con 0x5571c39c7440 2024-05-14T15:27:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5571c38e4900 con 0x5571c39c7850 2024-05-14T15:27:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.958+0000 7f5bb37fe700 1 -- 172.21.15.5:0/371046094 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5bb8009d10 con 0x7f5bb4099110 2024-05-14T15:27:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.959+0000 7f5bc2d28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c3848e40 0x5571c39c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.959+0000 7f5bc2d28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c3848e40 0x5571c39c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50276/0 (socket says 172.21.15.5:50276) 2024-05-14T15:27:04.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.959+0000 7f5bc2d28700 1 -- 172.21.15.5:0/382698884 learned_addr learned my addr 172.21.15.5:0/382698884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c39c7850 0x5571c39d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc2527700 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc1d26700 1 -- 172.21.15.5:0/382698884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 msgr2=0x5571c39d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc1d26700 1 -- 172.21.15.5:0/382698884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c3848e40 msgr2=0x5571c39c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c3848e40 0x5571c39c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.960+0000 7f5bc1d26700 1 -- 172.21.15.5:0/382698884 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c38358d0 con 0x5571c39c7850 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.961+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c39c7850 0x5571c39d2fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80e1740 tx=0x7f5bb80ec6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22beff8347f68bce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.962+0000 7f5bb37fe700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ba80b2b30 0x7f5ba80b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.962+0000 7f5bb37fe700 1 -- 172.21.15.5:0/371046094 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5bb80e4410 con 0x7f5bb4099110 2024-05-14T15:27:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.963+0000 7f5bb0ff9700 1 -- 172.21.15.5:0/382698884 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80e6520 con 0x5571c39c7850 2024-05-14T15:27:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.963+0000 7f5bb0ff9700 1 -- 172.21.15.5:0/382698884 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb80ece40 con 0x5571c39c7850 2024-05-14T15:27:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.964+0000 7f5bc2d28700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ba80b2b30 0x7f5ba80b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.965+0000 7f5bb0ff9700 1 -- 172.21.15.5:0/382698884 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80f58b0 con 0x5571c39c7850 2024-05-14T15:27:04.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.967+0000 7f5bc2d28700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ba80b2b30 0x7f5ba80b5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc000eb0 tx=0x7f5bbc06c040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.968+0000 7f5bd319fb80 1 -- 172.21.15.5:0/382698884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c39c7850 msgr2=0x5571c39d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.968+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c39c7850 0x5571c39d2fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80e1740 tx=0x7f5bb80ec6b0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.969+0000 7f5bd319fb80 1 -- 172.21.15.5:0/382698884 shutdown_connections 2024-05-14T15:27:04.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.969+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c39c7850 0x5571c39d2fd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.969+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c3848e40 0x5571c39c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.969+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/382698884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.969+0000 7f5bd319fb80 1 -- 172.21.15.5:0/382698884 >> 172.21.15.5:0/382698884 conn(0x5571c3890630 msgr2=0x5571c39c1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.970+0000 7f5bd319fb80 1 -- 172.21.15.5:0/382698884 shutdown_connections 2024-05-14T15:27:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.970+0000 7f5bd319fb80 1 -- 172.21.15.5:0/382698884 wait complete. 2024-05-14T15:27:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.971+0000 7f5bd319fb80 1 -- start start 2024-05-14T15:27:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.971+0000 7f5bd319fb80 1 Processor -- start 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.972+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.972+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.972+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.972+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571c39d3780 con 0x5571c39c7850 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.972+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5571c38a1ed0 con 0x5571c39c7440 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.972+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5571c39d3900 con 0x5571c3848e40 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bd319fb80 10 client.0 _ll_register_callbacks cb 0x5571c3880950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:40920/0 (socket says 172.21.15.5:40920) 2024-05-14T15:27:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 -- 172.21.15.5:0/3753912551 learned_addr learned my addr 172.21.15.5:0/3753912551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc2d28700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 -- 172.21.15.5:0/3753912551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 msgr2=0x5571c39d0530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d0530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 -- 172.21.15.5:0/3753912551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 msgr2=0x5571c39d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc1d26700 1 -- 172.21.15.5:0/3753912551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c389f760 con 0x5571c39c7440 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.973+0000 7f5bc2d28700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.974+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80a73e0 tx=0x7f5bb8005210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b127a90b00fa4116 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.974+0000 7f5ba5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.974+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/3753912551 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80a7220 con 0x5571c39c7440 2024-05-14T15:27:04.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.974+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/3753912551 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5bb80e4150 con 0x5571c39c7440 2024-05-14T15:27:04.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.974+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571c38f6320 con 0x5571c39c7440 2024-05-14T15:27:04.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.975+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/3753912551 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5bb80aaad0 con 0x5571c39c7440 2024-05-14T15:27:04.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.977+0000 7f5bd319fb80 10 client.25511 fetch_fsmap learned FSMap version 26 2024-05-14T15:27:04.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.977+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571c39d3780 con 0x5571c39c7440 2024-05-14T15:27:04.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.978+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/3753912551 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5571c38f6320 con 0x5571c39c7440 2024-05-14T15:27:04.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.978+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/3753912551 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5bb802c070 con 0x5571c39c7440 2024-05-14T15:27:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.979+0000 7f5bd319fb80 10 client.25511 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:27:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.979+0000 7f5bd319fb80 10 client.25511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:27:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.979+0000 7f5bd319fb80 20 client.25511 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.979+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c39e06b0 0x5571c39e2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.979+0000 7f5bd319fb80 4 client.25511 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:04.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.979+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c38f03c0 con 0x5571c39e06b0 2024-05-14T15:27:04.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.980+0000 7f5bc2527700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c39e06b0 0x5571c39e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:04.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.982+0000 7f5bc2527700 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c39e06b0 0x5571c39e2ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d68803ff9d63251 server_cookie=83b4267541c34f75 in_seq=0 out_seq=0 2024-05-14T15:27:04.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.982+0000 7f5ba5ffb700 10 client.25511 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.984+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/3753912551 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5571c38f03c0 con 0x5571c39e06b0 2024-05-14T15:27:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.984+0000 7f5ba5ffb700 10 client.25511 handle_command_reply: tid=0 2024-05-14T15:27:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.984+0000 7f5bd319fb80 1 client.25511 shutdown 2024-05-14T15:27:04.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.985+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 msgr2=0x5571c39d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:04.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.985+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80a73e0 tx=0x7f5bb8005210 comp rx=0 tx=0).stop 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 shutdown_connections 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c39e06b0 0x5571c39e2ad0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/3753912551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d0a70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 >> 172.21.15.5:0/3753912551 conn(0x5571c3890630 msgr2=0x5571c39c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.987+0000 7f5ba5ffb700 0 client.25511 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.988+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 shutdown_connections 2024-05-14T15:27:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.988+0000 7f5bd319fb80 1 -- 172.21.15.5:0/3753912551 wait complete. 2024-05-14T15:27:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:04.989+0000 7f5bd319fb80 20 client.25511 trim_cache size 0 max 16384 2024-05-14T15:27:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.022+0000 7f5bd319fb80 1 Processor -- start 2024-05-14T15:27:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.022+0000 7f5bd319fb80 1 -- start start 2024-05-14T15:27:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 0x5571c39cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7850 0x5571c39c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4164f00 con 0x5571c39c7440 2024-05-14T15:27:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4162790 con 0x5571c39c7850 2024-05-14T15:27:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41605e0 con 0x5571c3848e40 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bc1d26700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 0x5571c39cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bc1d26700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 0x5571c39cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50304/0 (socket says 172.21.15.5:50304) 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2745934860 learned_addr learned my addr 172.21.15.5:0/2745934860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.023+0000 7f5bc2d28700 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7850 0x5571c39c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.024+0000 7f5bc2527700 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.024+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2745934860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 msgr2=0x5571c39bbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.024+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39bbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.024+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2745934860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7850 msgr2=0x5571c39c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.024+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7850 0x5571c39c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.024+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2745934860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c39c2830 con 0x5571c39c7440 2024-05-14T15:27:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.025+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 0x5571c39cf490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800b830 tx=0x7f5bb8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e17035a995efa3c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.025+0000 7f5ba7fff700 1 -- 172.21.15.5:0/2745934860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80aacb0 con 0x5571c39c7440 2024-05-14T15:27:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.025+0000 7f5ba7fff700 1 -- 172.21.15.5:0/2745934860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb80aae70 con 0x5571c39c7440 2024-05-14T15:27:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.025+0000 7f5ba7fff700 1 -- 172.21.15.5:0/2745934860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80ab130 con 0x5571c39c7440 2024-05-14T15:27:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.027+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2745934860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 msgr2=0x5571c39cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.027+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 0x5571c39cf490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800b830 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.028+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2745934860 shutdown_connections 2024-05-14T15:27:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.028+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7850 0x5571c39c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.028+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.028+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2745934860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7440 0x5571c39cf490 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.028+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2745934860 >> 172.21.15.5:0/2745934860 conn(0x5571c3890630 msgr2=0x5571c39c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.030+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2745934860 shutdown_connections 2024-05-14T15:27:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.030+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2745934860 wait complete. 2024-05-14T15:27:05.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.031+0000 7f5bd319fb80 1 -- start start 2024-05-14T15:27:05.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.031+0000 7f5bd319fb80 1 Processor -- start 2024-05-14T15:27:05.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.032+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.032+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.032+0000 7f5bd319fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bc2527700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4160c50 con 0x5571c39c7850 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41620c0 con 0x5571c39c7440 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bd319fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4161320 con 0x5571c3848e40 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bc2527700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35970/0 (socket says 172.21.15.5:35970) 2024-05-14T15:27:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bc2527700 1 -- 172.21.15.5:0/2802927698 learned_addr learned my addr 172.21.15.5:0/2802927698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bd319fb80 10 client.0 _ll_register_callbacks cb 0x5571c3880950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.033+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.034+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2802927698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 msgr2=0x5571c39d92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.034+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.034+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2802927698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 msgr2=0x5571c39d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.034+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.034+0000 7f5bc1d26700 1 -- 172.21.15.5:0/2802927698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c39d4920 con 0x5571c39c7440 2024-05-14T15:27:05.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.036+0000 7f5bc1d26700 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d3bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ab360 tx=0x7f5bb800b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb130b0dd22c2921 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.036+0000 7f5ba5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.037+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/2802927698 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80aacb0 con 0x5571c39c7440 2024-05-14T15:27:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.037+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/2802927698 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7f5bb80b3ca0 con 0x5571c39c7440 2024-05-14T15:27:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.037+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb80aae70 con 0x5571c39c7440 2024-05-14T15:27:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.039+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/2802927698 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5bb80a7220 con 0x5571c39c7440 2024-05-14T15:27:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.040+0000 7f5bd319fb80 10 client.25514 fetch_fsmap learned FSMap version 26 2024-05-14T15:27:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.040+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571c39d4920 con 0x5571c39c7440 2024-05-14T15:27:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.041+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/2802927698 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb80aae70 con 0x5571c39c7440 2024-05-14T15:27:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.041+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/2802927698 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5bb802c020 con 0x5571c39c7440 2024-05-14T15:27:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bd319fb80 10 client.25514 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:27:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bd319fb80 10 client.25514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:27:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bd319fb80 20 client.25514 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c3a4a0a0 0x5571c3a4c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bd319fb80 4 client.25514 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571c3768250 con 0x5571c3a4a0a0 2024-05-14T15:27:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.042+0000 7f5bc2527700 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c3a4a0a0 0x5571c3a4c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.047+0000 7f5bc2527700 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c3a4a0a0 0x5571c3a4c4c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cce8e9766af1a45 server_cookie=41352c658f180f53 in_seq=0 out_seq=0 2024-05-14T15:27:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.047+0000 7f5ba5ffb700 10 client.25514 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.048+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/2802927698 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571c3768250 con 0x5571c3a4a0a0 2024-05-14T15:27:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.048+0000 7f5ba5ffb700 10 client.25514 handle_command_reply: tid=0 2024-05-14T15:27:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.049+0000 7f5bd319fb80 1 client.25514 shutdown 2024-05-14T15:27:05.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.050+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 msgr2=0x5571c39d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.050+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d3bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ab360 tx=0x7f5bb800b6a0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.051+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 shutdown_connections 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.051+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5571c3a4a0a0 0x5571c3a4c4c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.051+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c39c7850 0x5571c39d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.051+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571c3848e40 0x5571c39d92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.051+0000 7f5bd319fb80 1 --2- 172.21.15.5:0/2802927698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5571c39c7440 0x5571c39d3bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.051+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 >> 172.21.15.5:0/2802927698 conn(0x5571c3890630 msgr2=0x5571c39bbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.052+0000 7f5ba5ffb700 0 client.25514 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:05.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.053+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 shutdown_connections 2024-05-14T15:27:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.054+0000 7f5bd319fb80 1 -- 172.21.15.5:0/2802927698 wait complete. 2024-05-14T15:27:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.054+0000 7f5bd319fb80 20 client.25514 trim_cache size 0 max 16384 2024-05-14T15:27:05.056 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ba80b2b30 msgr2=0x7f5ba80b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ba80b2b30 0x7f5ba80b5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc000eb0 tx=0x7f5bbc06c040 comp rx=0 tx=0).stop 2024-05-14T15:27:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 msgr2=0x7f5bb40a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 0x7f5bb40a4a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8006010 tx=0x7f5bb800ff80 comp rx=0 tx=0).stop 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 shutdown_connections 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5ba80b2b30 0x7f5ba80b5040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb409e7f0 0x7f5bb40a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bb40987c0 0x7f5bb40aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 --2- 172.21.15.5:0/371046094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bb4099110 0x7f5bb40a4a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.056+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 >> 172.21.15.5:0/371046094 conn(0x7f5bb40942e0 msgr2=0x7f5bb400c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.057+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 shutdown_connections 2024-05-14T15:27:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:05.057+0000 7f5bc478b700 1 -- 172.21.15.5:0/371046094 wait complete. 2024-05-14T15:27:05.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 66271 14.31 MB/sec warmup 26 sec latency 143.730 ms 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 -- 172.21.15.5:0/3505915833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574108ba0 msgr2=0x7ff574102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 --2- 172.21.15.5:0/3505915833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574108ba0 0x7ff574102740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff564009ca0 tx=0x7ff56402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 -- 172.21.15.5:0/3505915833 shutdown_connections 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 --2- 172.21.15.5:0/3505915833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5741032f0 0x7ff57410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 --2- 172.21.15.5:0/3505915833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574109040 0x7ff574102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 --2- 172.21.15.5:0/3505915833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574108ba0 0x7ff574102740 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 -- 172.21.15.5:0/3505915833 >> 172.21.15.5:0/3505915833 conn(0x7ff5740782e0 msgr2=0x7ff574078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 -- 172.21.15.5:0/3505915833 shutdown_connections 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.526+0000 7ff579da8640 1 -- 172.21.15.5:0/3505915833 wait complete. 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 Processor -- start 2024-05-14T15:27:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 -- start start 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5741032f0 0x7ff5741997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574108ba0 0x7ff574199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 0x7ff57419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112d90 con 0x7ff574109040 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112c10 con 0x7ff574108ba0 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff579da8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112f10 con 0x7ff5741032f0 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff573fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 0x7ff57419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff573fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 0x7ff57419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50326/0 (socket says 172.21.15.5:50326) 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.527+0000 7ff573fff640 1 -- 172.21.15.5:0/1040848866 learned_addr learned my addr 172.21.15.5:0/1040848866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574108ba0 0x7ff574199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff573fff640 1 -- 172.21.15.5:0/1040848866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5741032f0 msgr2=0x7ff5741997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff573fff640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5741032f0 0x7ff5741997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff573fff640 1 -- 172.21.15.5:0/1040848866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574108ba0 msgr2=0x7ff574199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff573fff640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574108ba0 0x7ff574199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff573fff640 1 -- 172.21.15.5:0/1040848866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57419e630 con 0x7ff574109040 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574108ba0 0x7ff574199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff573fff640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 0x7ff57419dff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00d750 tx=0x7ff56c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff570ff9640 1 -- 172.21.15.5:0/1040848866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c017070 con 0x7ff574109040 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57419e920 con 0x7ff574109040 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff570ff9640 1 -- 172.21.15.5:0/1040848866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56c0054d0 con 0x7ff574109040 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff570ff9640 1 -- 172.21.15.5:0/1040848866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004c60 con 0x7ff574109040 2024-05-14T15:27:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.528+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a62a0 con 0x7ff574109040 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 Processor -- start 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff570ff9640 1 -- 172.21.15.5:0/1040848866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff56c010460 con 0x7ff574109040 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 -- start start 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac993ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac993cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac957b4d0 0x563ac993d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ac98a25d0 con 0x563ac95732f0 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563ac985b890 con 0x563ac957b4d0 2024-05-14T15:27:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.531+0000 7ff580c5b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ac989e590 con 0x563ac959c1b0 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac993cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac993cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50340/0 (socket says 172.21.15.5:50340) 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 -- 172.21.15.5:0/3871335055 learned_addr learned my addr 172.21.15.5:0/3871335055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff572ffd640 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac993ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff572ffd640 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac957b4d0 0x563ac993d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff570ff9640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5600b2950 0x7ff5600b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 -- 172.21.15.5:0/3871335055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 msgr2=0x563ac993ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac993ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 -- 172.21.15.5:0/3871335055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac957b4d0 msgr2=0x563ac993d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac957b4d0 0x563ac993d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 -- 172.21.15.5:0/3871335055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ac95b4210 con 0x563ac95732f0 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff573fff640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5600b2950 0x7ff5600b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff570ff9640 1 -- 172.21.15.5:0/1040848866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7ff56c0e2460 con 0x7ff574109040 2024-05-14T15:27:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.532+0000 7ff5737fe640 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac993cf70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff5640097b0 tx=0x7ff56402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72bf1946f56a219 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.533+0000 7ff569ffb640 1 -- 172.21.15.5:0/3871335055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564031cf0 con 0x563ac95732f0 2024-05-14T15:27:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.533+0000 7ff569ffb640 1 -- 172.21.15.5:0/3871335055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56403f070 con 0x563ac95732f0 2024-05-14T15:27:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.533+0000 7ff569ffb640 1 -- 172.21.15.5:0/3871335055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564004040 con 0x563ac95732f0 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 -- 172.21.15.5:0/3871335055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 msgr2=0x563ac993cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff573fff640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5600b2950 0x7ff5600b4e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0dfdf0 tx=0x7ff56c09dc00 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac993cf70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff5640097b0 tx=0x7ff56402f420 comp rx=0 tx=0).stop 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 -- 172.21.15.5:0/3871335055 shutdown_connections 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac957b4d0 0x563ac993d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac993cf70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3871335055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac993ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 -- 172.21.15.5:0/3871335055 >> 172.21.15.5:0/3871335055 conn(0x563ac9590300 msgr2=0x563ac9939340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 -- 172.21.15.5:0/3871335055 shutdown_connections 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 -- 172.21.15.5:0/3871335055 wait complete. 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 -- start start 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.534+0000 7ff580c5b740 1 Processor -- start 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac99493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac957b4d0 0x563ac99498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac9955390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ac994bf20 con 0x563ac95732f0 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563ac985f370 con 0x563ac959c1b0 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ac994c0a0 con 0x563ac957b4d0 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff580c5b740 10 client.0 _ll_register_callbacks cb 0x563ac95b3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac99493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac99493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50348/0 (socket says 172.21.15.5:50348) 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 -- 172.21.15.5:0/2615998529 learned_addr learned my addr 172.21.15.5:0/2615998529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac9955390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff5737fe640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac957b4d0 0x563ac99498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 -- 172.21.15.5:0/2615998529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac957b4d0 msgr2=0x563ac99498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac957b4d0 0x563ac99498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 -- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 msgr2=0x563ac9955390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac9955390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 -- 172.21.15.5:0/2615998529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ac9680620 con 0x563ac95732f0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff572ffd640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac99493b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff574109470 tx=0x7ff55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392920d811b2c45e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.535+0000 7ff552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff552ffd640 1 -- 172.21.15.5:0/2615998529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c005020 con 0x563ac95732f0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff552ffd640 1 -- 172.21.15.5:0/2615998529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7ff55c0051c0 con 0x563ac95732f0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56403f070 con 0x563ac95732f0 2024-05-14T15:27:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff552ffd640 1 -- 172.21.15.5:0/2615998529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff55c0053c0 con 0x563ac95732f0 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 10 client.15879 fetch_fsmap learned FSMap version 26 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ac99581e0 con 0x563ac95732f0 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff552ffd640 1 -- 172.21.15.5:0/2615998529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56403f070 con 0x563ac95732f0 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff552ffd640 1 -- 172.21.15.5:0/2615998529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff55c020030 con 0x563ac95732f0 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 10 client.15879 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 20 client.15879 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac955ad80 0x563ac995b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 4 client.15879 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.536+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ac98b01e0 con 0x563ac955ad80 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.537+0000 7ff5737fe640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac955ad80 0x563ac995b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.537+0000 7ff552ffd640 10 client.15879 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.537+0000 7ff5737fe640 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac955ad80 0x563ac995b6f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6ce7529ddd2369 server_cookie=58d09f7c6decf63 in_seq=0 out_seq=0 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff552ffd640 1 -- 172.21.15.5:0/2615998529 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ac98b01e0 con 0x563ac955ad80 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff552ffd640 10 client.15879 handle_command_reply: tid=0 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 client.15879 shutdown 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 msgr2=0x563ac99493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac99493b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff574109470 tx=0x7ff55c0046c0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 shutdown_connections 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac955ad80 0x563ac995b6f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac957b4d0 0x563ac99498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac9955390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff552ffd640 0 client.15879 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 --2- 172.21.15.5:0/2615998529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac95732f0 0x563ac99493b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 >> 172.21.15.5:0/2615998529 conn(0x563ac9590300 msgr2=0x563ac993d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 shutdown_connections 2024-05-14T15:27:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 1 -- 172.21.15.5:0/2615998529 wait complete. 2024-05-14T15:27:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.539+0000 7ff580c5b740 20 client.15879 trim_cache size 0 max 16384 2024-05-14T15:27:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 Processor -- start 2024-05-14T15:27:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 -- start start 2024-05-14T15:27:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac95732f0 0x563ac994b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac9942c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac993d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b1a90 con 0x563ac957b4d0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741af360 con 0x563ac959c1b0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.556+0000 7ff580c5b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741ad1b0 con 0x563ac95732f0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac9942c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff572ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac993d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac9942c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50356/0 (socket says 172.21.15.5:50356) 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 -- 172.21.15.5:0/643201650 learned_addr learned my addr 172.21.15.5:0/643201650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff572ffd640 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac95732f0 0x563ac994b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 -- 172.21.15.5:0/643201650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac95732f0 msgr2=0x563ac994b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac95732f0 0x563ac994b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 -- 172.21.15.5:0/643201650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 msgr2=0x563ac993d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac993d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 -- 172.21.15.5:0/643201650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ac994ba40 con 0x563ac957b4d0 2024-05-14T15:27:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5737fe640 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac9942c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff5640097b0 tx=0x7ff564003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af0a0512b98cbd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5697fa640 1 -- 172.21.15.5:0/643201650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564003a90 con 0x563ac957b4d0 2024-05-14T15:27:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5697fa640 1 -- 172.21.15.5:0/643201650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff564051e50 con 0x563ac957b4d0 2024-05-14T15:27:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.557+0000 7ff5697fa640 1 -- 172.21.15.5:0/643201650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56402fb60 con 0x563ac957b4d0 2024-05-14T15:27:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 -- 172.21.15.5:0/643201650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 msgr2=0x563ac9942c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac9942c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff5640097b0 tx=0x7ff564003c30 comp rx=0 tx=0).stop 2024-05-14T15:27:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 -- 172.21.15.5:0/643201650 shutdown_connections 2024-05-14T15:27:05.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac959c1b0 0x563ac993d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac9942c20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 --2- 172.21.15.5:0/643201650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac95732f0 0x563ac994b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 -- 172.21.15.5:0/643201650 >> 172.21.15.5:0/643201650 conn(0x563ac9590300 msgr2=0x563ac993c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 -- 172.21.15.5:0/643201650 shutdown_connections 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.558+0000 7ff580c5b740 1 -- 172.21.15.5:0/643201650 wait complete. 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 -- start start 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 Processor -- start 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac95732f0 0x563ac9953f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac99544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac99549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741ad820 con 0x563ac957b4d0 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741aec90 con 0x563ac95732f0 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741adef0 con 0x563ac959c1b0 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.559+0000 7ff580c5b740 10 client.0 _ll_register_callbacks cb 0x7ff55c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac99544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac99544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50364/0 (socket says 172.21.15.5:50364) 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 -- 172.21.15.5:0/3688971096 learned_addr learned my addr 172.21.15.5:0/3688971096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff572ffd640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac95732f0 0x563ac9953f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff572ffd640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac99549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 -- 172.21.15.5:0/3688971096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 msgr2=0x563ac99549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac99549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 -- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac95732f0 msgr2=0x563ac9953f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac95732f0 0x563ac9953f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 -- 172.21.15.5:0/3688971096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ac99502f0 con 0x563ac957b4d0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff5737fe640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac99544a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff5640094e0 tx=0x7ff564005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39319b29eabe161b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff552ffd640 1 -- 172.21.15.5:0/3688971096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff564055070 con 0x563ac957b4d0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff552ffd640 1 -- 172.21.15.5:0/3688971096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9474+0+0 (secure 0 0 0) 0x7ff564031450 con 0x563ac957b4d0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.560+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff564051e50 con 0x563ac957b4d0 2024-05-14T15:27:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff552ffd640 1 -- 172.21.15.5:0/3688971096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff564031650 con 0x563ac957b4d0 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ac9951580 con 0x563ac957b4d0 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff552ffd640 1 -- 172.21.15.5:0/3688971096 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff564051e50 con 0x563ac957b4d0 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff552ffd640 1 -- 172.21.15.5:0/3688971096 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff56402fbc0 con 0x563ac957b4d0 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 20 client.15885 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac95301c0 0x563ac9951810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 4 client.15885 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ac98aa050 con 0x563ac95301c0 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.561+0000 7ff572ffd640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac95301c0 0x563ac9951810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.562+0000 7ff552ffd640 10 client.15885 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.562+0000 7ff572ffd640 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac95301c0 0x563ac9951810 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f21d8075155b0d9 server_cookie=1bb9d1d41a16c305 in_seq=0 out_seq=0 2024-05-14T15:27:05.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff552ffd640 1 -- 172.21.15.5:0/3688971096 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ac98aa050 con 0x563ac95301c0 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff552ffd640 10 client.15885 handle_command_reply: tid=0 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 client.15885 shutdown 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 msgr2=0x563ac99544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac99544a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff5640094e0 tx=0x7ff564005c60 comp rx=0 tx=0).stop 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 shutdown_connections 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x563ac95301c0 0x563ac9951810 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ac959c1b0 0x563ac99549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ac957b4d0 0x563ac99544a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 --2- 172.21.15.5:0/3688971096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ac95732f0 0x563ac9953f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 >> 172.21.15.5:0/3688971096 conn(0x563ac9590300 msgr2=0x563ac993ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.939+0000 7ff552ffd640 0 client.15885 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 shutdown_connections 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff580c5b740 1 -- 172.21.15.5:0/3688971096 wait complete. 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff580c5b740 20 client.15885 trim_cache size 0 max 16384 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:27:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5600b2950 msgr2=0x7ff5600b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff579da8640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5600b2950 0x7ff5600b4e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0dfdf0 tx=0x7ff56c09dc00 comp rx=0 tx=0).stop 2024-05-14T15:27:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 msgr2=0x7ff57419dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.940+0000 7ff579da8640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 0x7ff57419dff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00d750 tx=0x7ff56c00dc10 comp rx=0 tx=0).stop 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 shutdown_connections 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5600b2950 0x7ff5600b4e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574109040 0x7ff57419dff0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574108ba0 0x7ff574199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 --2- 172.21.15.5:0/1040848866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5741032f0 0x7ff5741997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 >> 172.21.15.5:0/1040848866 conn(0x7ff5740782e0 msgr2=0x7ff5740ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 shutdown_connections 2024-05-14T15:27:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:05.941+0000 7ff579da8640 1 -- 172.21.15.5:0/1040848866 wait complete. 2024-05-14T15:27:06.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:05 smithi005 ceph-mon[31559]: pgmap v70: 129 pgs: 129 active+clean; 21 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 355 op/s 2024-05-14T15:27:06.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 66850 13.99 MB/sec warmup 27 sec latency 358.606 ms 2024-05-14T15:27:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:05 smithi144 ceph-mon[35650]: pgmap v70: 129 pgs: 129 active+clean; 21 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 355 op/s 2024-05-14T15:27:06.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:05 smithi193 ceph-mon[36001]: pgmap v70: 129 pgs: 129 active+clean; 21 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 355 op/s 2024-05-14T15:27:06.798 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setuser ceph since I am not root 2024-05-14T15:27:06.798 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setgroup ceph since I am not root 2024-05-14T15:27:06.798 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:06 smithi144 ceph-mds[65770]: -- 172.21.15.144:0/3505780221 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563081609a40 con 0x563081651a80 2024-05-14T15:27:06.798 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: starting mds.e at 2024-05-14T15:27:07.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 67860 13.72 MB/sec warmup 28 sec latency 53.260 ms 2024-05-14T15:27:07.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:06 smithi144 ceph-mon[35650]: daemon mds.e restarted 2024-05-14T15:27:07.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:06 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:27:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:06 smithi193 ceph-mon[36001]: daemon mds.e restarted 2024-05-14T15:27:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:06 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:27:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:06 smithi005 ceph-mon[31559]: daemon mds.e restarted 2024-05-14T15:27:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:06 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 -- 172.21.15.5:0/748366917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf01034c0 msgr2=0x7f7bf01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/748366917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf01034c0 0x7f7bf01038d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7be00099c0 tx=0x7f7be0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 -- 172.21.15.5:0/748366917 shutdown_connections 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/748366917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf0108740 0x7f7bf01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/748366917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf01034c0 0x7f7bf01038d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/748366917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0107df0 0x7f7bf0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 -- 172.21.15.5:0/748366917 >> 172.21.15.5:0/748366917 conn(0x7f7bf00fedc0 msgr2=0x7f7bf01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 -- 172.21.15.5:0/748366917 shutdown_connections 2024-05-14T15:27:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.092+0000 7f7bf7e75700 1 -- 172.21.15.5:0/748366917 wait complete. 2024-05-14T15:27:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 Processor -- start 2024-05-14T15:27:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 -- start start 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf01034c0 0x7f7bf019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 0x7f7bf019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0108740 0x7f7bf019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0113370 con 0x7f7bf0107df0 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01131f0 con 0x7f7bf01034c0 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf7e75700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01134f0 con 0x7f7bf0108740 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf5410700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 0x7f7bf019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf5410700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 0x7f7bf019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53514/0 (socket says 172.21.15.5:53514) 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf5410700 1 -- 172.21.15.5:0/31735638 learned_addr learned my addr 172.21.15.5:0/31735638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf6412700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0108740 0x7f7bf019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.093+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf01034c0 0x7f7bf019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf6412700 1 -- 172.21.15.5:0/31735638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf01034c0 msgr2=0x7f7bf019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf6412700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf01034c0 0x7f7bf019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf6412700 1 -- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 msgr2=0x7f7bf019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf6412700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 0x7f7bf019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf6412700 1 -- 172.21.15.5:0/31735638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf01a7000 con 0x7f7bf0108740 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf5410700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 0x7f7bf019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf6412700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0108740 0x7f7bf019c750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be800efc0 tx=0x7f7be800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7beeffd700 1 -- 172.21.15.5:0/31735638 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be80092e0 con 0x7f7bf0108740 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7beeffd700 1 -- 172.21.15.5:0/31735638 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8004bb0 con 0x7f7bf0108740 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf01a7290 con 0x7f7bf0108740 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7beeffd700 1 -- 172.21.15.5:0/31735638 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004020 con 0x7f7bf0108740 2024-05-14T15:27:08.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.094+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01a77d0 con 0x7f7bf0108740 2024-05-14T15:27:08.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.098+0000 7f7beeffd700 1 -- 172.21.15.5:0/31735638 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7be801b030 con 0x7f7bf0108740 2024-05-14T15:27:08.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.098+0000 7f7beeffd700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7bd80b2a70 0x7f7bd80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.098+0000 7f7c06889b80 1 Processor -- start 2024-05-14T15:27:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.098+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7bd80b2a70 0x7f7bd80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7beeffd700 1 -- 172.21.15.5:0/31735638 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f7be80e23f0 con 0x7f7bf0108740 2024-05-14T15:27:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 -- start start 2024-05-14T15:27:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7bd80b2a70 0x7f7bd80b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000c00 tx=0x7f7be001a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f4bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f50350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ea3e69860 con 0x564ea3f50890 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ea3e087c0 con 0x564ea3f4c070 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7c06889b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564ea3e65820 con 0x564ea3dc9e40 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7bf5410700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f50350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7bf5c11700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f4bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7bf5410700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f50350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41102/0 (socket says 172.21.15.5:41102) 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7bf6412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.099+0000 7f7bf5410700 1 -- 172.21.15.5:0/2668119481 learned_addr learned my addr 172.21.15.5:0/2668119481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf5410700 1 -- 172.21.15.5:0/2668119481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 msgr2=0x564ea3f4bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf5410700 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f4bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf5410700 1 -- 172.21.15.5:0/2668119481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 msgr2=0x564ea3f50ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf5410700 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f50ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf5410700 1 -- 172.21.15.5:0/2668119481 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea3db68d0 con 0x564ea3f4c070 2024-05-14T15:27:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf6412700 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f50ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bf5410700 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f50350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7be4011650 tx=0x7f7be4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d4e2b0338bedbf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bd7fff700 1 -- 172.21.15.5:0/2668119481 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be401e070 con 0x564ea3f4c070 2024-05-14T15:27:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bd7fff700 1 -- 172.21.15.5:0/2668119481 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be4019ae0 con 0x564ea3f4c070 2024-05-14T15:27:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.100+0000 7f7bd7fff700 1 -- 172.21.15.5:0/2668119481 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be400eb60 con 0x564ea3f4c070 2024-05-14T15:27:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 -- 172.21.15.5:0/2668119481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 msgr2=0x564ea3f50350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f50350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7be4011650 tx=0x7f7be4009f90 comp rx=0 tx=0).stop 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: pgmap v71: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 724 op/s 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6834/1884688714,v1:172.21.15.144:6835/1884688714] up:boot 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:27:08.104 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:07 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:27:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 -- 172.21.15.5:0/2668119481 shutdown_connections 2024-05-14T15:27:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f50ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f4bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 --2- 172.21.15.5:0/2668119481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f50350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.101+0000 7f7c06889b80 1 -- 172.21.15.5:0/2668119481 >> 172.21.15.5:0/2668119481 conn(0x564ea3e11600 msgr2=0x564ea3f424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.102+0000 7f7c06889b80 1 -- 172.21.15.5:0/2668119481 shutdown_connections 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.102+0000 7f7c06889b80 1 -- 172.21.15.5:0/2668119481 wait complete. 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.102+0000 7f7c06889b80 1 -- start start 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.102+0000 7f7c06889b80 1 Processor -- start 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f55500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f55a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f55f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ea3f58840 con 0x564ea3f50890 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ea3dcfc30 con 0x564ea3f4c070 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564ea3f589c0 con 0x564ea3dc9e40 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7bf6412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f55f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7bf6412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f55f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53544/0 (socket says 172.21.15.5:53544) 2024-05-14T15:27:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7bf6412700 1 -- 172.21.15.5:0/3402986750 learned_addr learned my addr 172.21.15.5:0/3402986750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7c06889b80 10 client.0 _ll_register_callbacks cb 0x564ea3e01920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.103+0000 7f7bf5410700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f55a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf6412700 1 -- 172.21.15.5:0/3402986750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 msgr2=0x564ea3f55500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf6412700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f55500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf6412700 1 -- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 msgr2=0x564ea3f55a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf6412700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f55a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf6412700 1 -- 172.21.15.5:0/3402986750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea3e05690 con 0x564ea3f50890 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf5410700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bf6412700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f55f80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7be80a90a0 tx=0x7f7be80a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f50193c2a5003c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bd57fa700 1 -- 172.21.15.5:0/3402986750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004d60 con 0x564ea3f50890 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7bd57fa700 1 -- 172.21.15.5:0/3402986750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f7be80a8600 con 0x564ea3f50890 2024-05-14T15:27:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.104+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be4019ae0 con 0x564ea3f50890 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7bd57fa700 1 -- 172.21.15.5:0/3402986750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7be80a8eb0 con 0x564ea3f50890 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 10 client.15900 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ea3f63410 con 0x564ea3f50890 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7bd57fa700 1 -- 172.21.15.5:0/3402986750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be4019ae0 con 0x564ea3f50890 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7bd57fa700 1 -- 172.21.15.5:0/3402986750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7be80a3ac0 con 0x564ea3f50890 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 20 client.15900 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3f659a0 0x564ea3f67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 4 client.15900 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.105+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea3e77240 con 0x564ea3f659a0 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.106+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3f659a0 0x564ea3f67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.106+0000 7f7bd57fa700 10 client.15900 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.106+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3f659a0 0x564ea3f67de0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44bdd79204315350 server_cookie=cdb42001227526c7 in_seq=0 out_seq=0 2024-05-14T15:27:08.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.108+0000 7f7bd57fa700 1 -- 172.21.15.5:0/3402986750 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ea3e77240 con 0x564ea3f659a0 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.108+0000 7f7bd57fa700 10 client.15900 handle_command_reply: tid=0 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.108+0000 7f7c06889b80 1 client.15900 shutdown 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 msgr2=0x564ea3f55f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f55f80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7be80a90a0 tx=0x7f7be80a98e0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 shutdown_connections 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3f659a0 0x564ea3f67de0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f55f80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3dc9e40 0x564ea3f55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7bd57fa700 0 client.15900 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 --2- 172.21.15.5:0/3402986750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f4c070 0x564ea3f55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 >> 172.21.15.5:0/3402986750 conn(0x564ea3e11600 msgr2=0x564ea3f46450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 shutdown_connections 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 1 -- 172.21.15.5:0/3402986750 wait complete. 2024-05-14T15:27:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.109+0000 7f7c06889b80 20 client.15900 trim_cache size 0 max 16384 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.124+0000 7f7c06889b80 1 Processor -- start 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.124+0000 7f7c06889b80 1 -- start start 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3dc9e40 0x564ea3f55dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7c06889b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01b0ca0 con 0x564ea3f50890 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7c06889b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ae530 con 0x564ea3dc9e40 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7c06889b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ac380 con 0x564ea3f4c070 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53552/0 (socket says 172.21.15.5:53552) 2024-05-14T15:27:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 -- 172.21.15.5:0/317759608 learned_addr learned my addr 172.21.15.5:0/317759608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:08.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 69827 13.49 MB/sec warmup 29 sec latency 134.795 ms 2024-05-14T15:27:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3dc9e40 0x564ea3f55dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf5410700 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 -- 172.21.15.5:0/317759608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 msgr2=0x564ea3f422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 -- 172.21.15.5:0/317759608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3dc9e40 msgr2=0x564ea3f55dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3dc9e40 0x564ea3f55dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 -- 172.21.15.5:0/317759608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea3f43320 con 0x564ea3f50890 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf5410700 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.125+0000 7f7bf6412700 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f618d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7be80dfd80 tx=0x7f7be80b1fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c71c8d872ad4419 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7bd77fe700 1 -- 172.21.15.5:0/317759608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be80a93a0 con 0x564ea3f50890 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7bd77fe700 1 -- 172.21.15.5:0/317759608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be80ad750 con 0x564ea3f50890 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7bd77fe700 1 -- 172.21.15.5:0/317759608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be80a4f70 con 0x564ea3f50890 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 -- 172.21.15.5:0/317759608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 msgr2=0x564ea3f618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f618d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7be80dfd80 tx=0x7f7be80b1fc0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 -- 172.21.15.5:0/317759608 shutdown_connections 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3f50890 0x564ea3f618d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3dc9e40 0x564ea3f55dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 --2- 172.21.15.5:0/317759608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.126+0000 7f7c06889b80 1 -- 172.21.15.5:0/317759608 >> 172.21.15.5:0/317759608 conn(0x564ea3e11600 msgr2=0x564ea3f41be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 -- 172.21.15.5:0/317759608 shutdown_connections 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 -- 172.21.15.5:0/317759608 wait complete. 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 -- start start 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 Processor -- start 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 0x564ea3f5b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.127+0000 7f7c06889b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f50890 0x564ea3f5c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7c06889b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ac9f0 con 0x564ea3dc9e40 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7c06889b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ade60 con 0x564ea3f50890 2024-05-14T15:27:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7c06889b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ad0c0 con 0x564ea3f4c070 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7c06889b80 10 client.0 _ll_register_callbacks cb 0x564ea3e01920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 0x564ea3f5b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 0x564ea3f5b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53562/0 (socket says 172.21.15.5:53562) 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf6412700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f50890 0x564ea3f5c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 -- 172.21.15.5:0/917736238 learned_addr learned my addr 172.21.15.5:0/917736238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5410700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 -- 172.21.15.5:0/917736238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 msgr2=0x564ea3f5bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f5bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 -- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f50890 msgr2=0x564ea3f5c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f50890 0x564ea3f5c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 -- 172.21.15.5:0/917736238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea3fcf160 con 0x564ea3dc9e40 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf6412700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f50890 0x564ea3f5c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5410700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bf5c11700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 0x564ea3f5b860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7be00099c0 tx=0x7f7be0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b88bb774d767011 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bd57fa700 1 -- 172.21.15.5:0/917736238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0038060 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7bd57fa700 1 -- 172.21.15.5:0/917736238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f7be0025460 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.128+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be80ad750 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7bd57fa700 1 -- 172.21.15.5:0/917736238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7be003f050 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 10 client.15906 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ea3fd0360 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7bd57fa700 1 -- 172.21.15.5:0/917736238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be80ad750 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7bd57fa700 1 -- 172.21.15.5:0/917736238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f7be0021070 con 0x564ea3dc9e40 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 20 client.15906 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3fd0a20 0x564ea3fd2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 4 client.15906 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ea3e712e0 con 0x564ea3fd0a20 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.129+0000 7f7bf5410700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3fd0a20 0x564ea3fd2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7bd57fa700 10 client.15906 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7bf5410700 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3fd0a20 0x564ea3fd2e40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3750cef757453615 server_cookie=5f1a23e9ddf087ca in_seq=0 out_seq=0 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7bd57fa700 1 -- 172.21.15.5:0/917736238 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ea3e712e0 con 0x564ea3fd0a20 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7bd57fa700 10 client.15906 handle_command_reply: tid=0 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 client.15906 shutdown 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 msgr2=0x564ea3f5b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 0x564ea3f5b860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7be00099c0 tx=0x7f7be0005040 comp rx=0 tx=0).stop 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 shutdown_connections 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ea3fd0a20 0x564ea3fd2e40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea3f50890 0x564ea3f5c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ea3dc9e40 0x564ea3f5b860 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7bd57fa700 0 client.15906 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 --2- 172.21.15.5:0/917736238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ea3f4c070 0x564ea3f5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.130+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 >> 172.21.15.5:0/917736238 conn(0x564ea3e11600 msgr2=0x564ea3f56b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.131+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 shutdown_connections 2024-05-14T15:27:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.131+0000 7f7c06889b80 1 -- 172.21.15.5:0/917736238 wait complete. 2024-05-14T15:27:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.131+0000 7f7c06889b80 20 client.15906 trim_cache size 0 max 16384 2024-05-14T15:27:08.138 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7bd80b2a70 msgr2=0x7f7bd80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7bd80b2a70 0x7f7bd80b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000c00 tx=0x7f7be001a040 comp rx=0 tx=0).stop 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0108740 msgr2=0x7f7bf019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0108740 0x7f7bf019c750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7be800efc0 tx=0x7f7be800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 shutdown_connections 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7bd80b2a70 0x7f7bd80b4f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bf0108740 0x7f7bf019c750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf01034c0 0x7f7bf019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 --2- 172.21.15.5:0/31735638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bf0107df0 0x7f7bf019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:08.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 >> 172.21.15.5:0/31735638 conn(0x7f7bf00fedc0 msgr2=0x7f7bf01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:08.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 shutdown_connections 2024-05-14T15:27:08.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:08.132+0000 7f7bf7e75700 1 -- 172.21.15.5:0/31735638 wait complete. 2024-05-14T15:27:08.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: pgmap v71: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 724 op/s 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6834/1884688714,v1:172.21.15.144:6835/1884688714] up:boot 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:27:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:07 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:27:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: pgmap v71: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 724 op/s 2024-05-14T15:27:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T15:27:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6834/1884688714,v1:172.21.15.144:6835/1884688714] up:boot 2024-05-14T15:27:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:27:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:27:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:27:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:27:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:27:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:07 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:27:09.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71906 13.47 MB/sec warmup 30 sec latency 12.476 ms 2024-05-14T15:27:09.836 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:27:09.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:27:10.105 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:10.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 74840 13.62 MB/sec warmup 31 sec latency 13.906 ms 2024-05-14T15:27:10.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:09 smithi005 ceph-mon[31559]: pgmap v73: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-05-14T15:27:10.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:09 smithi144 ceph-mon[35650]: pgmap v73: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-05-14T15:27:10.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:09 smithi193 ceph-mon[36001]: pgmap v73: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-05-14T15:27:11.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 77422 13.56 MB/sec warmup 32 sec latency 18.694 ms 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.965+0000 7f63fd4a1700 1 -- 172.21.15.5:0/3794560474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109890 msgr2=0x7f63f81111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.965+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/3794560474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109890 0x7f63f81111f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f63f00099c0 tx=0x7f63f0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 -- 172.21.15.5:0/3794560474 shutdown_connections 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/3794560474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109890 0x7f63f81111f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/3794560474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 0x7f63f8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/3794560474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63f8108f40 0x7f63f8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 -- 172.21.15.5:0/3794560474 >> 172.21.15.5:0/3794560474 conn(0x7f63f807af70 msgr2=0x7f63f807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 -- 172.21.15.5:0/3794560474 shutdown_connections 2024-05-14T15:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 -- 172.21.15.5:0/3794560474 wait complete. 2024-05-14T15:27:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.966+0000 7f63fd4a1700 1 Processor -- start 2024-05-14T15:27:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 -- start start 2024-05-14T15:27:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 0x7f63f819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 0x7f63f819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63f8109890 0x7f63f819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81144c0 con 0x7f63f8108f40 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8114340 con 0x7f63f8109890 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63fd4a1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8114640 con 0x7f63f8104610 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 0x7f63f819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 0x7f63f819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 0x7f63f819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53582/0 (socket says 172.21.15.5:53582) 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63f8109890 0x7f63f819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 0x7f63f819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37868/0 (socket says 172.21.15.5:37868) 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 -- 172.21.15.5:0/73223530 learned_addr learned my addr 172.21.15.5:0/73223530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 -- 172.21.15.5:0/73223530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 msgr2=0x7f63f819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 0x7f63f819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 -- 172.21.15.5:0/73223530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63f8109890 msgr2=0x7f63f819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63f8109890 0x7f63f819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.967+0000 7f63f67fc700 1 -- 172.21.15.5:0/73223530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f819fca0 con 0x7f63f8108f40 2024-05-14T15:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.968+0000 7f63f67fc700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 0x7f63f819b370 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f63e000e970 tx=0x7f63e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.968+0000 7f63effff700 1 -- 172.21.15.5:0/73223530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0004a30 con 0x7f63f8108f40 2024-05-14T15:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.968+0000 7f63effff700 1 -- 172.21.15.5:0/73223530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e000cc90 con 0x7f63f8108f40 2024-05-14T15:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.968+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f819ff90 con 0x7f63f8108f40 2024-05-14T15:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.968+0000 7f63effff700 1 -- 172.21.15.5:0/73223530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0005610 con 0x7f63f8108f40 2024-05-14T15:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.968+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f80759f0 con 0x7f63f8108f40 2024-05-14T15:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.971+0000 7f63effff700 1 -- 172.21.15.5:0/73223530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f63e0010430 con 0x7f63f8108f40 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f63effff700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63e40b29f0 0x7f63e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 Processor -- start 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 -- start start 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63e40b29f0 0x7f63e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f63effff700 1 -- 172.21.15.5:0/73223530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f63e00e2d40 con 0x7f63f8108f40 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9464b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fc9456c9c0 con 0x55fc94650e00 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fc944d1e20 con 0x55fc944ccec0 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.972+0000 7f640beb5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc94568980 con 0x55fc9464ae90 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53586/0 (socket says 172.21.15.5:53586) 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f67fc700 1 -- 172.21.15.5:0/722808225 learned_addr learned my addr 172.21.15.5:0/722808225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9464b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63e40b29f0 0x7f63e40b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009510 tx=0x7f63e8009230 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 -- 172.21.15.5:0/722808225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 msgr2=0x55fc946508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 -- 172.21.15.5:0/722808225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 msgr2=0x55fc9464a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 -- 172.21.15.5:0/722808225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc944b9950 con 0x55fc9464ae90 2024-05-14T15:27:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63f77fe700 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9464b2a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009990 tx=0x7f63f0005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f7081c0e920598 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63ed7fa700 1 -- 172.21.15.5:0/722808225 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0027070 con 0x55fc9464ae90 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63ed7fa700 1 -- 172.21.15.5:0/722808225 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f00037c0 con 0x55fc9464ae90 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.973+0000 7f63ed7fa700 1 -- 172.21.15.5:0/722808225 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000b9d0 con 0x55fc9464ae90 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 -- 172.21.15.5:0/722808225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 msgr2=0x55fc9464b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9464b2a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009990 tx=0x7f63f0005040 comp rx=0 tx=0).stop 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 -- 172.21.15.5:0/722808225 shutdown_connections 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9464b2a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 --2- 172.21.15.5:0/722808225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 -- 172.21.15.5:0/722808225 >> 172.21.15.5:0/722808225 conn(0x55fc945146b0 msgr2=0x55fc94645a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.974+0000 7f640beb5b80 1 -- 172.21.15.5:0/722808225 shutdown_connections 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 -- 172.21.15.5:0/722808225 wait complete. 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 -- start start 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 Processor -- start 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9464ae90 0x55fc94654a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc94650e00 0x55fc94654f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fc94657840 con 0x55fc9464ae90 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fc94525f50 con 0x55fc944ccec0 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.975+0000 7f640beb5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc946579c0 con 0x55fc94650e00 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f640beb5b80 10 client.0 _ll_register_callbacks cb 0x55fc945049d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9464ae90 0x55fc94654a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41190/0 (socket says 172.21.15.5:41190) 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f6ffd700 1 -- 172.21.15.5:0/387173802 learned_addr learned my addr 172.21.15.5:0/387173802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f77fe700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc94650e00 0x55fc94654f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 -- 172.21.15.5:0/387173802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc94650e00 msgr2=0x55fc94654f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc94650e00 0x55fc94654f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 -- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 msgr2=0x55fc946544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 -- 172.21.15.5:0/387173802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc945237e0 con 0x55fc9464ae90 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63f67fc700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9464ae90 0x55fc94654a20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f63e0005060 tx=0x7f63e0004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc78dca773dce125 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63de7fc700 1 -- 172.21.15.5:0/387173802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e00a5390 con 0x55fc9464ae90 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f63de7fc700 1 -- 172.21.15.5:0/387173802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f63e00c4290 con 0x55fc9464ae90 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.976+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f00037c0 con 0x55fc9464ae90 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f63de7fc700 1 -- 172.21.15.5:0/387173802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63e000ce40 con 0x55fc9464ae90 2024-05-14T15:27:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 10 client.15918 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc94662440 con 0x55fc9464ae90 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f63de7fc700 1 -- 172.21.15.5:0/387173802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f00037c0 con 0x55fc9464ae90 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f63de7fc700 1 -- 172.21.15.5:0/387173802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f63e00a8e60 con 0x55fc9464ae90 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 20 client.15918 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc94664a80 0x55fc94666ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 4 client.15918 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc9457a3a0 con 0x55fc94664a80 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.977+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc94664a80 0x55fc94666ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.978+0000 7f63de7fc700 10 client.15918 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.978+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc94664a80 0x55fc94666ee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467d0f635f076a84 server_cookie=22db15c82f7a4add in_seq=0 out_seq=0 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f63de7fc700 1 -- 172.21.15.5:0/387173802 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc9457a3a0 con 0x55fc94664a80 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f63de7fc700 10 client.15918 handle_command_reply: tid=0 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 client.15918 shutdown 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9464ae90 msgr2=0x55fc94654a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9464ae90 0x55fc94654a20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f63e0005060 tx=0x7f63e0004e90 comp rx=0 tx=0).stop 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 shutdown_connections 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc94664a80 0x55fc94666ee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc94650e00 0x55fc94654f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc946544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 --2- 172.21.15.5:0/387173802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9464ae90 0x55fc94654a20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 >> 172.21.15.5:0/387173802 conn(0x55fc945146b0 msgr2=0x55fc94649940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f63de7fc700 0 client.15918 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:11.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 shutdown_connections 2024-05-14T15:27:11.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 1 -- 172.21.15.5:0/387173802 wait complete. 2024-05-14T15:27:11.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:11.981+0000 7f640beb5b80 20 client.15918 trim_cache size 0 max 16384 2024-05-14T15:27:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 Processor -- start 2024-05-14T15:27:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 -- start start 2024-05-14T15:27:12.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 0x55fc9464a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9464ae90 0x55fc9464fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81b0ca0 con 0x55fc94650e00 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81ae530 con 0x55fc9464ae90 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f640beb5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81ac380 con 0x55fc944ccec0 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.003+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 0x55fc9464a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53596/0 (socket says 172.21.15.5:53596) 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 0x55fc9464a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37906/0 (socket says 172.21.15.5:37906) 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 -- 172.21.15.5:0/3495371923 learned_addr learned my addr 172.21.15.5:0/3495371923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f67fc700 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9464ae90 0x55fc9464fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 -- 172.21.15.5:0/3495371923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 msgr2=0x55fc9464a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 0x55fc9464a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 -- 172.21.15.5:0/3495371923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9464ae90 msgr2=0x55fc9464fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9464ae90 0x55fc9464fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 -- 172.21.15.5:0/3495371923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc94652a70 con 0x55fc94650e00 2024-05-14T15:27:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 0x55fc9464a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63f77fe700 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464e8e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f63f00099c0 tx=0x7f63f00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567d94c5c36fb28d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63ecff9700 1 -- 172.21.15.5:0/3495371923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0027040 con 0x55fc94650e00 2024-05-14T15:27:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63ecff9700 1 -- 172.21.15.5:0/3495371923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f0032070 con 0x55fc94650e00 2024-05-14T15:27:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.004+0000 7f63ecff9700 1 -- 172.21.15.5:0/3495371923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f000bb50 con 0x55fc94650e00 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 -- 172.21.15.5:0/3495371923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 msgr2=0x55fc9464e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464e8e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f63f00099c0 tx=0x7f63f00050b0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 -- 172.21.15.5:0/3495371923 shutdown_connections 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9464e8e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc944ccec0 0x55fc9464a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 --2- 172.21.15.5:0/3495371923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9464ae90 0x55fc9464fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 -- 172.21.15.5:0/3495371923 >> 172.21.15.5:0/3495371923 conn(0x55fc945146b0 msgr2=0x55fc9464e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.005+0000 7f640beb5b80 1 -- 172.21.15.5:0/3495371923 shutdown_connections 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.006+0000 7f640beb5b80 1 -- 172.21.15.5:0/3495371923 wait complete. 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.006+0000 7f640beb5b80 1 -- start start 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.006+0000 7f640beb5b80 1 Processor -- start 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc9466c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9466c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9466caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81ac9f0 con 0x55fc94650e00 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81ade60 con 0x55fc944ccec0 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81ad0c0 con 0x55fc9464ae90 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f640beb5b80 10 client.0 _ll_register_callbacks cb 0x55fc945049d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc9466c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc9466c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41212/0 (socket says 172.21.15.5:41212) 2024-05-14T15:27:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 -- 172.21.15.5:0/2317643814 learned_addr learned my addr 172.21.15.5:0/2317643814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f67fc700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9466c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f77fe700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9466caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 -- 172.21.15.5:0/2317643814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 msgr2=0x55fc9466c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9466c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 -- 172.21.15.5:0/2317643814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 msgr2=0x55fc9466caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9466caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.007+0000 7f63f6ffd700 1 -- 172.21.15.5:0/2317643814 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc946ce380 con 0x55fc944ccec0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f63f6ffd700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc9466c070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55fc94653d60 tx=0x7f63e8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbbd570a2848564d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f63de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f63de7fc700 1 -- 172.21.15.5:0/2317643814 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8021040 con 0x55fc944ccec0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f0032070 con 0x55fc944ccec0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f63de7fc700 1 -- 172.21.15.5:0/2317643814 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f63e802d540 con 0x55fc944ccec0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f63de7fc700 1 -- 172.21.15.5:0/2317643814 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63e803e050 con 0x55fc944ccec0 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f640beb5b80 10 client.25568 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:12.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc946cf660 con 0x55fc944ccec0 2024-05-14T15:27:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.008+0000 7f63de7fc700 1 -- 172.21.15.5:0/2317643814 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f0032070 con 0x55fc944ccec0 2024-05-14T15:27:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f63de7fc700 1 -- 172.21.15.5:0/2317643814 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f63e8034030 con 0x55fc944ccec0 2024-05-14T15:27:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f640beb5b80 10 client.25568 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f640beb5b80 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f640beb5b80 20 client.25568 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f640beb5b80 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc946cfcf0 0x55fc946d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f640beb5b80 4 client.25568 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc94667430 con 0x55fc946cfcf0 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.009+0000 7f63f67fc700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc946cfcf0 0x55fc946d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.010+0000 7f63f67fc700 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc946cfcf0 0x55fc946d2110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92cc18c2dfe475f server_cookie=5380a80f7931df3b in_seq=0 out_seq=0 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.010+0000 7f63de7fc700 10 client.25568 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f63de7fc700 1 -- 172.21.15.5:0/2317643814 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc94667430 con 0x55fc946cfcf0 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f63de7fc700 10 client.25568 handle_command_reply: tid=0 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 client.25568 shutdown 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 msgr2=0x55fc9466c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc9466c070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55fc94653d60 tx=0x7f63e8004b40 comp rx=0 tx=0).stop 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 shutdown_connections 2024-05-14T15:27:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fc946cfcf0 0x55fc946d2110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f63de7fc700 0 client.25568 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc94650e00 0x55fc9466caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc944ccec0 0x55fc9466c070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 --2- 172.21.15.5:0/2317643814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc9464ae90 0x55fc9466c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 >> 172.21.15.5:0/2317643814 conn(0x55fc945146b0 msgr2=0x55fc94654480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 shutdown_connections 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.011+0000 7f640beb5b80 1 -- 172.21.15.5:0/2317643814 wait complete. 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f640beb5b80 20 client.25568 trim_cache size 0 max 16384 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:27:12.015 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3174e95a-5548-49fa-a6ec-bf57e0cb1e39", 2024-05-14T15:27:12.016 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:27:12.016 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:27:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63e40b29f0 msgr2=0x7f63e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63e40b29f0 0x7f63e40b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009510 tx=0x7f63e8009230 comp rx=0 tx=0).stop 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 msgr2=0x7f63f819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 0x7f63f819b370 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f63e000e970 tx=0x7f63e000ec80 comp rx=0 tx=0).stop 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 shutdown_connections 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f63e40b29f0 0x7f63e40b4f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63f8109890 0x7f63f819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63f8104610 0x7f63f819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 --2- 172.21.15.5:0/73223530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8108f40 0x7f63f819b370 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.012+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 >> 172.21.15.5:0/73223530 conn(0x7f63f807af70 msgr2=0x7f63f81018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.013+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 shutdown_connections 2024-05-14T15:27:12.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:12.013+0000 7f63fd4a1700 1 -- 172.21.15.5:0/73223530 wait complete. 2024-05-14T15:27:12.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 78970 13.41 MB/sec warmup 33 sec latency 160.780 ms 2024-05-14T15:27:12.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:11 smithi144 ceph-mon[35650]: pgmap v74: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-05-14T15:27:12.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:11 smithi193 ceph-mon[36001]: pgmap v74: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-05-14T15:27:12.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:11 smithi005 ceph-mon[31559]: pgmap v74: 129 pgs: 129 active+clean; 46 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 569 op/s 2024-05-14T15:27:13.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 81926 13.54 MB/sec warmup 34 sec latency 15.212 ms 2024-05-14T15:27:13.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:12 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:27:13.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:12 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:13.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:12 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:27:13.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:12 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:13.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:12 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:27:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:12 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:12 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:27:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:12 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:13.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:12 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:27:13.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:12 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:13.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:12 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:27:13.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:12 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:13.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:27:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:14.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:13 smithi005 ceph-mon[31559]: pgmap v75: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 603 op/s 2024-05-14T15:27:14.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 84403 13.48 MB/sec warmup 35 sec latency 17.921 ms 2024-05-14T15:27:14.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:13 smithi144 ceph-mon[35650]: pgmap v75: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 603 op/s 2024-05-14T15:27:14.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:13 smithi193 ceph-mon[36001]: pgmap v75: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 603 op/s 2024-05-14T15:27:15.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 86300 13.39 MB/sec warmup 36 sec latency 19.115 ms 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 -- 172.21.15.5:0/473380675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 msgr2=0x7f1440102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 --2- 172.21.15.5:0/473380675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f1440102bf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f142c009990 tx=0x7f142c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 -- 172.21.15.5:0/473380675 shutdown_connections 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 --2- 172.21.15.5:0/473380675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1440103130 0x7f1440111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 --2- 172.21.15.5:0/473380675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f144010a3e0 0x7f144010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 --2- 172.21.15.5:0/473380675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f1440102bf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 -- 172.21.15.5:0/473380675 >> 172.21.15.5:0/473380675 conn(0x7f14400fe6f0 msgr2=0x7f1440100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 -- 172.21.15.5:0/473380675 shutdown_connections 2024-05-14T15:27:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.523+0000 7f1447165700 1 -- 172.21.15.5:0/473380675 wait complete. 2024-05-14T15:27:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 Processor -- start 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 -- start start 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f144019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1440103130 0x7f144019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144010a3e0 0x7f144019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1440114460 con 0x7f14401027e0 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14401142e0 con 0x7f144010a3e0 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.524+0000 7f1447165700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14401145e0 con 0x7f1440103130 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f1444f01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f144019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f1444f01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f144019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53630/0 (socket says 172.21.15.5:53630) 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f1444f01700 1 -- 172.21.15.5:0/4028255849 learned_addr learned my addr 172.21.15.5:0/4028255849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1440103130 0x7f144019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f1445702700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144010a3e0 0x7f144019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 -- 172.21.15.5:0/4028255849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144010a3e0 msgr2=0x7f144019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144010a3e0 0x7f144019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 -- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 msgr2=0x7f144019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f144019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 -- 172.21.15.5:0/4028255849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144019fd00 con 0x7f1440103130 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f1444f01700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f144019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143ffff700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1440103130 0x7f144019b3d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f142c0098e0 tx=0x7f142c00bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143dffb700 1 -- 172.21.15.5:0/4028255849 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c027070 con 0x7f1440103130 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144019ff90 con 0x7f1440103130 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.525+0000 7f143dffb700 1 -- 172.21.15.5:0/4028255849 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142c022470 con 0x7f1440103130 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.526+0000 7f143dffb700 1 -- 172.21.15.5:0/4028255849 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c017650 con 0x7f1440103130 2024-05-14T15:27:15.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.526+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401a0440 con 0x7f1440103130 2024-05-14T15:27:15.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.528+0000 7f143dffb700 1 -- 172.21.15.5:0/4028255849 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f142c02a070 con 0x7f1440103130 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.529+0000 7f143dffb700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f14300b29f0 0x7f14300b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.529+0000 7f1444f01700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f14300b29f0 0x7f14300b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.529+0000 7f143dffb700 1 -- 172.21.15.5:0/4028255849 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f142c0e38b0 con 0x7f1440103130 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.529+0000 7f1455b79b80 1 Processor -- start 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 -- start start 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa1248f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 0x55aa124898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa12489e10 0x55aa1248a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1444f01700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f14300b29f0 0x7f14300b4f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1434011550 tx=0x7f1434009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aa123ab940 con 0x55aa1248fd80 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aa12310da0 con 0x55aa12489e10 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1455b79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aa123a7900 con 0x55aa1230be40 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f143ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 0x55aa124898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f143ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 0x55aa124898d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53636/0 (socket says 172.21.15.5:53636) 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f143ffff700 1 -- 172.21.15.5:0/365836879 learned_addr learned my addr 172.21.15.5:0/365836879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1444f01700 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa1248f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1445702700 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa12489e10 0x55aa1248a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1445702700 1 -- 172.21.15.5:0/365836879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 msgr2=0x55aa1248f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1445702700 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa1248f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1445702700 1 -- 172.21.15.5:0/365836879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 msgr2=0x55aa124898d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1445702700 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 0x55aa124898d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f1445702700 1 -- 172.21.15.5:0/365836879 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa122f88d0 con 0x55aa12489e10 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.530+0000 7f143ffff700 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 0x55aa124898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.531+0000 7f1445702700 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa12489e10 0x55aa1248a220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1438003e90 tx=0x7f1438009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5fb0ff833c1e141 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.531+0000 7f142affd700 1 -- 172.21.15.5:0/365836879 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438004bb0 con 0x55aa12489e10 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.531+0000 7f142affd700 1 -- 172.21.15.5:0/365836879 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14380048a0 con 0x55aa12489e10 2024-05-14T15:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.531+0000 7f142affd700 1 -- 172.21.15.5:0/365836879 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438007cb0 con 0x55aa12489e10 2024-05-14T15:27:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 -- 172.21.15.5:0/365836879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa12489e10 msgr2=0x55aa1248a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa12489e10 0x55aa1248a220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1438003e90 tx=0x7f1438009c20 comp rx=0 tx=0).stop 2024-05-14T15:27:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 -- 172.21.15.5:0/365836879 shutdown_connections 2024-05-14T15:27:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa12489e10 0x55aa1248a220 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa1248f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 --2- 172.21.15.5:0/365836879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1248fd80 0x55aa124898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 -- 172.21.15.5:0/365836879 >> 172.21.15.5:0/365836879 conn(0x55aa12353630 msgr2=0x55aa12484a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 -- 172.21.15.5:0/365836879 shutdown_connections 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 -- 172.21.15.5:0/365836879 wait complete. 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 -- start start 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.532+0000 7f1455b79b80 1 Processor -- start 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa12493410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa12493950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aa1249ac30 con 0x55aa12489e10 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aa12364ed0 con 0x55aa1248fd80 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aa1249adb0 con 0x55aa1230be40 2024-05-14T15:27:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1455b79b80 10 client.0 _ll_register_callbacks cb 0x55aa12343950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f143ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa12493950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1444f01700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa12493410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f143ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa12493950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53644/0 (socket says 172.21.15.5:53644) 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f143ffff700 1 -- 172.21.15.5:0/2682520345 learned_addr learned my addr 172.21.15.5:0/2682520345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1445702700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1444f01700 1 -- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 msgr2=0x55aa12493e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1444f01700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1444f01700 1 -- 172.21.15.5:0/2682520345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 msgr2=0x55aa12493950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1444f01700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa12493950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1444f01700 1 -- 172.21.15.5:0/2682520345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa12362760 con 0x55aa1230be40 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f143ffff700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa12493950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.533+0000 7f1445702700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f1444f01700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa12493410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1434011ca0 tx=0x7f14340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8dd1df2766d9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f141ffff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f141ffff700 1 -- 172.21.15.5:0/2682520345 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143400ee10 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f141ffff700 1 -- 172.21.15.5:0/2682520345 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f143401a5b0 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14380048a0 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f141ffff700 1 -- 172.21.15.5:0/2682520345 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f143401a7b0 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f1455b79b80 10 client.25597 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa124a57e0 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f141ffff700 1 -- 172.21.15.5:0/2682520345 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14380048a0 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.534+0000 7f141ffff700 1 -- 172.21.15.5:0/2682520345 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f1434026070 con 0x55aa1230be40 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f1455b79b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f1455b79b80 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f1455b79b80 20 client.25597 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f1455b79b80 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa124a7ec0 0x55aa124aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f1455b79b80 4 client.25597 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa123b9320 con 0x55aa124a7ec0 2024-05-14T15:27:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f143ffff700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa124a7ec0 0x55aa124aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f143ffff700 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa124a7ec0 0x55aa124aa350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536ea4313c0226fd server_cookie=ac950990a6fc737c in_seq=0 out_seq=0 2024-05-14T15:27:15.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.535+0000 7f141ffff700 10 client.25597 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f141ffff700 1 -- 172.21.15.5:0/2682520345 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa123b9320 con 0x55aa124a7ec0 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f141ffff700 10 client.25597 handle_command_reply: tid=0 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 client.25597 shutdown 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 msgr2=0x55aa12493410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa12493410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1434011ca0 tx=0x7f14340047c0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 shutdown_connections 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa124a7ec0 0x55aa124aa350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa12493410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f141ffff700 0 client.25597 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 --2- 172.21.15.5:0/2682520345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa12493950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 >> 172.21.15.5:0/2682520345 conn(0x55aa12353630 msgr2=0x55aa124888c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 shutdown_connections 2024-05-14T15:27:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 1 -- 172.21.15.5:0/2682520345 wait complete. 2024-05-14T15:27:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.537+0000 7f1455b79b80 20 client.25597 trim_cache size 0 max 16384 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 Processor -- start 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 -- start start 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 0x55aa124898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa12489e10 0x55aa12492730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14401b0c20 con 0x55aa1230be40 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14401ae4b0 con 0x55aa1248fd80 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1455b79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14401ac300 con 0x55aa12489e10 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 0x55aa124898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 0x55aa124898d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53656/0 (socket says 172.21.15.5:53656) 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 -- 172.21.15.5:0/621094019 learned_addr learned my addr 172.21.15.5:0/621094019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 -- 172.21.15.5:0/621094019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa12489e10 msgr2=0x55aa12492730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa12489e10 0x55aa12492730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 -- 172.21.15.5:0/621094019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 msgr2=0x55aa12493cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 -- 172.21.15.5:0/621094019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1248f3a0 con 0x55aa1230be40 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.558+0000 7f1444f01700 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 0x55aa124898d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1434016040 tx=0x7f1434028f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a95eefa68a3ae05 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.559+0000 7f142a7fc700 1 -- 172.21.15.5:0/621094019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143403b070 con 0x55aa1230be40 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.559+0000 7f142a7fc700 1 -- 172.21.15.5:0/621094019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f143400ee10 con 0x55aa1230be40 2024-05-14T15:27:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.559+0000 7f142a7fc700 1 -- 172.21.15.5:0/621094019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434036480 con 0x55aa1230be40 2024-05-14T15:27:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.559+0000 7f1455b79b80 1 -- 172.21.15.5:0/621094019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 msgr2=0x55aa124898d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.559+0000 7f1455b79b80 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 0x55aa124898d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1434016040 tx=0x7f1434028f20 comp rx=0 tx=0).stop 2024-05-14T15:27:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 -- 172.21.15.5:0/621094019 shutdown_connections 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa12493cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa1230be40 0x55aa124898d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 --2- 172.21.15.5:0/621094019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa12489e10 0x55aa12492730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 -- 172.21.15.5:0/621094019 >> 172.21.15.5:0/621094019 conn(0x55aa12353630 msgr2=0x55aa1248e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 -- 172.21.15.5:0/621094019 shutdown_connections 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 -- 172.21.15.5:0/621094019 wait complete. 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 -- start start 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.560+0000 7f1455b79b80 1 Processor -- start 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa124a0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa124af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa124af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14401ac970 con 0x55aa12489e10 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14401adde0 con 0x55aa1248fd80 2024-05-14T15:27:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14401ad040 con 0x55aa1230be40 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1455b79b80 10 client.0 _ll_register_callbacks cb 0x55aa12343950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa124af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa124af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53670/0 (socket says 172.21.15.5:53670) 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1445702700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa124af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 -- 172.21.15.5:0/1110321856 learned_addr learned my addr 172.21.15.5:0/1110321856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f1444f01700 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa124a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 -- 172.21.15.5:0/1110321856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 msgr2=0x55aa124a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa124a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 -- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 msgr2=0x55aa124af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa124af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 -- 172.21.15.5:0/1110321856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa124afe60 con 0x55aa12489e10 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f143ffff700 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa124af120 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f142c009960 tx=0x7f142c0b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d341db5a444c20d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.561+0000 7f141ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f141ffff700 1 -- 172.21.15.5:0/1110321856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c0c4070 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f141ffff700 1 -- 172.21.15.5:0/1110321856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f142c0b10b0 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143400ee10 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f141ffff700 1 -- 172.21.15.5:0/1110321856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f142c0cd0b0 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 10 client.15942 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa12512700 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f141ffff700 1 -- 172.21.15.5:0/1110321856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f143400ee10 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f141ffff700 1 -- 172.21.15.5:0/1110321856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f142c0aaa60 con 0x55aa12489e10 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 20 client.15942 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa12512db0 0x55aa125151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 4 client.15942 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa123b33c0 con 0x55aa12512db0 2024-05-14T15:27:15.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.562+0000 7f1444f01700 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa12512db0 0x55aa125151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f141ffff700 10 client.15942 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1444f01700 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa12512db0 0x55aa125151d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=540a53e093e97d32 server_cookie=b853d22bacca0efc in_seq=0 out_seq=0 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f141ffff700 1 -- 172.21.15.5:0/1110321856 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aa123b33c0 con 0x55aa12512db0 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f141ffff700 10 client.15942 handle_command_reply: tid=0 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 client.15942 shutdown 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 msgr2=0x55aa124af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa124af120 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f142c009960 tx=0x7f142c0b1040 comp rx=0 tx=0).stop 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 shutdown_connections 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aa12512db0 0x55aa125151d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa1248fd80 0x55aa124af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa1230be40 0x55aa124a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 --2- 172.21.15.5:0/1110321856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa12489e10 0x55aa124af120 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 >> 172.21.15.5:0/1110321856 conn(0x55aa12353630 msgr2=0x55aa1248e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f141ffff700 0 client.15942 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:15.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.563+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 shutdown_connections 2024-05-14T15:27:15.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1455b79b80 1 -- 172.21.15.5:0/1110321856 wait complete. 2024-05-14T15:27:15.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1455b79b80 20 client.15942 trim_cache size 0 max 16384 2024-05-14T15:27:15.569 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:27:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:27:15.569 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:27:15.569 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f14300b29f0 msgr2=0x7f14300b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f14300b29f0 0x7f14300b4f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1434011550 tx=0x7f1434009260 comp rx=0 tx=0).stop 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1440103130 msgr2=0x7f144019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1440103130 0x7f144019b3d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f142c0098e0 tx=0x7f142c00bfc0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 shutdown_connections 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f14300b29f0 0x7f14300b4f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f144010a3e0 0x7f144019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14401027e0 0x7f144019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 --2- 172.21.15.5:0/4028255849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1440103130 0x7f144019b3d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 >> 172.21.15.5:0/4028255849 conn(0x7f14400fe6f0 msgr2=0x7f1440100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 shutdown_connections 2024-05-14T15:27:15.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:15.564+0000 7f1447165700 1 -- 172.21.15.5:0/4028255849 wait complete. 2024-05-14T15:27:16.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 89118 13.49 MB/sec warmup 37 sec latency 21.580 ms 2024-05-14T15:27:16.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:15 smithi005 ceph-mon[31559]: pgmap v76: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 603 op/s 2024-05-14T15:27:16.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:15 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:27:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:15 smithi144 ceph-mon[35650]: pgmap v76: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 603 op/s 2024-05-14T15:27:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:15 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:27:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:15 smithi193 ceph-mon[36001]: pgmap v76: 129 pgs: 129 active+clean; 32 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 603 op/s 2024-05-14T15:27:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:15 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:27:16.665 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:27:16.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:27:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:17.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 91596 13.43 MB/sec warmup 38 sec latency 40.233 ms 2024-05-14T15:27:18.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:17 smithi005 ceph-mon[31559]: pgmap v77: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-05-14T15:27:18.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 93439 13.34 MB/sec warmup 39 sec latency 14.157 ms 2024-05-14T15:27:18.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:17 smithi144 ceph-mon[35650]: pgmap v77: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-05-14T15:27:18.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:17 smithi193 ceph-mon[36001]: pgmap v77: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-05-14T15:27:18.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.823+0000 7fabf4d93700 1 -- 172.21.15.5:0/3902628576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf01039a0 msgr2=0x7fabf0103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.823+0000 7fabf4d93700 1 --2- 172.21.15.5:0/3902628576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf01039a0 0x7fabf0103db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009990 tx=0x7fabd8009ca0 comp rx=0 tx=0).stop 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 -- 172.21.15.5:0/3902628576 shutdown_connections 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 --2- 172.21.15.5:0/3902628576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf01042f0 0x7fabf0110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 --2- 172.21.15.5:0/3902628576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf0108200 0x7fabf0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 --2- 172.21.15.5:0/3902628576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf01039a0 0x7fabf0103db0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 -- 172.21.15.5:0/3902628576 >> 172.21.15.5:0/3902628576 conn(0x7fabf00fe520 msgr2=0x7fabf0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 -- 172.21.15.5:0/3902628576 shutdown_connections 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 -- 172.21.15.5:0/3902628576 wait complete. 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.824+0000 7fabf4d93700 1 Processor -- start 2024-05-14T15:27:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 -- start start 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf01039a0 0x7fabf00fea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf01042f0 0x7fabf00fef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 0x7fabf00ff480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0113380 con 0x7fabf0108200 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0113200 con 0x7fabf01039a0 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabf4d93700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0113500 con 0x7fabf01042f0 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabeed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 0x7fabf00ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabee59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf01039a0 0x7fabf00fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabeed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 0x7fabf00ff480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51222/0 (socket says 172.21.15.5:51222) 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabeed9d700 1 -- 172.21.15.5:0/2340326246 learned_addr learned my addr 172.21.15.5:0/2340326246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabedd9b700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf01042f0 0x7fabf00fef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabeed9d700 1 -- 172.21.15.5:0/2340326246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf01042f0 msgr2=0x7fabf00fef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabeed9d700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf01042f0 0x7fabf00fef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.825+0000 7fabeed9d700 1 -- 172.21.15.5:0/2340326246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf01039a0 msgr2=0x7fabf00fea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabeed9d700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf01039a0 0x7fabf00fea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabeed9d700 1 -- 172.21.15.5:0/2340326246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf01a9140 con 0x7fabf0108200 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabeed9d700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 0x7fabf00ff480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fabe8007ee0 tx=0x7fabe800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8004bb0 con 0x7fabf0108200 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe8004d50 con 0x7fabf0108200 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8005920 con 0x7fabf0108200 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf01a9430 con 0x7fabf0108200 2024-05-14T15:27:18.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.826+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf01a98f0 con 0x7fabf0108200 2024-05-14T15:27:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.827+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf00512c0 con 0x7fabf0108200 2024-05-14T15:27:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.829+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fabe801a020 con 0x7fabf0108200 2024-05-14T15:27:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.829+0000 7fabe77fe700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fabdc0b29f0 0x7fabdc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:18.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.829+0000 7fabee59c700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fabdc0b29f0 0x7fabdc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.830+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7fabe80e2090 con 0x7fabf0108200 2024-05-14T15:27:18.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.830+0000 7fabee59c700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fabdc0b29f0 0x7fabdc0b4f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fabf006c2f0 tx=0x7fabe000a380 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:18.834+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabe8010070 con 0x7fabf0108200 2024-05-14T15:27:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 -- 172.21.15.5:0/2637792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260104e30 msgr2=0x7f1260105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 --2- 172.21.15.5:0/2637792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260104e30 0x7f1260105240 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1250009900 tx=0x7f125002a510 comp rx=0 tx=0).stop 2024-05-14T15:27:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 -- 172.21.15.5:0/2637792 shutdown_connections 2024-05-14T15:27:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 --2- 172.21.15.5:0/2637792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f126010b890 0x7f126010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 --2- 172.21.15.5:0/2637792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f126010e240 0x7f1260110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 --2- 172.21.15.5:0/2637792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260104e30 0x7f1260105240 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 -- 172.21.15.5:0/2637792 >> 172.21.15.5:0/2637792 conn(0x7f12600782d0 msgr2=0x7f1260078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.047+0000 7f12655ba640 1 -- 172.21.15.5:0/2637792 shutdown_connections 2024-05-14T15:27:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.048+0000 7f12655ba640 1 -- 172.21.15.5:0/2637792 wait complete. 2024-05-14T15:27:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.048+0000 7f12655ba640 1 Processor -- start 2024-05-14T15:27:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.048+0000 7f12655ba640 1 -- start start 2024-05-14T15:27:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f12655ba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1260104e30 0x7f12601a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f12655ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 0x7f12601a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f12655ba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f126010e240 0x7f126019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f12655ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1260113ea0 con 0x7f126010b890 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f12655ba640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1260113d20 con 0x7f1260104e30 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f12655ba640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1260114020 con 0x7f126010e240 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 0x7f12601a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 0x7f12601a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51240/0 (socket says 172.21.15.5:51240) 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 -- 172.21.15.5:0/253382165 learned_addr learned my addr 172.21.15.5:0/253382165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 -- 172.21.15.5:0/253382165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f126010e240 msgr2=0x7f126019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f126010e240 0x7f126019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 -- 172.21.15.5:0/253382165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1260104e30 msgr2=0x7f12601a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1260104e30 0x7f12601a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 -- 172.21.15.5:0/253382165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126019ae70 con 0x7f126010b890 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.049+0000 7f125e7fc640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 0x7f12601a0b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f125400c970 tx=0x7f125400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.050+0000 7f1247fff640 1 -- 172.21.15.5:0/253382165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254007bb0 con 0x7f126010b890 2024-05-14T15:27:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.050+0000 7f1247fff640 1 -- 172.21.15.5:0/253382165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1254007d50 con 0x7f126010b890 2024-05-14T15:27:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.050+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126019b100 con 0x7f126010b890 2024-05-14T15:27:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.050+0000 7f1247fff640 1 -- 172.21.15.5:0/253382165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125400f450 con 0x7f126010b890 2024-05-14T15:27:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.050+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12601a7360 con 0x7f126010b890 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f1247fff640 1 -- 172.21.15.5:0/253382165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1254004f20 con 0x7f126010b890 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 Processor -- start 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 -- start start 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 0x56513a3c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0034d0 0x56513a3c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56513a32a5d0 con 0x56513a0241b0 2024-05-14T15:27:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56513a2e3890 con 0x565139ffb2f0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f126c46a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56513a326590 con 0x56513a0034d0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f125f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 0x56513a3c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f125f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 0x56513a3c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51242/0 (socket says 172.21.15.5:51242) 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f125f7fe640 1 -- 172.21.15.5:0/3010821995 learned_addr learned my addr 172.21.15.5:0/3010821995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f125e7fc640 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0034d0 0x56513a3c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.053+0000 7f125effd640 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125f7fe640 1 -- 172.21.15.5:0/3010821995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0034d0 msgr2=0x56513a3c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125f7fe640 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0034d0 0x56513a3c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125f7fe640 1 -- 172.21.15.5:0/3010821995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 msgr2=0x56513a3c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125f7fe640 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125f7fe640 1 -- 172.21.15.5:0/3010821995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56513a03c210 con 0x56513a0241b0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f1247fff640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12480b2950 0x7f12480b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125f7fe640 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 0x56513a3c4be0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1258009d50 tx=0x7f125800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6315dcb4d78753 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f1247fff640 1 -- 172.21.15.5:0/253382165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f12540e2790 con 0x7f126010b890 2024-05-14T15:27:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f125effd640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12480b2950 0x7f12480b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f12457fa640 1 -- 172.21.15.5:0/3010821995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258003970 con 0x56513a0241b0 2024-05-14T15:27:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f12457fa640 1 -- 172.21.15.5:0/3010821995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258004020 con 0x56513a0241b0 2024-05-14T15:27:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.054+0000 7f12457fa640 1 -- 172.21.15.5:0/3010821995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12580106c0 con 0x56513a0241b0 2024-05-14T15:27:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 -- 172.21.15.5:0/3010821995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 msgr2=0x56513a3c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 0x56513a3c4be0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1258009d50 tx=0x7f125800d6b0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f125effd640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12480b2950 0x7f12480b4e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x56513a3c2320 tx=0x7f1250031400 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 -- 172.21.15.5:0/3010821995 shutdown_connections 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0034d0 0x56513a3c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 --2- 172.21.15.5:0/3010821995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0241b0 0x56513a3c4be0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 -- 172.21.15.5:0/3010821995 >> 172.21.15.5:0/3010821995 conn(0x56513a018300 msgr2=0x56513a3c1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 -- 172.21.15.5:0/3010821995 shutdown_connections 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 -- 172.21.15.5:0/3010821995 wait complete. 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.055+0000 7f126c46a740 1 -- start start 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 Processor -- start 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 0x56513a3cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3db1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56513a3d1d10 con 0x56513a0034d0 2024-05-14T15:27:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56513a2e7370 con 0x565139ffb2f0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56513a3d1e90 con 0x56513a0241b0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f126c46a740 10 client.0 _ll_register_callbacks cb 0x56513a03ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 0x56513a3cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 0x56513a3cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51244/0 (socket says 172.21.15.5:51244) 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 -- 172.21.15.5:0/54531507 learned_addr learned my addr 172.21.15.5:0/54531507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125effd640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125f7fe640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3db1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 -- 172.21.15.5:0/54531507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 msgr2=0x56513a3db1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3db1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 -- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 msgr2=0x56513a3cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125e7fc640 1 -- 172.21.15.5:0/54531507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56513a108620 con 0x56513a0034d0 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.056+0000 7f125effd640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f125e7fc640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 0x56513a3cf710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f125400c940 tx=0x7f12540cc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a188fabdca05134 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f1242ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f1242ff5640 1 -- 172.21.15.5:0/54531507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125401d070 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f1242ff5640 1 -- 172.21.15.5:0/54531507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f12540a5940 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258004020 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f1242ff5640 1 -- 172.21.15.5:0/54531507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f12540a8e20 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f126c46a740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56513a3de0b0 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f1242ff5640 1 -- 172.21.15.5:0/54531507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1258004020 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.057+0000 7f1242ff5640 1 -- 172.21.15.5:0/54531507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f12540910b0 con 0x56513a0034d0 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f126c46a740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f126c46a740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f126c46a740 20 client.15960 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f126c46a740 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fe2d80 0x56513a3e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f126c46a740 4 client.15960 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56513a3381e0 con 0x565139fe2d80 2024-05-14T15:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f125effd640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fe2d80 0x56513a3e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f125effd640 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fe2d80 0x56513a3e1530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f45523be3470d8 server_cookie=5bdcc91208b7ba07 in_seq=0 out_seq=0 2024-05-14T15:27:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.058+0000 7f1242ff5640 10 client.15960 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f1242ff5640 1 -- 172.21.15.5:0/54531507 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56513a3381e0 con 0x565139fe2d80 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f1242ff5640 10 client.15960 handle_command_reply: tid=0 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 client.15960 shutdown 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 msgr2=0x56513a3cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 0x56513a3cf710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f125400c940 tx=0x7f12540cc110 comp rx=0 tx=0).stop 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 shutdown_connections 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fe2d80 0x56513a3e1530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3db1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56513a0034d0 0x56513a3cf710 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 --2- 172.21.15.5:0/54531507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565139ffb2f0 0x56513a3cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 >> 172.21.15.5:0/54531507 conn(0x56513a018300 msgr2=0x56513a3c4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f1242ff5640 0 client.15960 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 shutdown_connections 2024-05-14T15:27:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 1 -- 172.21.15.5:0/54531507 wait complete. 2024-05-14T15:27:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.061+0000 7f126c46a740 20 client.15960 trim_cache size 0 max 16384 2024-05-14T15:27:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.077+0000 7f126c46a740 1 Processor -- start 2024-05-14T15:27:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.077+0000 7f126c46a740 1 -- start start 2024-05-14T15:27:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.077+0000 7f126c46a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.077+0000 7f126c46a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f126c46a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f126c46a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12601b3d20 con 0x565139ffb2f0 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f126c46a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12601b15f0 con 0x56513a0034d0 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f126c46a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12601af440 con 0x56513a0241b0 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51250/0 (socket says 172.21.15.5:51250) 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 -- 172.21.15.5:0/2444886732 learned_addr learned my addr 172.21.15.5:0/2444886732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125e7fc640 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125f7fe640 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 -- 172.21.15.5:0/2444886732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 msgr2=0x56513a3e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 -- 172.21.15.5:0/2444886732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 msgr2=0x56513a3cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 -- 172.21.15.5:0/2444886732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565139fe2fb0 con 0x565139ffb2f0 2024-05-14T15:27:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f125effd640 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3c98b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f125002aa30 tx=0x7f1250040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d0cd59df737faa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.078+0000 7f1244ff9640 1 -- 172.21.15.5:0/2444886732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250035040 con 0x565139ffb2f0 2024-05-14T15:27:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.079+0000 7f1244ff9640 1 -- 172.21.15.5:0/2444886732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1250056070 con 0x565139ffb2f0 2024-05-14T15:27:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.079+0000 7f1244ff9640 1 -- 172.21.15.5:0/2444886732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250003ab0 con 0x565139ffb2f0 2024-05-14T15:27:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 -- 172.21.15.5:0/2444886732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 msgr2=0x56513a3c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3c98b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f125002aa30 tx=0x7f1250040000 comp rx=0 tx=0).stop 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 -- 172.21.15.5:0/2444886732 shutdown_connections 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a3e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 --2- 172.21.15.5:0/2444886732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3c98b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 -- 172.21.15.5:0/2444886732 >> 172.21.15.5:0/2444886732 conn(0x56513a018300 msgr2=0x56513a3ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 -- 172.21.15.5:0/2444886732 shutdown_connections 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 -- 172.21.15.5:0/2444886732 wait complete. 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.080+0000 7f126c46a740 1 -- start start 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 Processor -- start 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a447840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12601afab0 con 0x565139ffb2f0 2024-05-14T15:27:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12601b0f20 con 0x56513a0034d0 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12601b0180 con 0x56513a0241b0 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f126c46a740 10 client.0 _ll_register_callbacks cb 0x7f12540058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51266/0 (socket says 172.21.15.5:51266) 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 -- 172.21.15.5:0/659639626 learned_addr learned my addr 172.21.15.5:0/659639626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125f7fe640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a447840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 -- 172.21.15.5:0/659639626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 msgr2=0x56513a447840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a447840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 -- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 msgr2=0x56513a3d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125effd640 1 -- 172.21.15.5:0/659639626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56513a447d80 con 0x565139ffb2f0 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.081+0000 7f125f7fe640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a447840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f125effd640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3d24d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f12500057c0 tx=0x7f1250000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7faeeb136d7ea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f1242ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f1242ff5640 1 -- 172.21.15.5:0/659639626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250003c40 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f1242ff5640 1 -- 172.21.15.5:0/659639626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f1250003de0 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1250056070 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f1242ff5640 1 -- 172.21.15.5:0/659639626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f125006b050 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f126c46a740 10 client.15966 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56513a448ff0 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.082+0000 7f1242ff5640 1 -- 172.21.15.5:0/659639626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1250056070 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f1242ff5640 1 -- 172.21.15.5:0/659639626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f1250051ce0 con 0x565139ffb2f0 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f126c46a740 10 client.15966 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f126c46a740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f126c46a740 20 client.15966 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f126c46a740 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fb81c0 0x56513a44b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f126c46a740 4 client.15966 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:27:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56513a332050 con 0x565139fb81c0 2024-05-14T15:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.083+0000 7f125e7fc640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fb81c0 0x56513a44b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.084+0000 7f1242ff5640 10 client.15966 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.084+0000 7f125e7fc640 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fb81c0 0x56513a44b7a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5dc292adaeea06 server_cookie=b7f54a71b202a4d7 in_seq=0 out_seq=0 2024-05-14T15:27:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.114+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabf002e690 con 0x7fabf0108200 2024-05-14T15:27:19.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:18 smithi005 ceph-mon[31559]: pgmap v78: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-05-14T15:27:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.115+0000 7fabe77fe700 1 -- 172.21.15.5:0/2340326246 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11864 (secure 0 0 0) 0x7fabe80a96b0 con 0x7fabf0108200 2024-05-14T15:27:19.117 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:27:19.118 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:27:06.941843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25535":{"gid":25535,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1884688714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1884688714},{"type":"v1","addr":"172.21.15.144:6835","nonce":1884688714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fabdc0b29f0 msgr2=0x7fabdc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fabdc0b29f0 0x7fabdc0b4f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fabf006c2f0 tx=0x7fabe000a380 comp rx=0 tx=0).stop 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 msgr2=0x7fabf00ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 0x7fabf00ff480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fabe8007ee0 tx=0x7fabe800d3b0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 shutdown_connections 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fabdc0b29f0 0x7fabdc0b4f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0108200 0x7fabf00ff480 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf01039a0 0x7fabf00fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 --2- 172.21.15.5:0/2340326246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf01042f0 0x7fabf00fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 >> 172.21.15.5:0/2340326246 conn(0x7fabf00fe520 msgr2=0x7fabf0102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 shutdown_connections 2024-05-14T15:27:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:19.118+0000 7fabf4d93700 1 -- 172.21.15.5:0/2340326246 wait complete. 2024-05-14T15:27:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-14T15:27:19.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 95925 13.40 MB/sec warmup 40 sec latency 211.634 ms 2024-05-14T15:27:19.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:18 smithi144 ceph-mon[35650]: pgmap v78: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-05-14T15:27:19.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:18 smithi193 ceph-mon[36001]: pgmap v78: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-05-14T15:27:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f1242ff5640 1 -- 172.21.15.5:0/659639626 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56513a332050 con 0x565139fb81c0 2024-05-14T15:27:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f1242ff5640 10 client.15966 handle_command_reply: tid=0 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 client.15966 shutdown 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 msgr2=0x56513a3d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3d24d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f12500057c0 tx=0x7f1250000bc0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 shutdown_connections 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x565139fb81c0 0x56513a44b7a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56513a0241b0 0x56513a447840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56513a0034d0 0x56513a3d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 --2- 172.21.15.5:0/659639626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565139ffb2f0 0x56513a3d24d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 >> 172.21.15.5:0/659639626 conn(0x56513a018300 msgr2=0x56513a3c4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f1242ff5640 0 client.15966 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 shutdown_connections 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 1 -- 172.21.15.5:0/659639626 wait complete. 2024-05-14T15:27:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.324+0000 7f126c46a740 20 client.15966 trim_cache size 0 max 16384 2024-05-14T15:27:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:27:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:27:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:27:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:27:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12480b2950 msgr2=0x7f12480b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12480b2950 0x7f12480b4e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x56513a3c2320 tx=0x7f1250031400 comp rx=0 tx=0).stop 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 msgr2=0x7f12601a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 0x7f12601a0b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f125400c970 tx=0x7f125400ce30 comp rx=0 tx=0).stop 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 shutdown_connections 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12480b2950 0x7f12480b4e60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f126010e240 0x7f126019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f126010b890 0x7f12601a0b00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 --2- 172.21.15.5:0/253382165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1260104e30 0x7f12601a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 >> 172.21.15.5:0/253382165 conn(0x7f12600782d0 msgr2=0x7f12601012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 shutdown_connections 2024-05-14T15:27:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:19.325+0000 7f12655ba640 1 -- 172.21.15.5:0/253382165 wait complete. 2024-05-14T15:27:20.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 97511 13.23 MB/sec warmup 41 sec latency 103.355 ms 2024-05-14T15:27:20.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:19 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2340326246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:27:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:19 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2340326246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:27:20.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:19 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2340326246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:27:20.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:27:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:21.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 99914 13.25 MB/sec warmup 42 sec latency 21.661 ms 2024-05-14T15:27:21.177 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:20 smithi005 ceph-mon[31559]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-14T15:27:21.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:20 smithi144 ceph-mon[35650]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-14T15:27:21.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:20 smithi193 ceph-mon[36001]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-14T15:27:22.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 102713 13.28 MB/sec warmup 43 sec latency 20.244 ms 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 -- 172.21.15.5:0/488736216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 msgr2=0x7f0090108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 --2- 172.21.15.5:0/488736216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f0090108200 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f00780099c0 tx=0x7f0078009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 -- 172.21.15.5:0/488736216 shutdown_connections 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 --2- 172.21.15.5:0/488736216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0090108740 0x7f00901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 --2- 172.21.15.5:0/488736216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f00901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 --2- 172.21.15.5:0/488736216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f0090108200 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.421+0000 7f0095b56700 1 -- 172.21.15.5:0/488736216 >> 172.21.15.5:0/488736216 conn(0x7f00900fedf0 msgr2=0x7f0090101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.422+0000 7f0095b56700 1 -- 172.21.15.5:0/488736216 shutdown_connections 2024-05-14T15:27:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.422+0000 7f0095b56700 1 -- 172.21.15.5:0/488736216 wait complete. 2024-05-14T15:27:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.422+0000 7f0095b56700 1 Processor -- start 2024-05-14T15:27:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 -- start start 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f009019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f009019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0090108740 0x7f009019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0090113350 con 0x7f0090107df0 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00901131d0 con 0x7f00901034c0 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f0095b56700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00901134d0 con 0x7f0090108740 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f009019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f009019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f009019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42814/0 (socket says 172.21.15.5:42814) 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008f7fe700 1 -- 172.21.15.5:0/388769163 learned_addr learned my addr 172.21.15.5:0/388769163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008ffff700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0090108740 0x7f009019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008f7fe700 1 -- 172.21.15.5:0/388769163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0090108740 msgr2=0x7f009019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.423+0000 7f008f7fe700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0090108740 0x7f009019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008f7fe700 1 -- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 msgr2=0x7f009019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008f7fe700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f009019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008f7fe700 1 -- 172.21.15.5:0/388769163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00901a7060 con 0x7f00901034c0 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008effd700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f009019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008f7fe700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f009019bd10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f008000ef10 tx=0x7f008000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008cff9700 1 -- 172.21.15.5:0/388769163 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000c9c0 con 0x7f00901034c0 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00901a72f0 con 0x7f00901034c0 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008cff9700 1 -- 172.21.15.5:0/388769163 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0080019070 con 0x7f00901034c0 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f008cff9700 1 -- 172.21.15.5:0/388769163 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080014480 con 0x7f00901034c0 2024-05-14T15:27:22.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.424+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901a7800 con 0x7f00901034c0 2024-05-14T15:27:22.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.427+0000 7f008cff9700 1 -- 172.21.15.5:0/388769163 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0080012050 con 0x7f00901034c0 2024-05-14T15:27:22.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f008cff9700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f007c0b29f0 0x7f007c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f008effd700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f007c0b29f0 0x7f007c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f008cff9700 1 -- 172.21.15.5:0/388769163 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f00800e2990 con 0x7f00901034c0 2024-05-14T15:27:22.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 Processor -- start 2024-05-14T15:27:22.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 -- start start 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a25598840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a25593220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f008effd700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f007c0b29f0 0x7f007c0b4f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0078000c00 tx=0x7f007801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a254b4940 con 0x557a25414e40 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a25419da0 con 0x557a25592e10 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.428+0000 7f00a456ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a254b0900 con 0x557a25598d80 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a25598840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a25593220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a25598840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51288/0 (socket says 172.21.15.5:51288) 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 -- 172.21.15.5:0/3990459731 learned_addr learned my addr 172.21.15.5:0/3990459731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:22.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008effd700 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 -- 172.21.15.5:0/3990459731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 msgr2=0x557a255928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 -- 172.21.15.5:0/3990459731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 msgr2=0x557a25593220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a25593220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 -- 172.21.15.5:0/3990459731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a254018d0 con 0x557a25414e40 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008ffff700 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a25593220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f008f7fe700 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a25598840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f00800a9550 tx=0x7f00800a9710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b02320fae426f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f0085ffb700 1 -- 172.21.15.5:0/3990459731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800a9360 con 0x557a25414e40 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f0085ffb700 1 -- 172.21.15.5:0/3990459731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00800a9f50 con 0x557a25414e40 2024-05-14T15:27:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.429+0000 7f0085ffb700 1 -- 172.21.15.5:0/3990459731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800a9ab0 con 0x557a25414e40 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 -- 172.21.15.5:0/3990459731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 msgr2=0x557a25598840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a25598840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f00800a9550 tx=0x7f00800a9710 comp rx=0 tx=0).stop 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 -- 172.21.15.5:0/3990459731 shutdown_connections 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a25593220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a25598840 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3990459731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 -- 172.21.15.5:0/3990459731 >> 172.21.15.5:0/3990459731 conn(0x557a2545c630 msgr2=0x557a2558da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 -- 172.21.15.5:0/3990459731 shutdown_connections 2024-05-14T15:27:22.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.430+0000 7f00a456ab80 1 -- 172.21.15.5:0/3990459731 wait complete. 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 -- start start 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 Processor -- start 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a2559c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a2559cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a2559cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a2559f7c0 con 0x557a25414e40 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a2546ded0 con 0x557a25592e10 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a2559f940 con 0x557a25598d80 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.431+0000 7f00a456ab80 10 client.0 _ll_register_callbacks cb 0x557a2544c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a2559c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a2559c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51290/0 (socket says 172.21.15.5:51290) 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 -- 172.21.15.5:0/3562407951 learned_addr learned my addr 172.21.15.5:0/3562407951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:22.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008effd700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a2559cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008ffff700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a2559cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 -- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 msgr2=0x557a2559cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a2559cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 -- 172.21.15.5:0/3562407951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 msgr2=0x557a2559cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a2559cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 -- 172.21.15.5:0/3562407951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2546b760 con 0x557a25414e40 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008effd700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a2559cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f008f7fe700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a2559c570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f00800a97e0 tx=0x7f00800a5cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8f0c24aa92599 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f006effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f006effd700 1 -- 172.21.15.5:0/3562407951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800c3060 con 0x557a25414e40 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f006effd700 1 -- 172.21.15.5:0/3562407951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f00800b2430 con 0x557a25414e40 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.432+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00800a9f50 con 0x557a25414e40 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f006effd700 1 -- 172.21.15.5:0/3562407951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00800a9360 con 0x557a25414e40 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 10 client.15978 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a255aa420 con 0x557a25414e40 2024-05-14T15:27:22.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f006effd700 1 -- 172.21.15.5:0/3562407951 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00800a9f50 con 0x557a25414e40 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f006effd700 1 -- 172.21.15.5:0/3562407951 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f00800910d0 con 0x557a25414e40 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 10 client.15978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 20 client.15978 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a255ac8f0 0x557a255aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 4 client.15978 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a254c2320 con 0x557a255ac8f0 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.433+0000 7f008effd700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a255ac8f0 0x557a255aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.434+0000 7f006effd700 10 client.15978 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.434+0000 7f008effd700 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a255ac8f0 0x557a255aed10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8be41b742a3b6d4 server_cookie=5336b5350c8bdc46 in_seq=0 out_seq=0 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.434+0000 7f006effd700 1 -- 172.21.15.5:0/3562407951 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a254c2320 con 0x557a255ac8f0 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.434+0000 7f006effd700 10 client.15978 handle_command_reply: tid=0 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.434+0000 7f00a456ab80 1 client.15978 shutdown 2024-05-14T15:27:22.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 msgr2=0x557a2559c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a2559c570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f00800a97e0 tx=0x7f00800a5cf0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 shutdown_connections 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a2559cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25414e40 0x557a2559c570 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a255ac8f0 0x557a255aed10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 --2- 172.21.15.5:0/3562407951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25592e10 0x557a2559cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 >> 172.21.15.5:0/3562407951 conn(0x557a2545c630 msgr2=0x557a255918c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f006effd700 0 client.15978 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 shutdown_connections 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 1 -- 172.21.15.5:0/3562407951 wait complete. 2024-05-14T15:27:22.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.435+0000 7f00a456ab80 20 client.15978 trim_cache size 0 max 16384 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.450+0000 7f00a456ab80 1 Processor -- start 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.450+0000 7f00a456ab80 1 -- start start 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a2559a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f00a456ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b2f40 con 0x557a25592e10 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f00a456ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b07d0 con 0x557a25414e40 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f00a456ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00901ae620 con 0x557a25598d80 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a2559a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a2559a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51294/0 (socket says 172.21.15.5:51294) 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 -- 172.21.15.5:0/2730360778 learned_addr learned my addr 172.21.15.5:0/2730360778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008ffff700 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008f7fe700 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 -- 172.21.15.5:0/2730360778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 msgr2=0x557a255a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 -- 172.21.15.5:0/2730360778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 msgr2=0x557a255928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 -- 172.21.15.5:0/2730360778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2559f700 con 0x557a25592e10 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008ffff700 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.451+0000 7f008effd700 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a2559a1a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0090076b40 tx=0x7f007801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85833e1940b912b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00857fa700 1 -- 172.21.15.5:0/2730360778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078004ab0 con 0x557a25592e10 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00857fa700 1 -- 172.21.15.5:0/2730360778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0078033e50 con 0x557a25592e10 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00857fa700 1 -- 172.21.15.5:0/2730360778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007800f750 con 0x557a25592e10 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 -- 172.21.15.5:0/2730360778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 msgr2=0x557a2559a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a2559a1a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0090076b40 tx=0x7f007801f000 comp rx=0 tx=0).stop 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 -- 172.21.15.5:0/2730360778 shutdown_connections 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2730360778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a2559a1a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.452+0000 7f00a456ab80 1 -- 172.21.15.5:0/2730360778 >> 172.21.15.5:0/2730360778 conn(0x557a2545c630 msgr2=0x557a2559ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.453+0000 7f00a456ab80 1 -- 172.21.15.5:0/2730360778 shutdown_connections 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.453+0000 7f00a456ab80 1 -- 172.21.15.5:0/2730360778 wait complete. 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.453+0000 7f00a456ab80 1 -- start start 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.453+0000 7f00a456ab80 1 Processor -- start 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a255ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00901aec90 con 0x557a25592e10 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00901b0100 con 0x557a25414e40 2024-05-14T15:27:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00901af360 con 0x557a25598d80 2024-05-14T15:27:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f00a456ab80 10 client.0 _ll_register_callbacks cb 0x557a2544c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a255ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a255ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51300/0 (socket says 172.21.15.5:51300) 2024-05-14T15:27:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 -- 172.21.15.5:0/2223272129 learned_addr learned my addr 172.21.15.5:0/2223272129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 -- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 msgr2=0x557a255aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 -- 172.21.15.5:0/2223272129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 msgr2=0x557a255b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 -- 172.21.15.5:0/2223272129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a255af6d0 con 0x557a25592e10 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008f7fe700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008ffff700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f008effd700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a255ae900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0078009990 tx=0x7f007800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8153a73130beb75d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.454+0000 7f006effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f006effd700 1 -- 172.21.15.5:0/2223272129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078042060 con 0x557a25592e10 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f006effd700 1 -- 172.21.15.5:0/2223272129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f0078004ab0 con 0x557a25592e10 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0078033e50 con 0x557a25592e10 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f006effd700 1 -- 172.21.15.5:0/2223272129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0078049050 con 0x557a25592e10 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 10 client.15984 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a25615e50 con 0x557a25592e10 2024-05-14T15:27:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f006effd700 1 -- 172.21.15.5:0/2223272129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0078033e50 con 0x557a25592e10 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f006effd700 1 -- 172.21.15.5:0/2223272129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f007800f720 con 0x557a25592e10 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 10 client.15984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 20 client.15984 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a25616390 0x557a256187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 4 client.15984 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a254bc3c0 con 0x557a25616390 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.455+0000 7f008f7fe700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a25616390 0x557a256187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f006effd700 10 client.15984 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f008f7fe700 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a25616390 0x557a256187b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=222b3e41015be823 server_cookie=f452001715c39865 in_seq=0 out_seq=0 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f006effd700 1 -- 172.21.15.5:0/2223272129 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a254bc3c0 con 0x557a25616390 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f006effd700 10 client.15984 handle_command_reply: tid=0 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 client.15984 shutdown 2024-05-14T15:27:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 msgr2=0x557a255ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a255ae900 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0078009990 tx=0x7f007800bf00 comp rx=0 tx=0).stop 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 shutdown_connections 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a25598d80 0x557a255aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x557a25616390 0x557a256187b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a25414e40 0x557a255b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 --2- 172.21.15.5:0/2223272129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a25592e10 0x557a255ae900 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 >> 172.21.15.5:0/2223272129 conn(0x557a2545c630 msgr2=0x557a255a8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f006effd700 0 client.15984 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 shutdown_connections 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.456+0000 7f00a456ab80 1 -- 172.21.15.5:0/2223272129 wait complete. 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f00a456ab80 20 client.15984 trim_cache size 0 max 16384 2024-05-14T15:27:22.463 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f007c0b29f0 msgr2=0x7f007c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f007c0b29f0 0x7f007c0b4f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0078000c00 tx=0x7f007801a040 comp rx=0 tx=0).stop 2024-05-14T15:27:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 msgr2=0x7f009019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f009019bd10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f008000ef10 tx=0x7f008000c3a0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 shutdown_connections 2024-05-14T15:27:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f007c0b29f0 0x7f007c0b4f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0090108740 0x7f009019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00901034c0 0x7f009019bd10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 --2- 172.21.15.5:0/388769163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0090107df0 0x7f009019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 >> 172.21.15.5:0/388769163 conn(0x7f00900fedf0 msgr2=0x7f00901096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 shutdown_connections 2024-05-14T15:27:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:22.457+0000 7f0095b56700 1 -- 172.21.15.5:0/388769163 wait complete. 2024-05-14T15:27:23.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 104686 13.20 MB/sec warmup 44 sec latency 16.481 ms 2024-05-14T15:27:23.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:27:23.511 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:23.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:23 smithi005 ceph-mon[31559]: pgmap v80: 129 pgs: 129 active+clean; 32 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 442 op/s 2024-05-14T15:27:24.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:23 smithi144 ceph-mon[35650]: pgmap v80: 129 pgs: 129 active+clean; 32 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 442 op/s 2024-05-14T15:27:24.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:23 smithi193 ceph-mon[36001]: pgmap v80: 129 pgs: 129 active+clean; 32 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 442 op/s 2024-05-14T15:27:24.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 107117 13.22 MB/sec warmup 45 sec latency 15.552 ms 2024-05-14T15:27:25.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110095 13.31 MB/sec warmup 46 sec latency 17.155 ms 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 -- 172.21.15.5:0/761459272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 msgr2=0x7f1ef0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 --2- 172.21.15.5:0/761459272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef0109330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80099c0 tx=0x7f1ed8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 -- 172.21.15.5:0/761459272 shutdown_connections 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 --2- 172.21.15.5:0/761459272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 0x7f1ef01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 --2- 172.21.15.5:0/761459272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 0x7f1ef0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 --2- 172.21.15.5:0/761459272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef0109330 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 -- 172.21.15.5:0/761459272 >> 172.21.15.5:0/761459272 conn(0x7f1ef00fffe0 msgr2=0x7f1ef0102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 -- 172.21.15.5:0/761459272 shutdown_connections 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.304+0000 7f1ef50da700 1 -- 172.21.15.5:0/761459272 wait complete. 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.305+0000 7f1ef50da700 1 Processor -- start 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 -- start start 2024-05-14T15:27:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 0x7f1ef019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 0x7f1ef019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01144d0 con 0x7f1ef0108f20 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0114350 con 0x7f1ef0109870 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ef50da700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0114650 con 0x7f1ef01045f0 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ee659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 0x7f1ef019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ee659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51314/0 (socket says 172.21.15.5:51314) 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 0x7f1ef019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:47070/0 (socket says 172.21.15.5:47070) 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ee659c700 1 -- 172.21.15.5:0/3774218720 learned_addr learned my addr 172.21.15.5:0/3774218720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1eef59e700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 0x7f1ef019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ee659c700 1 -- 172.21.15.5:0/3774218720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 msgr2=0x7f1ef019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.306+0000 7f1ee659c700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 0x7f1ef019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1ee659c700 1 -- 172.21.15.5:0/3774218720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 msgr2=0x7f1ef019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1ee659c700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 0x7f1ef019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1ee659c700 1 -- 172.21.15.5:0/3774218720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef019fd30 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1eef59e700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 0x7f1ef019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1ee659c700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef019b400 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b580 tx=0x7f1ed8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1eecd99700 1 -- 172.21.15.5:0/3774218720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8009e60 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef019ffc0 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1eecd99700 1 -- 172.21.15.5:0/3774218720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed8003c70 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef01a0490 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.307+0000 7f1eecd99700 1 -- 172.21.15.5:0/3774218720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8004910 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.310+0000 7f1eecd99700 1 -- 172.21.15.5:0/3774218720 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1ed8004ab0 con 0x7f1ef0108f20 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.310+0000 7f1eecd99700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1edc0b29f0 0x7f1edc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 Processor -- start 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1edc0b29f0 0x7f1edc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eecd99700 1 -- 172.21.15.5:0/3774218720 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f1ed80e32b0 con 0x7f1ef0108f20 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 -- start start 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1edc0b29f0 0x7f1edc0b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009c60 tx=0x7f1ee0009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c7d80 0x55e3a76c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3a75e3940 con 0x55e3a76c1e10 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3a7548da0 con 0x55e3a7543e40 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e3a75df900 con 0x55e3a76c7d80 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eef59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42872/0 (socket says 172.21.15.5:42872) 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.311+0000 7f1eeed9d700 1 -- 172.21.15.5:0/4170857382 learned_addr learned my addr 172.21.15.5:0/4170857382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1ee659c700 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c7d80 0x55e3a76c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1eeed9d700 1 -- 172.21.15.5:0/4170857382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c7d80 msgr2=0x55e3a76c18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c7d80 0x55e3a76c18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1eeed9d700 1 -- 172.21.15.5:0/4170857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 msgr2=0x55e3a76c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1eeed9d700 1 -- 172.21.15.5:0/4170857382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a75308d0 con 0x55e3a7543e40 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76c7840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e4e0 tx=0x7f1ee0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f8426081be62e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1ee559a700 1 -- 172.21.15.5:0/4170857382 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000ee00 con 0x55e3a7543e40 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.312+0000 7f1ee559a700 1 -- 172.21.15.5:0/4170857382 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ee001c8c0 con 0x55e3a7543e40 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1ee559a700 1 -- 172.21.15.5:0/4170857382 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee001da20 con 0x55e3a7543e40 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/4170857382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 msgr2=0x55e3a76c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76c7840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e4e0 tx=0x7f1ee0014000 comp rx=0 tx=0).stop 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/4170857382 shutdown_connections 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76c7840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/4170857382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c7d80 0x55e3a76c18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/4170857382 >> 172.21.15.5:0/4170857382 conn(0x55e3a758b630 msgr2=0x55e3a76bca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.313+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/4170857382 shutdown_connections 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/4170857382 wait complete. 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 -- start start 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 Processor -- start 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a7543e40 0x55e3a76cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3a76ce870 con 0x55e3a76c1e10 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3a759ced0 con 0x55e3a76c7d80 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e3a76ce9f0 con 0x55e3a7543e40 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.314+0000 7f1f03aeeb80 10 client.0 _ll_register_callbacks cb 0x55e3a757b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1ee659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42876/0 (socket says 172.21.15.5:42876) 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 -- 172.21.15.5:0/1907779198 learned_addr learned my addr 172.21.15.5:0/1907779198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a7543e40 0x55e3a76cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 -- 172.21.15.5:0/1907779198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a7543e40 msgr2=0x55e3a76cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a7543e40 0x55e3a76cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 -- 172.21.15.5:0/1907779198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 msgr2=0x55e3a76cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 -- 172.21.15.5:0/1907779198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a759a760 con 0x55e3a76c7d80 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1ee659c700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1eef59e700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76cbe30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8003d50 tx=0x7f1ee800e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f77e4b8b2ee04c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1ed67fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1907779198 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80092d0 con 0x55e3a76c7d80 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee001c8c0 con 0x55e3a76c7d80 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.315+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1907779198 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f1ee8007500 con 0x55e3a76c7d80 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1907779198 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ee8027050 con 0x55e3a76c7d80 2024-05-14T15:27:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 10 client.25628 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3a76d9460 con 0x55e3a76c7d80 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1907779198 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ee001c8c0 con 0x55e3a76c7d80 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1907779198 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f1ee801e520 con 0x55e3a76c7d80 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 10 client.25628 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 10 client.25628 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 20 client.25628 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a76db9f0 0x55e3a76dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 4 client.25628 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3a75f1320 con 0x55e3a76db9f0 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.316+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a76db9f0 0x55e3a76dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.317+0000 7f1ed67fc700 10 client.25628 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:25.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.317+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a76db9f0 0x55e3a76dde30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12e9c3679690c778 server_cookie=7f2a2119511f688e in_seq=0 out_seq=0 2024-05-14T15:27:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1907779198 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3a75f1320 con 0x55e3a76db9f0 2024-05-14T15:27:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1ed67fc700 10 client.25628 handle_command_reply: tid=0 2024-05-14T15:27:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 client.25628 shutdown 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 msgr2=0x55e3a76cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76cbe30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8003d50 tx=0x7f1ee800e7d0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 shutdown_connections 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a76db9f0 0x55e3a76dde30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76cbe30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a7543e40 0x55e3a76cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1ed67fc700 0 client.25628 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1907779198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c1e10 0x55e3a76cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 >> 172.21.15.5:0/1907779198 conn(0x55e3a758b630 msgr2=0x55e3a76c08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 shutdown_connections 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1907779198 wait complete. 2024-05-14T15:27:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.318+0000 7f1f03aeeb80 20 client.25628 trim_cache size 0 max 16384 2024-05-14T15:27:25.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 Processor -- start 2024-05-14T15:27:25.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 -- start start 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 0x55e3a76c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76d7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01b2eb0 con 0x55e3a7543e40 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01b0740 con 0x55e3a76c7d80 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01ae590 con 0x55e3a76c1e10 2024-05-14T15:27:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 0x55e3a76c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 0x55e3a76c18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51338/0 (socket says 172.21.15.5:51338) 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1eeed9d700 1 -- 172.21.15.5:0/55230853 learned_addr learned my addr 172.21.15.5:0/55230853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.334+0000 7f1ee659c700 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eef59e700 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76d7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eeed9d700 1 -- 172.21.15.5:0/55230853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 msgr2=0x55e3a76cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eeed9d700 1 -- 172.21.15.5:0/55230853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 msgr2=0x55e3a76d7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76d7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eeed9d700 1 -- 172.21.15.5:0/55230853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a76cc1b0 con 0x55e3a7543e40 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 0x55e3a76c18d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e4e0 tx=0x7f1ee0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b5f4704ab312f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1ee4d99700 1 -- 172.21.15.5:0/55230853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0039070 con 0x55e3a7543e40 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1ee4d99700 1 -- 172.21.15.5:0/55230853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ee000ecd0 con 0x55e3a7543e40 2024-05-14T15:27:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.335+0000 7f1ee4d99700 1 -- 172.21.15.5:0/55230853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0003bd0 con 0x55e3a7543e40 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/55230853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 msgr2=0x55e3a76c18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 0x55e3a76c18d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e4e0 tx=0x7f1ee0011ae0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/55230853 shutdown_connections 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a76c7d80 0x55e3a76d7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a7543e40 0x55e3a76c18d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/55230853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/55230853 >> 172.21.15.5:0/55230853 conn(0x55e3a758b630 msgr2=0x55e3a76c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/55230853 shutdown_connections 2024-05-14T15:27:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.336+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/55230853 wait complete. 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 -- start start 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 Processor -- start 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c7d80 0x55e3a76e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01aec00 con 0x55e3a76c7d80 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01b0070 con 0x55e3a7543e40 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01af2d0 con 0x55e3a76c1e10 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1f03aeeb80 10 client.0 _ll_register_callbacks cb 0x55e3a757b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76d43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42890/0 (socket says 172.21.15.5:42890) 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1eeed9d700 1 -- 172.21.15.5:0/1499254783 learned_addr learned my addr 172.21.15.5:0/1499254783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1ee659c700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.337+0000 7f1eef59e700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c7d80 0x55e3a76e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ee659c700 1 -- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 msgr2=0x55e3a76d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ee659c700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ee659c700 1 -- 172.21.15.5:0/1499254783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c7d80 msgr2=0x55e3a76e33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ee659c700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c7d80 0x55e3a76e33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ee659c700 1 -- 172.21.15.5:0/1499254783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a76e3b40 con 0x55e3a76c1e10 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ee659c700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76e2d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80e0640 tx=0x7f1ed80b2e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a6d34ef65d37eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ed67fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1499254783 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80a99d0 con 0x55e3a76c1e10 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1499254783 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f1ed80039d0 con 0x55e3a76c1e10 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e3a75eb3c0 con 0x55e3a76c1e10 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1499254783 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ed80aa740 con 0x55e3a76c1e10 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1f03aeeb80 10 client.25651 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:25.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e3a7746230 con 0x55e3a76c1e10 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.338+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1499254783 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e3a75eb3c0 con 0x55e3a76c1e10 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1499254783 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f1ed80ae160 con 0x55e3a76c1e10 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1f03aeeb80 10 client.25651 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1f03aeeb80 10 client.25651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1f03aeeb80 20 client.25651 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a7746960 0x55e3a7748d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1f03aeeb80 4 client.25651 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3a7463250 con 0x55e3a7746960 2024-05-14T15:27:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a7746960 0x55e3a7748d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1eeed9d700 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a7746960 0x55e3a7748d80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=119d25a881b26fed server_cookie=240ef38412ef9871 in_seq=0 out_seq=0 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.339+0000 7f1ed67fc700 10 client.25651 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ed67fc700 1 -- 172.21.15.5:0/1499254783 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3a7463250 con 0x55e3a7746960 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ed67fc700 10 client.25651 handle_command_reply: tid=0 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 client.25651 shutdown 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 msgr2=0x55e3a76e2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76e2d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80e0640 tx=0x7f1ed80b2e80 comp rx=0 tx=0).stop 2024-05-14T15:27:25.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 shutdown_connections 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55e3a7746960 0x55e3a7748d80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3a76c7d80 0x55e3a76e33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3a7543e40 0x55e3a76d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ed67fc700 0 client.25651 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 --2- 172.21.15.5:0/1499254783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3a76c1e10 0x55e3a76e2d20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 >> 172.21.15.5:0/1499254783 conn(0x55e3a758b630 msgr2=0x55e3a76c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 shutdown_connections 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 1 -- 172.21.15.5:0/1499254783 wait complete. 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1f03aeeb80 20 client.25651 trim_cache size 0 max 16384 2024-05-14T15:27:25.346 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1edc0b29f0 msgr2=0x7f1edc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ef50da700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1edc0b29f0 0x7f1edc0b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009c60 tx=0x7f1ee0009260 comp rx=0 tx=0).stop 2024-05-14T15:27:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 msgr2=0x7f1ef019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.340+0000 7f1ef50da700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef019b400 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b580 tx=0x7f1ed8004270 comp rx=0 tx=0).stop 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 shutdown_connections 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1edc0b29f0 0x7f1edc0b4f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ef0109870 0x7f1ef019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef01045f0 0x7f1ef019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 --2- 172.21.15.5:0/3774218720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ef0108f20 0x7f1ef019b400 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 >> 172.21.15.5:0/3774218720 conn(0x7f1ef00fffe0 msgr2=0x7f1ef0106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 shutdown_connections 2024-05-14T15:27:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:25.341+0000 7f1ef50da700 1 -- 172.21.15.5:0/3774218720 wait complete. 2024-05-14T15:27:26.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:25 smithi144 ceph-mon[35650]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-05-14T15:27:26.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:25 smithi193 ceph-mon[36001]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-05-14T15:27:26.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:25 smithi005 ceph-mon[31559]: pgmap v81: 129 pgs: 129 active+clean; 32 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-05-14T15:27:26.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 112389 13.21 MB/sec warmup 47 sec latency 23.856 ms 2024-05-14T15:27:26.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:27:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:27.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 114232 13.19 MB/sec warmup 48 sec latency 13.708 ms 2024-05-14T15:27:27.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:27 smithi005 ceph-mon[31559]: pgmap v82: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 392 op/s 2024-05-14T15:27:28.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:27 smithi144 ceph-mon[35650]: pgmap v82: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 392 op/s 2024-05-14T15:27:28.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:27 smithi193 ceph-mon[36001]: pgmap v82: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 392 op/s 2024-05-14T15:27:28.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117099 13.27 MB/sec warmup 49 sec latency 58.164 ms 2024-05-14T15:27:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 -- 172.21.15.5:0/2940557781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1039e0 msgr2=0x7f22ac103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 --2- 172.21.15.5:0/2940557781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1039e0 0x7f22ac103df0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f2294009ca0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 -- 172.21.15.5:0/2940557781 shutdown_connections 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 --2- 172.21.15.5:0/2940557781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ac104330 0x7f22ac110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 --2- 172.21.15.5:0/2940557781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac1092d0 0x7f22ac1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 --2- 172.21.15.5:0/2940557781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1039e0 0x7f22ac103df0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 -- 172.21.15.5:0/2940557781 >> 172.21.15.5:0/2940557781 conn(0x7f22ac0fe4e0 msgr2=0x7f22ac100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 -- 172.21.15.5:0/2940557781 shutdown_connections 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.193+0000 7f22b1b53700 1 -- 172.21.15.5:0/2940557781 wait complete. 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 Processor -- start 2024-05-14T15:27:28.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 -- start start 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ac1039e0 0x7f22ac199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac104330 0x7f22ac19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 0x7f22ac19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac113350 con 0x7f22ac1092d0 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1131d0 con 0x7f22ac1039e0 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22b1b53700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1134d0 con 0x7f22ac104330 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 0x7f22ac19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 0x7f22ac19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50976/0 (socket says 172.21.15.5:50976) 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22abfff700 1 -- 172.21.15.5:0/502421864 learned_addr learned my addr 172.21.15.5:0/502421864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.194+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ac1039e0 0x7f22ac199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22a3fff700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac104330 0x7f22ac19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22abfff700 1 -- 172.21.15.5:0/502421864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac104330 msgr2=0x7f22ac19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22abfff700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac104330 0x7f22ac19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22abfff700 1 -- 172.21.15.5:0/502421864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ac1039e0 msgr2=0x7f22ac199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22abfff700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ac1039e0 0x7f22ac199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22abfff700 1 -- 172.21.15.5:0/502421864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac19ec20 con 0x7f22ac1092d0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22a3fff700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac104330 0x7f22ac19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22abfff700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 0x7f22ac19e580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f22a400ef30 tx=0x7f22a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22a97fa700 1 -- 172.21.15.5:0/502421864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a40092e0 con 0x7f22ac1092d0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac19eeb0 con 0x7f22ac1092d0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22a97fa700 1 -- 172.21.15.5:0/502421864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a4004bb0 con 0x7f22ac1092d0 2024-05-14T15:27:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22a97fa700 1 -- 172.21.15.5:0/502421864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4004280 con 0x7f22ac1092d0 2024-05-14T15:27:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.195+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac19f3a0 con 0x7f22ac1092d0 2024-05-14T15:27:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.199+0000 7f22a97fa700 1 -- 172.21.15.5:0/502421864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f22a401b030 con 0x7f22ac1092d0 2024-05-14T15:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22a97fa700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22980b29f0 0x7f22980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 Processor -- start 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22980b29f0 0x7f22980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22a97fa700 1 -- 172.21.15.5:0/502421864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f22a40e25a0 con 0x7f22ac1092d0 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 -- start start 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc6d3860 con 0x55e4cc633e40 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22980b29f0 0x7f22980b4f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f229c005950 tx=0x7f229c00a300 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc6727c0 con 0x55e4cc7b6470 2024-05-14T15:27:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.200+0000 7f22c0567b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc6cf820 con 0x55e4cc7b1c60 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50990/0 (socket says 172.21.15.5:50990) 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 -- 172.21.15.5:0/4006130448 learned_addr learned my addr 172.21.15.5:0/4006130448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22a3fff700 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 -- 172.21.15.5:0/4006130448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 msgr2=0x55e4cc7b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 -- 172.21.15.5:0/4006130448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 msgr2=0x55e4cc7b6880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b6880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 -- 172.21.15.5:0/4006130448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cc6208d0 con 0x55e4cc633e40 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7b1720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f229c00fd70 tx=0x7f229c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac91b631725cbd03 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22a1ffb700 1 -- 172.21.15.5:0/4006130448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c016070 con 0x55e4cc633e40 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22a1ffb700 1 -- 172.21.15.5:0/4006130448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c010d70 con 0x55e4cc633e40 2024-05-14T15:27:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.201+0000 7f22a1ffb700 1 -- 172.21.15.5:0/4006130448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c01cd90 con 0x55e4cc633e40 2024-05-14T15:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 -- 172.21.15.5:0/4006130448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 msgr2=0x55e4cc7b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7b1720 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f229c00fd70 tx=0x7f229c012000 comp rx=0 tx=0).stop 2024-05-14T15:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 -- 172.21.15.5:0/4006130448 shutdown_connections 2024-05-14T15:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7b1720 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 --2- 172.21.15.5:0/4006130448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 -- 172.21.15.5:0/4006130448 >> 172.21.15.5:0/4006130448 conn(0x55e4cc67b600 msgr2=0x55e4cc7ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 -- 172.21.15.5:0/4006130448 shutdown_connections 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.202+0000 7f22c0567b80 1 -- 172.21.15.5:0/4006130448 wait complete. 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.203+0000 7f22c0567b80 1 -- start start 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.203+0000 7f22c0567b80 1 Processor -- start 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc7be420 con 0x55e4cc633e40 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc639c30 con 0x55e4cc7b6470 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc7be5a0 con 0x55e4cc7b1c60 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22c0567b80 10 client.0 _ll_register_callbacks cb 0x55e4cc66b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:28.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7bb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51002/0 (socket says 172.21.15.5:51002) 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 -- 172.21.15.5:0/725631069 learned_addr learned my addr 172.21.15.5:0/725631069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22abfff700 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 -- 172.21.15.5:0/725631069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 msgr2=0x55e4cc7bb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 -- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 msgr2=0x55e4cc7bbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7bbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 -- 172.21.15.5:0/725631069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cc66f690 con 0x55e4cc633e40 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7bb1b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f229c01c910 tx=0x7f229c01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcdb2a0b3e1de2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f228affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.204+0000 7f228affd700 1 -- 172.21.15.5:0/725631069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c010ad0 con 0x55e4cc633e40 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f228affd700 1 -- 172.21.15.5:0/725631069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f229c01ca10 con 0x55e4cc633e40 2024-05-14T15:27:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c010d70 con 0x55e4cc633e40 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f228affd700 1 -- 172.21.15.5:0/725631069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f229c03b050 con 0x55e4cc633e40 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 10 client.16014 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4cc7c9050 con 0x55e4cc633e40 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f228affd700 1 -- 172.21.15.5:0/725631069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c010d70 con 0x55e4cc633e40 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f228affd700 1 -- 172.21.15.5:0/725631069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f229c004850 con 0x55e4cc633e40 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 20 client.16014 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc7cb5d0 0x55e4cc7cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 4 client.16014 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.205+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4cc6e1240 con 0x55e4cc7cb5d0 2024-05-14T15:27:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.206+0000 7f22a3fff700 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc7cb5d0 0x55e4cc7cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.207+0000 7f22a3fff700 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc7cb5d0 0x55e4cc7cd9f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fed6af5d85e882a server_cookie=bd61ed17daa4a063 in_seq=0 out_seq=0 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.207+0000 7f228affd700 10 client.16014 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.208+0000 7f228affd700 1 -- 172.21.15.5:0/725631069 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4cc6e1240 con 0x55e4cc7cb5d0 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f228affd700 10 client.16014 handle_command_reply: tid=0 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 client.16014 shutdown 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 msgr2=0x55e4cc7bb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7bb1b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f229c01c910 tx=0x7f229c01c650 comp rx=0 tx=0).stop 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 shutdown_connections 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc7cb5d0 0x55e4cc7cd9f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b6470 0x55e4cc7bbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7bb1b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 --2- 172.21.15.5:0/725631069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7bb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 >> 172.21.15.5:0/725631069 conn(0x55e4cc67b600 msgr2=0x55e4cc7b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f228affd700 0 client.16014 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 shutdown_connections 2024-05-14T15:27:28.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 1 -- 172.21.15.5:0/725631069 wait complete. 2024-05-14T15:27:28.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.209+0000 7f22c0567b80 20 client.16014 trim_cache size 0 max 16384 2024-05-14T15:27:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 Processor -- start 2024-05-14T15:27:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 -- start start 2024-05-14T15:27:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1b5260 con 0x55e4cc633e40 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1b2af0 con 0x55e4cc7b1c60 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.227+0000 7f22c0567b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1b0940 con 0x55e4cc7b6470 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7ba4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51004/0 (socket says 172.21.15.5:51004) 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2475703890 learned_addr learned my addr 172.21.15.5:0/2475703890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22a3fff700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2475703890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 msgr2=0x55e4cc7b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22abfff700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2475703890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b1c60 msgr2=0x55e4cc7c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2475703890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cc7b21d0 con 0x55e4cc633e40 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22a3fff700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22abfff700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7ba4d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f229c00a3e0 tx=0x7f229c00a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0bf0cee3ae47ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22a17fa700 1 -- 172.21.15.5:0/2475703890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c016050 con 0x55e4cc633e40 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22a17fa700 1 -- 172.21.15.5:0/2475703890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c032070 con 0x55e4cc633e40 2024-05-14T15:27:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.228+0000 7f22a17fa700 1 -- 172.21.15.5:0/2475703890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c004eb0 con 0x55e4cc633e40 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 -- 172.21.15.5:0/2475703890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 msgr2=0x55e4cc7ba4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7ba4d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f229c00a3e0 tx=0x7f229c00a370 comp rx=0 tx=0).stop 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 -- 172.21.15.5:0/2475703890 shutdown_connections 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc633e40 0x55e4cc7ba4d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2475703890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 -- 172.21.15.5:0/2475703890 >> 172.21.15.5:0/2475703890 conn(0x55e4cc67b600 msgr2=0x55e4cc7abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.229+0000 7f22c0567b80 1 -- 172.21.15.5:0/2475703890 shutdown_connections 2024-05-14T15:27:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.230+0000 7f22c0567b80 1 -- 172.21.15.5:0/2475703890 wait complete. 2024-05-14T15:27:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.230+0000 7f22c0567b80 1 -- start start 2024-05-14T15:27:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.230+0000 7f22c0567b80 1 Processor -- start 2024-05-14T15:27:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 0x55e4cc7cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1b0fb0 con 0x55e4cc7b1c60 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1b2420 con 0x55e4cc633e40 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1b1680 con 0x55e4cc7b6470 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.231+0000 7f22c0567b80 10 client.0 _ll_register_callbacks cb 0x55e4cc66b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.232+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 0x55e4cc7cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.232+0000 7f22abfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.233+0000 7f22ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 0x55e4cc7cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34570/0 (socket says 172.21.15.5:34570) 2024-05-14T15:27:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.233+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2424224005 learned_addr learned my addr 172.21.15.5:0/2424224005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.233+0000 7f22a3fff700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2424224005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 msgr2=0x55e4cc7ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2424224005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc7b1c60 msgr2=0x55e4cc7cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22ab7fe700 1 -- 172.21.15.5:0/2424224005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cc833b10 con 0x55e4cc633e40 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22abfff700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f22ab7fe700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 0x55e4cc7cd5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f229c004cc0 tx=0x7f229c010d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c82209ab488077 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f228affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f228affd700 1 -- 172.21.15.5:0/2424224005 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c016040 con 0x55e4cc633e40 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.234+0000 7f228affd700 1 -- 172.21.15.5:0/2424224005 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f229c01cab0 con 0x55e4cc633e40 2024-05-14T15:27:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.235+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f229c032070 con 0x55e4cc633e40 2024-05-14T15:27:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.235+0000 7f228affd700 1 -- 172.21.15.5:0/2424224005 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f229c01ccb0 con 0x55e4cc633e40 2024-05-14T15:27:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.235+0000 7f22c0567b80 10 client.25649 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.235+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4cc834c90 con 0x55e4cc633e40 2024-05-14T15:27:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f228affd700 1 -- 172.21.15.5:0/2424224005 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c032070 con 0x55e4cc633e40 2024-05-14T15:27:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f228affd700 1 -- 172.21.15.5:0/2424224005 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f229c0485d0 con 0x55e4cc633e40 2024-05-14T15:27:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f22c0567b80 10 client.25649 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f22c0567b80 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f22c0567b80 20 client.25649 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc835270 0x55e4cc837690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f22c0567b80 4 client.25649 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.236+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4cc6db2e0 con 0x55e4cc835270 2024-05-14T15:27:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.239+0000 7f22a3fff700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc835270 0x55e4cc837690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.243+0000 7f22a3fff700 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc835270 0x55e4cc837690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e054cf6c3eabf176 server_cookie=216e004ef3f4e7c7 in_seq=0 out_seq=0 2024-05-14T15:27:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.243+0000 7f228affd700 10 client.25649 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.244+0000 7f228affd700 1 -- 172.21.15.5:0/2424224005 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4cc6db2e0 con 0x55e4cc835270 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.244+0000 7f228affd700 10 client.25649 handle_command_reply: tid=0 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.244+0000 7f22c0567b80 1 client.25649 shutdown 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 msgr2=0x55e4cc7cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 0x55e4cc7cd5e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f229c004cc0 tx=0x7f229c010d60 comp rx=0 tx=0).stop 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 shutdown_connections 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e4cc835270 0x55e4cc837690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f228affd700 0 client.25649 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4cc7b6470 0x55e4cc7ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4cc633e40 0x55e4cc7cd5e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 --2- 172.21.15.5:0/2424224005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4cc7b1c60 0x55e4cc7cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 >> 172.21.15.5:0/2424224005 conn(0x55e4cc67b600 msgr2=0x55e4cc7ba230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 shutdown_connections 2024-05-14T15:27:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 1 -- 172.21.15.5:0/2424224005 wait complete. 2024-05-14T15:27:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.245+0000 7f22c0567b80 20 client.25649 trim_cache size 0 max 16384 2024-05-14T15:27:28.249 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.246+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22980b29f0 msgr2=0x7f22980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.246+0000 7f22b1b53700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22980b29f0 0x7f22980b4f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f229c005950 tx=0x7f229c00a300 comp rx=0 tx=0).stop 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 msgr2=0x7f22ac19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 0x7f22ac19e580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f22a400ef30 tx=0x7f22a400c3a0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 shutdown_connections 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22980b29f0 0x7f22980b4f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac1092d0 0x7f22ac19e580 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22ac1039e0 0x7f22ac199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 --2- 172.21.15.5:0/502421864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22ac104330 0x7f22ac19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.247+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 >> 172.21.15.5:0/502421864 conn(0x7f22ac0fe4e0 msgr2=0x7f22ac100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.248+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 shutdown_connections 2024-05-14T15:27:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:28.248+0000 7f22b1b53700 1 -- 172.21.15.5:0/502421864 wait complete. 2024-05-14T15:27:29.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 119340 13.16 MB/sec warmup 50 sec latency 19.398 ms 2024-05-14T15:27:29.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:27:29.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:27:29.999 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:29 smithi005 ceph-mon[31559]: pgmap v83: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 306 op/s 2024-05-14T15:27:30.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:29 smithi144 ceph-mon[35650]: pgmap v83: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 306 op/s 2024-05-14T15:27:30.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:29 smithi193 ceph-mon[36001]: pgmap v83: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 306 op/s 2024-05-14T15:27:30.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 121446 13.17 MB/sec warmup 51 sec latency 20.847 ms 2024-05-14T15:27:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:31.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124421 13.27 MB/sec warmup 52 sec latency 14.827 ms 2024-05-14T15:27:32.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:31 smithi144 ceph-mon[35650]: pgmap v84: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 306 op/s 2024-05-14T15:27:32.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:31 smithi193 ceph-mon[36001]: pgmap v84: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 306 op/s 2024-05-14T15:27:32.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:31 smithi005 ceph-mon[31559]: pgmap v84: 129 pgs: 129 active+clean; 38 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 306 op/s 2024-05-14T15:27:32.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 126556 13.16 MB/sec warmup 53 sec latency 58.584 ms 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 -- 172.21.15.5:0/2503856520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 msgr2=0x7f3c7c109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2503856520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 0x7f3c7c109330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c700099c0 tx=0x7f3c70009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 -- 172.21.15.5:0/2503856520 shutdown_connections 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2503856520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c109870 0x7f3c7c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2503856520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c1045f0 0x7f3c7c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2503856520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 0x7f3c7c109330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.252+0000 7f3c8177e700 1 -- 172.21.15.5:0/2503856520 >> 172.21.15.5:0/2503856520 conn(0x7f3c7c100040 msgr2=0x7f3c7c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.253+0000 7f3c8177e700 1 -- 172.21.15.5:0/2503856520 shutdown_connections 2024-05-14T15:27:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.253+0000 7f3c8177e700 1 -- 172.21.15.5:0/2503856520 wait complete. 2024-05-14T15:27:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.254+0000 7f3c8177e700 1 Processor -- start 2024-05-14T15:27:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 -- start start 2024-05-14T15:27:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c1045f0 0x7f3c7c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 0x7f3c7c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c109870 0x7f3c7c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c112340 con 0x7f3c7c108f20 2024-05-14T15:27:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1121c0 con 0x7f3c7c1045f0 2024-05-14T15:27:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c8177e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1124c0 con 0x7f3c7c109870 2024-05-14T15:27:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.255+0000 7f3c7b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c109870 0x7f3c7c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.256+0000 7f3c7b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c109870 0x7f3c7c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:44754/0 (socket says 172.21.15.5:44754) 2024-05-14T15:27:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.256+0000 7f3c7b7fe700 1 -- 172.21.15.5:0/2556911775 learned_addr learned my addr 172.21.15.5:0/2556911775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.256+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c1045f0 0x7f3c7c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.256+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 0x7f3c7c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.257+0000 7f3c7affd700 1 -- 172.21.15.5:0/2556911775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c109870 msgr2=0x7f3c7c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.257+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c109870 0x7f3c7c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.257+0000 7f3c7affd700 1 -- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 msgr2=0x7f3c7c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.257+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 0x7f3c7c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.257+0000 7f3c7affd700 1 -- 172.21.15.5:0/2556911775 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c199150 con 0x7f3c7c1045f0 2024-05-14T15:27:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.257+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c1045f0 0x7f3c7c19e880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00d950 tx=0x7f3c6c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.258+0000 7f3c63fff700 1 -- 172.21.15.5:0/2556911775 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c017070 con 0x7f3c7c1045f0 2024-05-14T15:27:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.258+0000 7f3c63fff700 1 -- 172.21.15.5:0/2556911775 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c00be10 con 0x7f3c7c1045f0 2024-05-14T15:27:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.259+0000 7f3c63fff700 1 -- 172.21.15.5:0/2556911775 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00f3c0 con 0x7f3c7c1045f0 2024-05-14T15:27:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.259+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c199440 con 0x7f3c7c1045f0 2024-05-14T15:27:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.259+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c199890 con 0x7f3c7c1045f0 2024-05-14T15:27:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.264+0000 7f3c63fff700 1 -- 172.21.15.5:0/2556911775 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3c6c01a070 con 0x7f3c7c1045f0 2024-05-14T15:27:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.265+0000 7f3c63fff700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3c640b2bc0 0x7f3c640b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.265+0000 7f3c63fff700 1 -- 172.21.15.5:0/2556911775 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f3c6c0e2270 con 0x7f3c7c1045f0 2024-05-14T15:27:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.265+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3c640b2bc0 0x7f3c640b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.266+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3c640b2bc0 0x7f3c640b50d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c70000c00 tx=0x7f3c7001a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 Processor -- start 2024-05-14T15:27:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 -- start start 2024-05-14T15:27:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 0x564ef422c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef422cce0 0x564ef4230fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef42314f0 0x564ef4231900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ef414e8e0 con 0x564ef422cce0 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ef40ed840 con 0x564ef40aeec0 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.268+0000 7f3c90192b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564ef414a8a0 con 0x564ef42314f0 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 0x564ef422c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 0x564ef422c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34606/0 (socket says 172.21.15.5:34606) 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 -- 172.21.15.5:0/2425187057 learned_addr learned my addr 172.21.15.5:0/2425187057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7b7fe700 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef42314f0 0x564ef4231900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 -- 172.21.15.5:0/2425187057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef42314f0 msgr2=0x564ef4231900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef42314f0 0x564ef4231900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 -- 172.21.15.5:0/2425187057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef422cce0 msgr2=0x564ef4230fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef422cce0 0x564ef4230fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 -- 172.21.15.5:0/2425187057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef409b950 con 0x564ef40aeec0 2024-05-14T15:27:32.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.269+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 0x564ef422c7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0a51b0 tx=0x7f3c6c00df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf68d1ee97bd1fae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.270+0000 7f3c617fa700 1 -- 172.21.15.5:0/2425187057 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c021040 con 0x564ef40aeec0 2024-05-14T15:27:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.270+0000 7f3c617fa700 1 -- 172.21.15.5:0/2425187057 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c0a9130 con 0x564ef40aeec0 2024-05-14T15:27:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.270+0000 7f3c617fa700 1 -- 172.21.15.5:0/2425187057 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0a95d0 con 0x564ef40aeec0 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.271+0000 7f3c90192b80 1 -- 172.21.15.5:0/2425187057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 msgr2=0x564ef422c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.271+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 0x564ef422c7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0a51b0 tx=0x7f3c6c00df50 comp rx=0 tx=0).stop 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 -- 172.21.15.5:0/2425187057 shutdown_connections 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef42314f0 0x564ef4231900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef40aeec0 0x564ef422c7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2425187057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef422cce0 0x564ef4230fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 -- 172.21.15.5:0/2425187057 >> 172.21.15.5:0/2425187057 conn(0x564ef40f6680 msgr2=0x564ef4227530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 -- 172.21.15.5:0/2425187057 shutdown_connections 2024-05-14T15:27:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 -- 172.21.15.5:0/2425187057 wait complete. 2024-05-14T15:27:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 -- start start 2024-05-14T15:27:32.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.272+0000 7f3c90192b80 1 Processor -- start 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4236300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4236840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4236d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ef4239440 con 0x564ef40aeec0 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ef40b4cb0 con 0x564ef42314f0 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564ef42395c0 con 0x564ef422cce0 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c90192b80 10 client.0 _ll_register_callbacks cb 0x564ef40e69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:32.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4236300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4236300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51062/0 (socket says 172.21.15.5:51062) 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 -- 172.21.15.5:0/2836597081 learned_addr learned my addr 172.21.15.5:0/2836597081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7b7fe700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4236d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4236840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 -- 172.21.15.5:0/2836597081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 msgr2=0x564ef4236840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4236840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 -- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 msgr2=0x564ef4236d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4236d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 -- 172.21.15.5:0/2836597081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef40ea710 con 0x564ef40aeec0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7b7fe700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4236d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.273+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4236300 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c005950 tx=0x7f3c6c0ad350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cc6a93dba19961 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c5eff5700 1 -- 172.21.15.5:0/2836597081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c021070 con 0x564ef40aeec0 2024-05-14T15:27:32.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c5eff5700 1 -- 172.21.15.5:0/2836597081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f3c6c0a95a0 con 0x564ef40aeec0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c0a9130 con 0x564ef40aeec0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c5eff5700 1 -- 172.21.15.5:0/2836597081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c0ad4e0 con 0x564ef40aeec0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c90192b80 10 client.16029 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ef42440f0 con 0x564ef40aeec0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c5eff5700 1 -- 172.21.15.5:0/2836597081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6c0a9130 con 0x564ef40aeec0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c5eff5700 1 -- 172.21.15.5:0/2836597081 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f3c6c0a4aa0 con 0x564ef40aeec0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c90192b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.274+0000 7f3c90192b80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c90192b80 20 client.16029 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c90192b80 4 client.16029 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef415c2c0 con 0x564ef42465c0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42489e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526e079ba1b42099 server_cookie=a08aa6035b0a37bb in_seq=0 out_seq=0 2024-05-14T15:27:32.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.275+0000 7f3c5eff5700 10 client.16029 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.276+0000 7f3c5eff5700 1 -- 172.21.15.5:0/2836597081 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ef415c2c0 con 0x564ef42465c0 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.276+0000 7f3c5eff5700 10 client.16029 handle_command_reply: tid=0 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 client.16029 shutdown 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 msgr2=0x564ef4236300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4236300 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c005950 tx=0x7f3c6c0ad350 comp rx=0 tx=0).stop 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 shutdown_connections 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42489e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4236d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4236300 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c5eff5700 0 client.16029 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2836597081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4236840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 >> 172.21.15.5:0/2836597081 conn(0x564ef40f6680 msgr2=0x564ef4231900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 shutdown_connections 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 1 -- 172.21.15.5:0/2836597081 wait complete. 2024-05-14T15:27:32.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.277+0000 7f3c90192b80 20 client.16029 trim_cache size 0 max 16384 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 Processor -- start 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 -- start start 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4227690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4232910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4242860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1b0da0 con 0x564ef40aeec0 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ae630 con 0x564ef42314f0 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c90192b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ac480 con 0x564ef422cce0 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4227690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4232910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4227690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51078/0 (socket says 172.21.15.5:51078) 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4232910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:44770/0 (socket says 172.21.15.5:44770) 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7affd700 1 -- 172.21.15.5:0/2227495698 learned_addr learned my addr 172.21.15.5:0/2227495698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7b7fe700 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4242860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7a7fc700 1 -- 172.21.15.5:0/2227495698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 msgr2=0x564ef4242860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4242860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7a7fc700 1 -- 172.21.15.5:0/2227495698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 msgr2=0x564ef4227690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.300+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4227690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.301+0000 7f3c7a7fc700 1 -- 172.21.15.5:0/2227495698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef4242da0 con 0x564ef422cce0 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.301+0000 7f3c7affd700 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4227690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.301+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4232910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009990 tx=0x7f3c70009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8490425623dd1af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.301+0000 7f3c60ff9700 1 -- 172.21.15.5:0/2227495698 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70037070 con 0x564ef422cce0 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.301+0000 7f3c60ff9700 1 -- 172.21.15.5:0/2227495698 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c7000bc40 con 0x564ef422cce0 2024-05-14T15:27:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.301+0000 7f3c60ff9700 1 -- 172.21.15.5:0/2227495698 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70030460 con 0x564ef422cce0 2024-05-14T15:27:32.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 -- 172.21.15.5:0/2227495698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 msgr2=0x564ef4232910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4232910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009990 tx=0x7f3c70009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 -- 172.21.15.5:0/2227495698 shutdown_connections 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef42314f0 0x564ef4242860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef40aeec0 0x564ef4227690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 --2- 172.21.15.5:0/2227495698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef422cce0 0x564ef4232910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 -- 172.21.15.5:0/2227495698 >> 172.21.15.5:0/2227495698 conn(0x564ef40f6680 msgr2=0x564ef4221870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 -- 172.21.15.5:0/2227495698 shutdown_connections 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 -- 172.21.15.5:0/2227495698 wait complete. 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.302+0000 7f3c90192b80 1 -- start start 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 Processor -- start 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef40aeec0 0x564ef4249fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 0x564ef424a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef42314f0 0x564ef42afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1acaf0 con 0x564ef42314f0 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1adf60 con 0x564ef422cce0 2024-05-14T15:27:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1ad1c0 con 0x564ef40aeec0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c90192b80 10 client.0 _ll_register_callbacks cb 0x564ef40e69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 0x564ef424a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef40aeec0 0x564ef4249fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef40aeec0 0x564ef4249fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:44780/0 (socket says 172.21.15.5:44780) 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 0x564ef424a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34638/0 (socket says 172.21.15.5:34638) 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7affd700 1 -- 172.21.15.5:0/3243799787 learned_addr learned my addr 172.21.15.5:0/3243799787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7b7fe700 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef42314f0 0x564ef42afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 -- 172.21.15.5:0/3243799787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef40aeec0 msgr2=0x564ef4249fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef40aeec0 0x564ef4249fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 -- 172.21.15.5:0/3243799787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef42314f0 msgr2=0x564ef42afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef42314f0 0x564ef42afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.303+0000 7f3c7a7fc700 1 -- 172.21.15.5:0/3243799787 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef424adc0 con 0x564ef422cce0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c7a7fc700 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 0x564ef424a500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x564ef421f660 tx=0x7f3c70009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f200356410cdb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c5eff5700 1 -- 172.21.15.5:0/3243799787 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70040060 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c7000bc40 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c5eff5700 1 -- 172.21.15.5:0/3243799787 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f3c7000b990 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c5eff5700 1 -- 172.21.15.5:0/3243799787 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c70047050 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c90192b80 10 client.25667 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ef42b10f0 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.304+0000 7f3c5eff5700 1 -- 172.21.15.5:0/3243799787 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c7000bc40 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c5eff5700 1 -- 172.21.15.5:0/3243799787 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f3c70022070 con 0x564ef422cce0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c90192b80 10 client.25667 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c90192b80 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c90192b80 20 client.25667 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c90192b80 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c90192b80 4 client.25667 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ef4156360 con 0x564ef42465c0 2024-05-14T15:27:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c7affd700 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c7affd700 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42b3740 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bf97cb08128188 server_cookie=8562cb749cadc18a in_seq=0 out_seq=0 2024-05-14T15:27:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.305+0000 7f3c5eff5700 10 client.25667 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c5eff5700 1 -- 172.21.15.5:0/3243799787 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ef4156360 con 0x564ef42465c0 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c5eff5700 10 client.25667 handle_command_reply: tid=0 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 client.25667 shutdown 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 msgr2=0x564ef424a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 0x564ef424a500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x564ef421f660 tx=0x7f3c70009fd0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 shutdown_connections 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564ef42465c0 0x564ef42b3740 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ef42314f0 0x564ef42afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.306+0000 7f3c90192b80 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ef40aeec0 0x564ef4249fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c90192b80 1 --2- 172.21.15.5:0/3243799787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ef422cce0 0x564ef424a500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 >> 172.21.15.5:0/3243799787 conn(0x564ef40f6680 msgr2=0x564ef4221870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c5eff5700 0 client.25667 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 shutdown_connections 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c90192b80 1 -- 172.21.15.5:0/3243799787 wait complete. 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c90192b80 20 client.25667 trim_cache size 0 max 16384 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "02b255a4-da27-4f1f-ab61-237df15d88a7", 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:27:32.312 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3c640b2bc0 msgr2=0x7f3c640b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3c640b2bc0 0x7f3c640b50d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c70000c00 tx=0x7f3c7001a040 comp rx=0 tx=0).stop 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c1045f0 msgr2=0x7f3c7c19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c1045f0 0x7f3c7c19e880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00d950 tx=0x7f3c6c00dd10 comp rx=0 tx=0).stop 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 shutdown_connections 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3c640b2bc0 0x7f3c640b50d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c7c109870 0x7f3c7c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c7c1045f0 0x7f3c7c19e880 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 --2- 172.21.15.5:0/2556911775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c7c108f20 0x7f3c7c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.307+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 >> 172.21.15.5:0/2556911775 conn(0x7f3c7c100040 msgr2=0x7f3c7c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.308+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 shutdown_connections 2024-05-14T15:27:32.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:32.308+0000 7f3c8177e700 1 -- 172.21.15.5:0/2556911775 wait complete. 2024-05-14T15:27:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 -- 172.21.15.5:0/3264540454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 msgr2=0x7f3374103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 --2- 172.21.15.5:0/3264540454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f3374103870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3364009900 tx=0x7f336402a410 comp rx=0 tx=0).stop 2024-05-14T15:27:32.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 -- 172.21.15.5:0/3264540454 shutdown_connections 2024-05-14T15:27:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 --2- 172.21.15.5:0/3264540454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f337410a170 0x7f3374103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 --2- 172.21.15.5:0/3264540454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3374104420 0x7f33741107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 --2- 172.21.15.5:0/3264540454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f3374103870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 -- 172.21.15.5:0/3264540454 >> 172.21.15.5:0/3264540454 conn(0x7f33740793d0 msgr2=0x7f3374079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 -- 172.21.15.5:0/3264540454 shutdown_connections 2024-05-14T15:27:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.430+0000 7f337a186640 1 -- 172.21.15.5:0/3264540454 wait complete. 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 Processor -- start 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 -- start start 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 0x7f337419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f337419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f337410a170 0x7f33741a8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3374113ec0 con 0x7f3374109cd0 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3374113d40 con 0x7f337410a170 2024-05-14T15:27:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f337a186640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3374114040 con 0x7f3374104420 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f3372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f337419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f33737fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 0x7f337419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.431+0000 7f33737fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 0x7f337419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:44796/0 (socket says 172.21.15.5:44796) 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f33737fe640 1 -- 172.21.15.5:0/3942158924 learned_addr learned my addr 172.21.15.5:0/3942158924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f337419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51094/0 (socket says 172.21.15.5:51094) 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3373fff640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f337410a170 0x7f33741a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 -- 172.21.15.5:0/3942158924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 msgr2=0x7f337419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 0x7f337419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 -- 172.21.15.5:0/3942158924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f337410a170 msgr2=0x7f33741a8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f337410a170 0x7f33741a8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 -- 172.21.15.5:0/3942158924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33741a8d60 con 0x7f3374109cd0 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f33737fe640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 0x7f337419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3372ffd640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f337419dd00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f337406a330 tx=0x7f336800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3370ff9640 1 -- 172.21.15.5:0/3942158924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336800cc40 con 0x7f3374109cd0 2024-05-14T15:27:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741a8f90 con 0x7f3374109cd0 2024-05-14T15:27:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3370ff9640 1 -- 172.21.15.5:0/3942158924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336800cde0 con 0x7f3374109cd0 2024-05-14T15:27:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f3370ff9640 1 -- 172.21.15.5:0/3942158924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368010430 con 0x7f3374109cd0 2024-05-14T15:27:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.432+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a9400 con 0x7f3374109cd0 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.435+0000 7f3370ff9640 1 -- 172.21.15.5:0/3942158924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3368003680 con 0x7f3374109cd0 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.435+0000 7f338103f740 1 Processor -- start 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.435+0000 7f338103f740 1 -- start start 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f338103f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a271fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f338103f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e322f0 0x560a271fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f338103f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e3a4d0 0x560a271fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f338103f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a271615d0 con 0x560a26e3a4d0 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f338103f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560a2711a890 con 0x560a26e5b1b0 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f338103f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a2715d590 con 0x560a26e322f0 2024-05-14T15:27:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3372ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e3a4d0 0x560a271fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a271fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f33737fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e322f0 0x560a271fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a271fbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34664/0 (socket says 172.21.15.5:34664) 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 -- 172.21.15.5:0/2220655960 learned_addr learned my addr 172.21.15.5:0/2220655960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 -- 172.21.15.5:0/2220655960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e322f0 msgr2=0x560a271fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e322f0 0x560a271fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 -- 172.21.15.5:0/2220655960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e3a4d0 msgr2=0x560a271fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e3a4d0 0x560a271fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 -- 172.21.15.5:0/2220655960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a26e73210 con 0x560a26e5b1b0 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3370ff9640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f335c0b2950 0x7f335c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3372ffd640 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e3a4d0 0x560a271fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f33737fe640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f335c0b2950 0x7f335c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3373fff640 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a271fbbe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f336c009fd0 tx=0x7f336c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=202986790bd16e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.436+0000 7f3370ff9640 1 -- 172.21.15.5:0/3942158924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f33680e2cf0 con 0x7f3374109cd0 2024-05-14T15:27:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.437+0000 7f3359ffb640 1 -- 172.21.15.5:0/2220655960 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c004d20 con 0x560a26e5b1b0 2024-05-14T15:27:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.437+0000 7f3359ffb640 1 -- 172.21.15.5:0/2220655960 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c004050 con 0x560a26e5b1b0 2024-05-14T15:27:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.437+0000 7f3359ffb640 1 -- 172.21.15.5:0/2220655960 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c005260 con 0x560a26e5b1b0 2024-05-14T15:27:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 -- 172.21.15.5:0/2220655960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 msgr2=0x560a271fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f33737fe640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f335c0b2950 0x7f335c0b4e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x560a271f9320 tx=0x7f3364032ed0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a271fbbe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f336c009fd0 tx=0x7f336c0118d0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 -- 172.21.15.5:0/2220655960 shutdown_connections 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e322f0 0x560a271fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e3a4d0 0x560a271fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 --2- 172.21.15.5:0/2220655960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a271fbbe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 -- 172.21.15.5:0/2220655960 >> 172.21.15.5:0/2220655960 conn(0x560a26e4f300 msgr2=0x560a271f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 -- 172.21.15.5:0/2220655960 shutdown_connections 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 -- 172.21.15.5:0/2220655960 wait complete. 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 -- start start 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.438+0000 7f338103f740 1 Processor -- start 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a272061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a27206700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a272121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a27208d10 con 0x560a26e5b1b0 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560a2711e370 con 0x560a26e322f0 2024-05-14T15:27:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a27208e90 con 0x560a26e3a4d0 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f338103f740 10 client.0 _ll_register_callbacks cb 0x560a26e72a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a272121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a27206700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f33737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a272061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a272121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51108/0 (socket says 172.21.15.5:51108) 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3373fff640 1 -- 172.21.15.5:0/1350574325 learned_addr learned my addr 172.21.15.5:0/1350574325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 -- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 msgr2=0x560a272061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a272061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 -- 172.21.15.5:0/1350574325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 msgr2=0x560a272121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a272121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 -- 172.21.15.5:0/1350574325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a26f3f620 con 0x560a26e3a4d0 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3373fff640 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a272121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3372ffd640 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a27206700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33680e0080 tx=0x7f3368009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee45b2e28ab3639 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.439+0000 7f3352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f3352ffd640 1 -- 172.21.15.5:0/1350574325 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680a9560 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336c004050 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f3352ffd640 1 -- 172.21.15.5:0/1350574325 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f33680b27f0 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f3352ffd640 1 -- 172.21.15.5:0/1350574325 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33680a9380 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 10 client.25687 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a272150f0 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f3352ffd640 1 -- 172.21.15.5:0/1350574325 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c004050 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f3352ffd640 1 -- 172.21.15.5:0/1350574325 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f33680a5580 con 0x560a26e3a4d0 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 10 client.25687 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 20 client.25687 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26e19d80 0x560a27218570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 4 client.25687 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a2716f1e0 con 0x560a26e19d80 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.440+0000 7f33737fe640 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26e19d80 0x560a27218570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.441+0000 7f33737fe640 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26e19d80 0x560a27218570 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb4617ec00848219 server_cookie=890498abf5767dcf in_seq=0 out_seq=0 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.441+0000 7f3352ffd640 10 client.25687 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f3352ffd640 1 -- 172.21.15.5:0/1350574325 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a2716f1e0 con 0x560a26e19d80 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f3352ffd640 10 client.25687 handle_command_reply: tid=0 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 client.25687 shutdown 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 msgr2=0x560a27206700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a27206700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33680e0080 tx=0x7f3368009f60 comp rx=0 tx=0).stop 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 shutdown_connections 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26e19d80 0x560a27218570 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a27206700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f3352ffd640 0 client.25687 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a272121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 --2- 172.21.15.5:0/1350574325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a272061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 >> 172.21.15.5:0/1350574325 conn(0x560a26e4f300 msgr2=0x560a271fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 shutdown_connections 2024-05-14T15:27:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 1 -- 172.21.15.5:0/1350574325 wait complete. 2024-05-14T15:27:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.442+0000 7f338103f740 20 client.25687 trim_cache size 0 max 16384 2024-05-14T15:27:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 Processor -- start 2024-05-14T15:27:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 -- start start 2024-05-14T15:27:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 0x560a272008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a271fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a27211da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b5e60 con 0x560a26e322f0 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b3730 con 0x560a26e5b1b0 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.459+0000 7f338103f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b1580 con 0x560a26e3a4d0 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 0x560a272008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 0x560a272008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51116/0 (socket says 172.21.15.5:51116) 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 -- 172.21.15.5:0/582554080 learned_addr learned my addr 172.21.15.5:0/582554080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f3372ffd640 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a271fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f3373fff640 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a27211da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 -- 172.21.15.5:0/582554080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 msgr2=0x560a271fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a271fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 -- 172.21.15.5:0/582554080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 msgr2=0x560a27211da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a27211da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 -- 172.21.15.5:0/582554080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a27208850 con 0x560a26e322f0 2024-05-14T15:27:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f3372ffd640 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a271fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33737fe640 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 0x560a272008e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3364000c00 tx=0x7f336403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cff76355b57546d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33597fa640 1 -- 172.21.15.5:0/582554080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364043070 con 0x560a26e322f0 2024-05-14T15:27:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33597fa640 1 -- 172.21.15.5:0/582554080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364057070 con 0x560a26e322f0 2024-05-14T15:27:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.460+0000 7f33597fa640 1 -- 172.21.15.5:0/582554080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364035040 con 0x560a26e322f0 2024-05-14T15:27:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 -- 172.21.15.5:0/582554080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 msgr2=0x560a272008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 0x560a272008e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3364000c00 tx=0x7f336403f000 comp rx=0 tx=0).stop 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 -- 172.21.15.5:0/582554080 shutdown_connections 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a271fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e5b1b0 0x560a27211da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 --2- 172.21.15.5:0/582554080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e322f0 0x560a272008e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 -- 172.21.15.5:0/582554080 >> 172.21.15.5:0/582554080 conn(0x560a26e4f300 msgr2=0x560a271f7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 -- 172.21.15.5:0/582554080 shutdown_connections 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.461+0000 7f338103f740 1 -- 172.21.15.5:0/582554080 wait complete. 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 -- start start 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 Processor -- start 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a2720e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a2720b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a2720bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b1bf0 con 0x560a26e5b1b0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b3060 con 0x560a26e322f0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b22c0 con 0x560a26e3a4d0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f338103f740 10 client.0 _ll_register_callbacks cb 0x7f33680a5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f3373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a2720bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f3373fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a2720bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51118/0 (socket says 172.21.15.5:51118) 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f3373fff640 1 -- 172.21.15.5:0/3727938527 learned_addr learned my addr 172.21.15.5:0/3727938527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f3372ffd640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a2720b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.462+0000 7f33737fe640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a2720e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3373fff640 1 -- 172.21.15.5:0/3727938527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 msgr2=0x560a2720b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3373fff640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a2720b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3373fff640 1 -- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 msgr2=0x560a2720e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3373fff640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a2720e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3373fff640 1 -- 172.21.15.5:0/3727938527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2720c120 con 0x560a26e5b1b0 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3372ffd640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a2720b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f33737fe640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a2720e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3373fff640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a2720bbe0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f336c01b040 tx=0x7f336c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c623ce9899151d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3352ffd640 1 -- 172.21.15.5:0/3727938527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c004600 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3352ffd640 1 -- 172.21.15.5:0/3727938527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9474+0+0 (secure 0 0 0) 0x7f336c00f4e0 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3364057070 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f3352ffd640 1 -- 172.21.15.5:0/3727938527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f336c036050 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f338103f740 10 client.16053 fetch_fsmap learned FSMap version 28 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.463+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a27280400 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f3352ffd640 1 -- 172.21.15.5:0/3727938527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364057070 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f3352ffd640 1 -- 172.21.15.5:0/3727938527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f336c0304d0 con 0x560a26e5b1b0 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f338103f740 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f338103f740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f338103f740 20 client.16053 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f338103f740 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26def1c0 0x560a272829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f338103f740 4 client.16053 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:27:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a27169050 con 0x560a26def1c0 2024-05-14T15:27:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f33737fe640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26def1c0 0x560a272829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f3352ffd640 10 client.16053 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.464+0000 7f33737fe640 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26def1c0 0x560a272829c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f873cc2192f1f72b server_cookie=fe1226e73b2b8c2b in_seq=0 out_seq=0 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f3352ffd640 1 -- 172.21.15.5:0/3727938527 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a27169050 con 0x560a26def1c0 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f3352ffd640 10 client.16053 handle_command_reply: tid=0 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 client.16053 shutdown 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 msgr2=0x560a2720bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a2720bbe0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f336c01b040 tx=0x7f336c004280 comp rx=0 tx=0).stop 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 shutdown_connections 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560a26def1c0 0x560a272829c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f3352ffd640 0 client.16053 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a26e3a4d0 0x560a2720b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a26e5b1b0 0x560a2720bbe0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 --2- 172.21.15.5:0/3727938527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a26e322f0 0x560a2720e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 >> 172.21.15.5:0/3727938527 conn(0x560a26e4f300 msgr2=0x560a271fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 shutdown_connections 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.684+0000 7f338103f740 1 -- 172.21.15.5:0/3727938527 wait complete. 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f338103f740 20 client.16053 trim_cache size 0 max 16384 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f335c0b2950 msgr2=0x7f335c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f335c0b2950 0x7f335c0b4e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x560a271f9320 tx=0x7f3364032ed0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 msgr2=0x7f337419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f337419dd00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f337406a330 tx=0x7f336800ee50 comp rx=0 tx=0).stop 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 shutdown_connections 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f335c0b2950 0x7f335c0b4e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f337410a170 0x7f33741a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3374109cd0 0x7f337419dd00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 --2- 172.21.15.5:0/3942158924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3374104420 0x7f337419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 >> 172.21.15.5:0/3942158924 conn(0x7f33740793d0 msgr2=0x7f33741023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 shutdown_connections 2024-05-14T15:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:32.685+0000 7f337a186640 1 -- 172.21.15.5:0/3942158924 wait complete. 2024-05-14T15:27:33.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:32 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:27:33.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:32 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:33.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:32 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:27:33.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:32 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:32 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:27:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:32 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:32 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:27:33.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:32 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:33.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:32 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:27:33.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:32 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:33.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:32 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:27:33.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:32 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:33.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 128214 13.14 MB/sec warmup 54 sec latency 118.022 ms 2024-05-14T15:27:33.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:27:33.469 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:33.547 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:33 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setuser ceph since I am not root 2024-05-14T15:27:33.547 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:33 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setgroup ceph since I am not root 2024-05-14T15:27:33.548 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:33 smithi144 ceph-mds[65770]: -- 172.21.15.144:0/2339654928 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a4b1cda40 con 0x560a4b215a80 2024-05-14T15:27:33.548 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:27:33 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: starting mds.e at 2024-05-14T15:27:33.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:33 smithi005 ceph-mon[31559]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 388 op/s 2024-05-14T15:27:33.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:33 smithi005 ceph-mon[31559]: daemon mds.e restarted 2024-05-14T15:27:34.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:33 smithi144 ceph-mon[35650]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 388 op/s 2024-05-14T15:27:34.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:33 smithi144 ceph-mon[35650]: daemon mds.e restarted 2024-05-14T15:27:34.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:33 smithi193 ceph-mon[36001]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 388 op/s 2024-05-14T15:27:34.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:33 smithi193 ceph-mon[36001]: daemon mds.e restarted 2024-05-14T15:27:34.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130742 13.13 MB/sec warmup 55 sec latency 23.304 ms 2024-05-14T15:27:35.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:34 smithi005 ceph-mon[31559]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T15:27:35.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:34 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6834/2401236249,v1:172.21.15.144:6835/2401236249] up:boot 2024-05-14T15:27:35.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:34 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:27:35.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:34 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:27:35.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:34 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:27:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:34 smithi144 ceph-mon[35650]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T15:27:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:34 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6834/2401236249,v1:172.21.15.144:6835/2401236249] up:boot 2024-05-14T15:27:35.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:34 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:27:35.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:34 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:27:35.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:34 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:27:35.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:34 smithi193 ceph-mon[36001]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T15:27:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:34 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6834/2401236249,v1:172.21.15.144:6835/2401236249] up:boot 2024-05-14T15:27:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:34 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:27:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:34 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:27:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:34 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:27:35.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 132539 13.05 MB/sec warmup 56 sec latency 13.646 ms 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.490+0000 7ff88a59b700 1 -- 172.21.15.5:0/3136135826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884107df0 msgr2=0x7ff884108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.490+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3136135826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884107df0 0x7ff884108200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff8780099c0 tx=0x7ff878009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 -- 172.21.15.5:0/3136135826 shutdown_connections 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3136135826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff884108740 0x7ff8841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3136135826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8841034c0 0x7ff8841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3136135826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884107df0 0x7ff884108200 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 -- 172.21.15.5:0/3136135826 >> 172.21.15.5:0/3136135826 conn(0x7ff8840fedc0 msgr2=0x7ff8841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:35.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 -- 172.21.15.5:0/3136135826 shutdown_connections 2024-05-14T15:27:35.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 -- 172.21.15.5:0/3136135826 wait complete. 2024-05-14T15:27:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 Processor -- start 2024-05-14T15:27:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.491+0000 7ff88a59b700 1 -- start start 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff88a59b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 0x7ff88419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff88a59b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 0x7ff88419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff88a59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884108740 0x7ff88419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff88a59b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff884113350 con 0x7ff884108740 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff88a59b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841131d0 con 0x7ff8841034c0 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff88a59b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841134d0 con 0x7ff884107df0 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 0x7ff88419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff883fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 0x7ff88419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 0x7ff88419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:44850/0 (socket says 172.21.15.5:44850) 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff883fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 0x7ff88419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34718/0 (socket says 172.21.15.5:34718) 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 -- 172.21.15.5:0/3642268508 learned_addr learned my addr 172.21.15.5:0/3642268508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff888b38700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884108740 0x7ff88419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 -- 172.21.15.5:0/3642268508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 msgr2=0x7ff88419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 0x7ff88419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 -- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884108740 msgr2=0x7ff88419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884108740 0x7ff88419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff8837fe700 1 -- 172.21.15.5:0/3642268508 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88419fc80 con 0x7ff884107df0 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.492+0000 7ff883fff700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 0x7ff88419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.493+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 0x7ff88419b350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8780045e0 tx=0x7ff878004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.493+0000 7ff8817fa700 1 -- 172.21.15.5:0/3642268508 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878026070 con 0x7ff884107df0 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.493+0000 7ff8817fa700 1 -- 172.21.15.5:0/3642268508 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff87800f460 con 0x7ff884107df0 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.493+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88419ff10 con 0x7ff884107df0 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.493+0000 7ff8817fa700 1 -- 172.21.15.5:0/3642268508 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878009e60 con 0x7ff884107df0 2024-05-14T15:27:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.493+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a7890 con 0x7ff884107df0 2024-05-14T15:27:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.496+0000 7ff8817fa700 1 -- 172.21.15.5:0/3642268508 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff878029070 con 0x7ff884107df0 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff8817fa700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff86c0b2a70 0x7ff86c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff898fafb80 1 Processor -- start 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff883fff700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff86c0b2a70 0x7ff86c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff8817fa700 1 -- 172.21.15.5:0/3642268508 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7ff8780e30b0 con 0x7ff884107df0 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff898fafb80 1 -- start start 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb744b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 0x559fb744ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.497+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb7450470 0x559fb7450880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff898fafb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fb736d860 con 0x559fb7450470 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff898fafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559fb730c7c0 con 0x559fb744bc60 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff898fafb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fb7369820 con 0x559fb72cde40 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff883fff700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff86c0b2a70 0x7ff86c0b4f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff874005950 tx=0x7ff87400b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 0x559fb744ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff888b38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb7450470 0x559fb7450880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 0x559fb744ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34734/0 (socket says 172.21.15.5:34734) 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 -- 172.21.15.5:0/71053263 learned_addr learned my addr 172.21.15.5:0/71053263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff883fff700 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb744b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 -- 172.21.15.5:0/71053263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 msgr2=0x559fb744b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb744b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 -- 172.21.15.5:0/71053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb7450470 msgr2=0x559fb7450880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb7450470 0x559fb7450880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 -- 172.21.15.5:0/71053263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb72ba8d0 con 0x559fb744bc60 2024-05-14T15:27:35.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff888b38700 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb7450470 0x559fb7450880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.498+0000 7ff8837fe700 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 0x559fb744ff30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8780a6080 tx=0x7ff8780a9770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8f6ff645c34bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.499+0000 7ff86a7fc700 1 -- 172.21.15.5:0/71053263 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780aa700 con 0x559fb744bc60 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.499+0000 7ff86a7fc700 1 -- 172.21.15.5:0/71053263 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8780aa000 con 0x559fb744bc60 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.499+0000 7ff86a7fc700 1 -- 172.21.15.5:0/71053263 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780aa460 con 0x559fb744bc60 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 -- 172.21.15.5:0/71053263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 msgr2=0x559fb744ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 0x559fb744ff30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8780a6080 tx=0x7ff8780a9770 comp rx=0 tx=0).stop 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 -- 172.21.15.5:0/71053263 shutdown_connections 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb7450470 0x559fb7450880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb744b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 --2- 172.21.15.5:0/71053263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb744bc60 0x559fb744ff30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 -- 172.21.15.5:0/71053263 >> 172.21.15.5:0/71053263 conn(0x559fb7315600 msgr2=0x559fb74464b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 -- 172.21.15.5:0/71053263 shutdown_connections 2024-05-14T15:27:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 -- 172.21.15.5:0/71053263 wait complete. 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 -- start start 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.500+0000 7ff898fafb80 1 Processor -- start 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb74551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb74556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7455c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fb74581e0 con 0x559fb744bc60 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559fb72d3c30 con 0x559fb7450470 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fb7458360 con 0x559fb72cde40 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff898fafb80 10 client.0 _ll_register_callbacks cb 0x559fb7305920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb74556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb74556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51158/0 (socket says 172.21.15.5:51158) 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 -- 172.21.15.5:0/2531644798 learned_addr learned my addr 172.21.15.5:0/2531644798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff888b38700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7455c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff883fff700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb74551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 -- 172.21.15.5:0/2531644798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 msgr2=0x559fb74551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb74551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 -- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 msgr2=0x559fb7455c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7455c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.501+0000 7ff8837fe700 1 -- 172.21.15.5:0/2531644798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb7309690 con 0x559fb744bc60 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8837fe700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb74556f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c010 tx=0x7ff8780a6510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52db9c16548cf88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8637fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8637fe700 1 -- 172.21.15.5:0/2531644798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780c8020 con 0x559fb744bc60 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8780aa000 con 0x559fb744bc60 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8637fe700 1 -- 172.21.15.5:0/2531644798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7ff8780aa470 con 0x559fb744bc60 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8637fe700 1 -- 172.21.15.5:0/2531644798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8780b2dd0 con 0x559fb744bc60 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff898fafb80 10 client.16068 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:35.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fb74581e0 con 0x559fb744bc60 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8637fe700 1 -- 172.21.15.5:0/2531644798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8780aa000 con 0x559fb744bc60 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.502+0000 7ff8637fe700 1 -- 172.21.15.5:0/2531644798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff8780a5970 con 0x559fb744bc60 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff898fafb80 10 client.16068 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff898fafb80 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff898fafb80 20 client.16068 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff898fafb80 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb7465050 0x559fb7467470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff898fafb80 4 client.16068 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fb737b240 con 0x559fb7465050 2024-05-14T15:27:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff883fff700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb7465050 0x559fb7467470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff883fff700 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb7465050 0x559fb7467470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c868fce7ab36c8 server_cookie=2ef6d22de0e7822 in_seq=0 out_seq=0 2024-05-14T15:27:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.503+0000 7ff8637fe700 10 client.16068 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.505+0000 7ff8637fe700 1 -- 172.21.15.5:0/2531644798 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559fb737b240 con 0x559fb7465050 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.505+0000 7ff8637fe700 10 client.16068 handle_command_reply: tid=0 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.505+0000 7ff898fafb80 1 client.16068 shutdown 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 msgr2=0x559fb74556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb74556f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c010 tx=0x7ff8780a6510 comp rx=0 tx=0).stop 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 shutdown_connections 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb7465050 0x559fb7467470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7455c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb74551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 --2- 172.21.15.5:0/2531644798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb74556f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 >> 172.21.15.5:0/2531644798 conn(0x559fb7315600 msgr2=0x559fb744a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff8637fe700 0 client.16068 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 shutdown_connections 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 1 -- 172.21.15.5:0/2531644798 wait complete. 2024-05-14T15:27:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.506+0000 7ff898fafb80 20 client.16068 trim_cache size 0 max 16384 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 Processor -- start 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 -- start start 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb7446610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb7455a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7461230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841b2d90 con 0x559fb744bc60 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841b0620 con 0x559fb7450470 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff898fafb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841ae470 con 0x559fb72cde40 2024-05-14T15:27:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb7455a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb7455a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51166/0 (socket says 172.21.15.5:51166) 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff8837fe700 1 -- 172.21.15.5:0/3790220955 learned_addr learned my addr 172.21.15.5:0/3790220955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff888b38700 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7461230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.527+0000 7ff883fff700 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb7446610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff8837fe700 1 -- 172.21.15.5:0/3790220955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 msgr2=0x559fb7446610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb7446610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff8837fe700 1 -- 172.21.15.5:0/3790220955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 msgr2=0x559fb7461230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7461230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff8837fe700 1 -- 172.21.15.5:0/3790220955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb74501a0 con 0x559fb744bc60 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff888b38700 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7461230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb7455a70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff8780ae4f0 tx=0x7ff87800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bc4e78ec995546 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff869ffb700 1 -- 172.21.15.5:0/3790220955 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780c8020 con 0x559fb744bc60 2024-05-14T15:27:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff869ffb700 1 -- 172.21.15.5:0/3790220955 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8780aa000 con 0x559fb744bc60 2024-05-14T15:27:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.528+0000 7ff869ffb700 1 -- 172.21.15.5:0/3790220955 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780aa470 con 0x559fb744bc60 2024-05-14T15:27:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 -- 172.21.15.5:0/3790220955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 msgr2=0x559fb7455a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb7455a70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff8780ae4f0 tx=0x7ff87800fc20 comp rx=0 tx=0).stop 2024-05-14T15:27:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 -- 172.21.15.5:0/3790220955 shutdown_connections 2024-05-14T15:27:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb7461230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb72cde40 0x559fb7446610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3790220955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb744bc60 0x559fb7455a70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 -- 172.21.15.5:0/3790220955 >> 172.21.15.5:0/3790220955 conn(0x559fb7315600 msgr2=0x559fb7454110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 -- 172.21.15.5:0/3790220955 shutdown_connections 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 -- 172.21.15.5:0/3790220955 wait complete. 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.529+0000 7ff898fafb80 1 -- start start 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 Processor -- start 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb72cde40 0x559fb745d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb744bc60 0x559fb745ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb745b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841aeae0 con 0x559fb72cde40 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841aff50 con 0x559fb7450470 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841af1b0 con 0x559fb744bc60 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff898fafb80 10 client.0 _ll_register_callbacks cb 0x559fb7305920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff883fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb72cde40 0x559fb745d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff888b38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb745b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff883fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb72cde40 0x559fb745d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51170/0 (socket says 172.21.15.5:51170) 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff888b38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb745b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:34768/0 (socket says 172.21.15.5:34768) 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff883fff700 1 -- 172.21.15.5:0/3444358226 learned_addr learned my addr 172.21.15.5:0/3444358226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.530+0000 7ff8837fe700 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb744bc60 0x559fb745ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff888b38700 1 -- 172.21.15.5:0/3444358226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb744bc60 msgr2=0x559fb745ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff888b38700 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb744bc60 0x559fb745ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff888b38700 1 -- 172.21.15.5:0/3444358226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb72cde40 msgr2=0x559fb745d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff888b38700 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb72cde40 0x559fb745d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff888b38700 1 -- 172.21.15.5:0/3444358226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb745b740 con 0x559fb7450470 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff888b38700 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb745b1d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004630 tx=0x7ff87c004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838a4accca429715 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff8637fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff8637fe700 1 -- 172.21.15.5:0/3444358226 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c004bb0 con 0x559fb7450470 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8780aa000 con 0x559fb7450470 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.531+0000 7ff8637fe700 1 -- 172.21.15.5:0/3444358226 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7ff87c004d50 con 0x559fb7450470 2024-05-14T15:27:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff8637fe700 1 -- 172.21.15.5:0/3444358226 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff87c02b050 con 0x559fb7450470 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 10 client.25709 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fb74cf3b0 con 0x559fb7450470 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff8637fe700 1 -- 172.21.15.5:0/3444358226 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8780aa000 con 0x559fb7450470 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff8637fe700 1 -- 172.21.15.5:0/3444358226 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff87c020bc0 con 0x559fb7450470 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 20 client.25709 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb74cfb20 0x559fb74d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 4 client.25709 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fb73752e0 con 0x559fb74cfb20 2024-05-14T15:27:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.532+0000 7ff883fff700 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb74cfb20 0x559fb74d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff8637fe700 10 client.25709 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff883fff700 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb74cfb20 0x559fb74d1f40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3357e88cd519f0 server_cookie=c083cec398eb24f9 in_seq=0 out_seq=0 2024-05-14T15:27:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff8637fe700 1 -- 172.21.15.5:0/3444358226 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559fb73752e0 con 0x559fb74cfb20 2024-05-14T15:27:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff8637fe700 10 client.25709 handle_command_reply: tid=0 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff898fafb80 1 client.25709 shutdown 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 msgr2=0x559fb745b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.533+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb745b1d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004630 tx=0x7ff87c004940 comp rx=0 tx=0).stop 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 shutdown_connections 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb74cfb20 0x559fb74d1f40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb7450470 0x559fb745b1d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb72cde40 0x559fb745d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff8637fe700 0 client.25709 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 --2- 172.21.15.5:0/3444358226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb744bc60 0x559fb745ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 >> 172.21.15.5:0/3444358226 conn(0x559fb7315600 msgr2=0x559fb744a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 shutdown_connections 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 1 -- 172.21.15.5:0/3444358226 wait complete. 2024-05-14T15:27:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff898fafb80 20 client.25709 trim_cache size 0 max 16384 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff86c0b2a70 msgr2=0x7ff86c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff86c0b2a70 0x7ff86c0b4f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff874005950 tx=0x7ff87400b410 comp rx=0 tx=0).stop 2024-05-14T15:27:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 msgr2=0x7ff88419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.534+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 0x7ff88419b350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8780045e0 tx=0x7ff878004610 comp rx=0 tx=0).stop 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 shutdown_connections 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff86c0b2a70 0x7ff86c0b4f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff884108740 0x7ff88419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8841034c0 0x7ff88419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 --2- 172.21.15.5:0/3642268508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff884107df0 0x7ff88419b350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 >> 172.21.15.5:0/3642268508 conn(0x7ff8840fedc0 msgr2=0x7ff88406b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 shutdown_connections 2024-05-14T15:27:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:35.535+0000 7ff88a59b700 1 -- 172.21.15.5:0/3642268508 wait complete. 2024-05-14T15:27:36.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:35 smithi144 ceph-mon[35650]: pgmap v87: 129 pgs: 129 active+clean; 44 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-05-14T15:27:36.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:35 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:27:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:35 smithi193 ceph-mon[36001]: pgmap v87: 129 pgs: 129 active+clean; 44 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-05-14T15:27:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:35 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:27:36.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:35 smithi005 ceph-mon[31559]: pgmap v87: 129 pgs: 129 active+clean; 44 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-05-14T15:27:36.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:35 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:27:36.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 135283 13.12 MB/sec warmup 57 sec latency 13.467 ms 2024-05-14T15:27:36.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:27:36.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:27:36.809 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:37.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 138147 13.14 MB/sec warmup 58 sec latency 19.532 ms 2024-05-14T15:27:38.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:37 smithi144 ceph-mon[35650]: pgmap v88: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:38.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:37 smithi193 ceph-mon[36001]: pgmap v88: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:38.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:37 smithi005 ceph-mon[31559]: pgmap v88: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:38.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 139886 13.08 MB/sec warmup 59 sec latency 14.121 ms 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 -- 172.21.15.5:0/3970188953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 msgr2=0x7fa2a8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 --2- 172.21.15.5:0/3970188953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a8109350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2900099c0 tx=0x7fa290009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 -- 172.21.15.5:0/3970188953 shutdown_connections 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 --2- 172.21.15.5:0/3970188953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2a8109890 0x7fa2a81111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 --2- 172.21.15.5:0/3970188953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 0x7fa2a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 --2- 172.21.15.5:0/3970188953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a8109350 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 -- 172.21.15.5:0/3970188953 >> 172.21.15.5:0/3970188953 conn(0x7fa2a807af70 msgr2=0x7fa2a807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 -- 172.21.15.5:0/3970188953 shutdown_connections 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.718+0000 7fa2ae169700 1 -- 172.21.15.5:0/3970188953 wait complete. 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.719+0000 7fa2ae169700 1 Processor -- start 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.719+0000 7fa2ae169700 1 -- start start 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2ae169700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 0x7fa2a819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2ae169700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2ae169700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2a8109890 0x7fa2a819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2ae169700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81144a0 con 0x7fa2a8108f40 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2ae169700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8114320 con 0x7fa2a8109890 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2ae169700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8114620 con 0x7fa2a8104610 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 0x7fa2a819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55902/0 (socket says 172.21.15.5:55902) 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 -- 172.21.15.5:0/2113779630 learned_addr learned my addr 172.21.15.5:0/2113779630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2a7fff700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2a8109890 0x7fa2a819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 -- 172.21.15.5:0/2113779630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 msgr2=0x7fa2a819af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 0x7fa2a819af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 -- 172.21.15.5:0/2113779630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2a8109890 msgr2=0x7fa2a819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2a8109890 0x7fa2a819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 -- 172.21.15.5:0/2113779630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a819fde0 con 0x7fa2a8108f40 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2a77fe700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 0x7fa2a819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa29effd700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a819b4b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa290006010 tx=0x7fa290005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.720+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290026070 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.721+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290005680 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.721+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81a0070 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.721+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000fb70 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.721+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a0550 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.722+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a80512c0 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.724+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa29000fd10 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.724+0000 7fa2a57fa700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2940b29f0 0x7fa2940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.725+0000 7fa2a77fe700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2940b29f0 0x7fa2940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.725+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fa2900e3580 con 0x7fa2a8108f40 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.725+0000 7fa2a77fe700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2940b29f0 0x7fa2940b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa298011490 tx=0x7fa298009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:38.729+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa290013070 con 0x7fa2a8108f40 2024-05-14T15:27:39.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.013+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2a802e5f0 con 0x7fa2a8108f40 2024-05-14T15:27:39.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.014+0000 7fa2a57fa700 1 -- 172.21.15.5:0/2113779630 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11864 (secure 0 0 0) 0x7fa2900aac30 con 0x7fa2a8108f40 2024-05-14T15:27:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:27:39.015 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:27:33.754380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25691":{"gid":25691,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2401236249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2401236249},{"type":"v1","addr":"172.21.15.144:6835","nonce":2401236249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:27:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2940b29f0 msgr2=0x7fa2940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2940b29f0 0x7fa2940b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa298011490 tx=0x7fa298009260 comp rx=0 tx=0).stop 2024-05-14T15:27:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 msgr2=0x7fa2a819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a819b4b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa290006010 tx=0x7fa290005280 comp rx=0 tx=0).stop 2024-05-14T15:27:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 shutdown_connections 2024-05-14T15:27:39.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2940b29f0 0x7fa2940b4f00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2a8109890 0x7fa2a819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a8104610 0x7fa2a819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 --2- 172.21.15.5:0/2113779630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a8108f40 0x7fa2a819b4b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 >> 172.21.15.5:0/2113779630 conn(0x7fa2a807af70 msgr2=0x7fa2a8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 shutdown_connections 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:39.017+0000 7fa2ae169700 1 -- 172.21.15.5:0/2113779630 wait complete. 2024-05-14T15:27:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-05-14T15:27:39.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142360 13.10 MB/sec warmup 60 sec latency 17.409 ms 2024-05-14T15:27:39.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:27:40.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:39 smithi144 ceph-mon[35650]: pgmap v89: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:40.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:39 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2113779630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:27:40.050 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:40.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:39 smithi193 ceph-mon[36001]: pgmap v89: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:40.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:39 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2113779630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:27:40.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:39 smithi005 ceph-mon[31559]: pgmap v89: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:40.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:39 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2113779630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:27:40.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145082 13.10 MB/sec warmup 61 sec latency 25.513 ms 2024-05-14T15:27:41.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 146070 12.98 MB/sec warmup 62 sec latency 152.866 ms 2024-05-14T15:27:42.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:41 smithi005 ceph-mon[31559]: pgmap v90: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:42.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:41 smithi144 ceph-mon[35650]: pgmap v90: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:42.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:41 smithi193 ceph-mon[36001]: pgmap v90: 129 pgs: 129 active+clean; 38 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 361 op/s 2024-05-14T15:27:42.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 147049 12.86 MB/sec warmup 63 sec latency 377.771 ms 2024-05-14T15:27:42.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 -- 172.21.15.5:0/3187132813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640103550 msgr2=0x7f1640103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 --2- 172.21.15.5:0/3187132813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640103550 0x7f1640103960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f16300099c0 tx=0x7f1630009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 -- 172.21.15.5:0/3187132813 shutdown_connections 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 --2- 172.21.15.5:0/3187132813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640108780 0x7f16401100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 --2- 172.21.15.5:0/3187132813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640103550 0x7f1640103960 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 --2- 172.21.15.5:0/3187132813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1640107e30 0x7f1640108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 -- 172.21.15.5:0/3187132813 >> 172.21.15.5:0/3187132813 conn(0x7f1640078e00 msgr2=0x7f1640079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 -- 172.21.15.5:0/3187132813 shutdown_connections 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.279+0000 7f164631d700 1 -- 172.21.15.5:0/3187132813 wait complete. 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.280+0000 7f164631d700 1 Processor -- start 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.280+0000 7f164631d700 1 -- start start 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f164631d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 0x7f164019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f164631d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1640107e30 0x7f164019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f164631d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640108780 0x7f164019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f164631d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16401133d0 con 0x7f1640108780 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f164631d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1640113250 con 0x7f1640107e30 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f164631d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1640113550 con 0x7f1640103550 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f163ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 0x7f164019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640108780 0x7f164019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1640107e30 0x7f164019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f163ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 0x7f164019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36916/0 (socket says 172.21.15.5:36916) 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f163ffff700 1 -- 172.21.15.5:0/3722811167 learned_addr learned my addr 172.21.15.5:0/3722811167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 -- 172.21.15.5:0/3722811167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 msgr2=0x7f164019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 0x7f164019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 -- 172.21.15.5:0/3722811167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1640107e30 msgr2=0x7f164019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1640107e30 0x7f164019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 -- 172.21.15.5:0/3722811167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16401a7000 con 0x7f1640108780 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f163ffff700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 0x7f164019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.281+0000 7f16448ba700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640108780 0x7f164019c750 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f163800ab40 tx=0x7f163800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.282+0000 7f163d7fa700 1 -- 172.21.15.5:0/3722811167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638017020 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.282+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16401a72f0 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.282+0000 7f163d7fa700 1 -- 172.21.15.5:0/3722811167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1638004bb0 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.282+0000 7f163d7fa700 1 -- 172.21.15.5:0/3722811167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638005280 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.282+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401a7830 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.284+0000 7f163d7fa700 1 -- 172.21.15.5:0/3722811167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1638005420 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.285+0000 7f163d7fa700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f16280b29f0 0x7f16280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.285+0000 7f163ffff700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f16280b29f0 0x7f16280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.285+0000 7f163d7fa700 1 -- 172.21.15.5:0/3722811167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f16380e1f20 con 0x7f1640108780 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.285+0000 7f1654d31b80 1 Processor -- start 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 -- start start 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8aa8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8aad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596f89ca860 con 0x5596f8aa8c60 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596f89697c0 con 0x5596f892ae40 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f1654d31b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5596f89c6820 con 0x5596f8aad470 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163ffff700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f16280b29f0 0x7f16280b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1630003800 tx=0x7f1630005fb0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8aa8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aacf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55918/0 (socket says 172.21.15.5:55918) 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 -- 172.21.15.5:0/1480752400 learned_addr learned my addr 172.21.15.5:0/1480752400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 -- 172.21.15.5:0/1480752400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 msgr2=0x5596f8aad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8aad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 -- 172.21.15.5:0/1480752400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 msgr2=0x5596f8aa8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8aa8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.286+0000 7f163f7fe700 1 -- 172.21.15.5:0/1480752400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f89178d0 con 0x5596f8aa8c60 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.287+0000 7f163f7fe700 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aacf30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f163400bcf0 tx=0x7f16340103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216714e1677bd199 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.287+0000 7f16267fc700 1 -- 172.21.15.5:0/1480752400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634010d10 con 0x5596f8aa8c60 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.287+0000 7f16267fc700 1 -- 172.21.15.5:0/1480752400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16340044e0 con 0x5596f8aa8c60 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.287+0000 7f16267fc700 1 -- 172.21.15.5:0/1480752400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634005280 con 0x5596f8aa8c60 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 -- 172.21.15.5:0/1480752400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 msgr2=0x5596f8aacf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aacf30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f163400bcf0 tx=0x7f16340103d0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 -- 172.21.15.5:0/1480752400 shutdown_connections 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8aad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8aa8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 --2- 172.21.15.5:0/1480752400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aacf30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 -- 172.21.15.5:0/1480752400 >> 172.21.15.5:0/1480752400 conn(0x5596f8972600 msgr2=0x5596f8aa34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 -- 172.21.15.5:0/1480752400 shutdown_connections 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 -- 172.21.15.5:0/1480752400 wait complete. 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.288+0000 7f1654d31b80 1 -- start start 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 Processor -- start 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 0x5596f8ab21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f8aa8c60 0x5596f8ab2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596f8ab5420 con 0x5596f892ae40 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596f8930c30 con 0x5596f8aa8c60 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5596f8ab55a0 con 0x5596f8aad470 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f1654d31b80 10 client.0 _ll_register_callbacks cb 0x5596f8962920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f163ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 0x5596f8ab21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f8aa8c60 0x5596f8ab2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f163ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 0x5596f8ab21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55930/0 (socket says 172.21.15.5:55930) 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f163ffff700 1 -- 172.21.15.5:0/3826511080 learned_addr learned my addr 172.21.15.5:0/3826511080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f16448ba700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f163f7fe700 1 -- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 msgr2=0x5596f8ab2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.289+0000 7f163f7fe700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f163f7fe700 1 -- 172.21.15.5:0/3826511080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 msgr2=0x5596f8ab21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f163f7fe700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 0x5596f8ab21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f163f7fe700 1 -- 172.21.15.5:0/3826511080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f8966690 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f163ffff700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 0x5596f8ab21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f163f7fe700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f8aa8c60 0x5596f8ab2720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1634004cc0 tx=0x7f1634004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d9b1472ca69546 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f161f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f161f7fe700 1 -- 172.21.15.5:0/3826511080 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163402a060 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16340044e0 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f161f7fe700 1 -- 172.21.15.5:0/3826511080 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f1634004dc0 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f161f7fe700 1 -- 172.21.15.5:0/3826511080 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1634031050 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f1654d31b80 10 client.25727 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.290+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596f8ac0080 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f161f7fe700 1 -- 172.21.15.5:0/3826511080 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16340044e0 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f161f7fe700 1 -- 172.21.15.5:0/3826511080 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f16340116b0 con 0x5596f8aa8c60 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f1654d31b80 10 client.25727 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f1654d31b80 10 client.25727 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f1654d31b80 20 client.25727 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f1654d31b80 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8ac2660 0x5596f8ac4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f1654d31b80 4 client.25727 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f89d8240 con 0x5596f8ac2660 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.291+0000 7f163ffff700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8ac2660 0x5596f8ac4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.292+0000 7f163ffff700 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8ac2660 0x5596f8ac4a80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7af57f052b246a0 server_cookie=6e90d5649311d635 in_seq=0 out_seq=0 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.292+0000 7f161f7fe700 10 client.25727 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.292+0000 7f161f7fe700 1 -- 172.21.15.5:0/3826511080 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596f89d8240 con 0x5596f8ac2660 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.292+0000 7f161f7fe700 10 client.25727 handle_command_reply: tid=0 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 client.25727 shutdown 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f8aa8c60 msgr2=0x5596f8ab2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f8aa8c60 0x5596f8ab2720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1634004cc0 tx=0x7f1634004980 comp rx=0 tx=0).stop 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 shutdown_connections 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f892ae40 0x5596f8ab21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8ac2660 0x5596f8ac4a80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 --2- 172.21.15.5:0/3826511080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f8aa8c60 0x5596f8ab2720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 >> 172.21.15.5:0/3826511080 conn(0x5596f8972600 msgr2=0x5596f8aa7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f161f7fe700 0 client.25727 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 shutdown_connections 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 1 -- 172.21.15.5:0/3826511080 wait complete. 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.293+0000 7f1654d31b80 20 client.25727 trim_cache size 0 max 16384 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.309+0000 7f1654d31b80 1 Processor -- start 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.309+0000 7f1654d31b80 1 -- start start 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ab1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aa32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f1654d31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16401b0d50 con 0x5596f8aa8c60 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f1654d31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ae5e0 con 0x5596f892ae40 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f1654d31b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ac430 con 0x5596f8aad470 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aa32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aa32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55934/0 (socket says 172.21.15.5:55934) 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ab1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 -- 172.21.15.5:0/471262282 learned_addr learned my addr 172.21.15.5:0/471262282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f16448ba700 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 -- 172.21.15.5:0/471262282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 msgr2=0x5596f8ab21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 -- 172.21.15.5:0/471262282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 msgr2=0x5596f8ab1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ab1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 -- 172.21.15.5:0/471262282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f8a9dbb0 con 0x5596f8aa8c60 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.310+0000 7f163f7fe700 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aa32f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1634011c70 tx=0x7f163400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bedb1d9b4d1354 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.311+0000 7f1625ffb700 1 -- 172.21.15.5:0/471262282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163402a040 con 0x5596f8aa8c60 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.311+0000 7f1625ffb700 1 -- 172.21.15.5:0/471262282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16340044e0 con 0x5596f8aa8c60 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.311+0000 7f1625ffb700 1 -- 172.21.15.5:0/471262282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340038e0 con 0x5596f8aa8c60 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 -- 172.21.15.5:0/471262282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 msgr2=0x5596f8aa32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aa32f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1634011c70 tx=0x7f163400b410 comp rx=0 tx=0).stop 2024-05-14T15:27:42.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 -- 172.21.15.5:0/471262282 shutdown_connections 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aad470 0x5596f8ab21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ab1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 --2- 172.21.15.5:0/471262282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aa8c60 0x5596f8aa32f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 -- 172.21.15.5:0/471262282 >> 172.21.15.5:0/471262282 conn(0x5596f8972600 msgr2=0x5596f8aa2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 -- 172.21.15.5:0/471262282 shutdown_connections 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 -- 172.21.15.5:0/471262282 wait complete. 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 -- start start 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.312+0000 7f1654d31b80 1 Processor -- start 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ac7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aa8c60 0x5596f8ac61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aad470 0x5596f8ac66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16401acaa0 con 0x5596f8aad470 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16401adf10 con 0x5596f892ae40 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ad170 con 0x5596f8aa8c60 2024-05-14T15:27:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f1654d31b80 10 client.0 _ll_register_callbacks cb 0x5596f8962920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f163ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ac7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aad470 0x5596f8ac66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f163ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ac7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42846/0 (socket says 172.21.15.5:42846) 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f163ffff700 1 -- 172.21.15.5:0/4070221697 learned_addr learned my addr 172.21.15.5:0/4070221697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f163f7fe700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aa8c60 0x5596f8ac61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 -- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aa8c60 msgr2=0x5596f8ac61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aa8c60 0x5596f8ac61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 -- 172.21.15.5:0/4070221697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 msgr2=0x5596f8ac7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ac7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 -- 172.21.15.5:0/4070221697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f8ac6ca0 con 0x5596f8aad470 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f163f7fe700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aa8c60 0x5596f8ac61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f163ffff700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ac7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f16448ba700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aad470 0x5596f8ac66e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f16380df2b0 tx=0x7f1638004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b9209aeb358980 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:42.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.313+0000 7f161f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f161f7fe700 1 -- 172.21.15.5:0/4070221697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16380ad280 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f161f7fe700 1 -- 172.21.15.5:0/4070221697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f16380b1810 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16340044e0 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f161f7fe700 1 -- 172.21.15.5:0/4070221697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16380b1ca0 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 10 client.16098 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596f8b2b960 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f161f7fe700 1 -- 172.21.15.5:0/4070221697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16340044e0 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f161f7fe700 1 -- 172.21.15.5:0/4070221697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1638012070 con 0x5596f8aad470 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 10 client.16098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 20 client.16098 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8b2c0c0 0x5596f8b2e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 4 client.16098 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.314+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596f89d22e0 con 0x5596f8b2c0c0 2024-05-14T15:27:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f163ffff700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8b2c0c0 0x5596f8b2e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f161f7fe700 10 client.16098 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f163ffff700 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8b2c0c0 0x5596f8b2e4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce49f1eef830a02d server_cookie=85260bb8f5032e3 in_seq=0 out_seq=0 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f161f7fe700 1 -- 172.21.15.5:0/4070221697 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596f89d22e0 con 0x5596f8b2c0c0 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f161f7fe700 10 client.16098 handle_command_reply: tid=0 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 client.16098 shutdown 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aad470 msgr2=0x5596f8ac66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aad470 0x5596f8ac66e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f16380df2b0 tx=0x7f1638004f90 comp rx=0 tx=0).stop 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 shutdown_connections 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f8aad470 0x5596f8ac66e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5596f8b2c0c0 0x5596f8b2e4e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f892ae40 0x5596f8ac7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 --2- 172.21.15.5:0/4070221697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596f8aa8c60 0x5596f8ac61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 >> 172.21.15.5:0/4070221697 conn(0x5596f8972600 msgr2=0x5596f8ab21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.315+0000 7f161f7fe700 0 client.16098 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:27:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 shutdown_connections 2024-05-14T15:27:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f1654d31b80 1 -- 172.21.15.5:0/4070221697 wait complete. 2024-05-14T15:27:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f1654d31b80 20 client.16098 trim_cache size 0 max 16384 2024-05-14T15:27:42.630 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f16280b29f0 msgr2=0x7f16280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f16280b29f0 0x7f16280b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1630003800 tx=0x7f1630005fb0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640108780 msgr2=0x7f164019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640108780 0x7f164019c750 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f163800ab40 tx=0x7f163800ae50 comp rx=0 tx=0).stop 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 shutdown_connections 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f16280b29f0 0x7f16280b4f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1640108780 0x7f164019c750 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1640103550 0x7f164019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 --2- 172.21.15.5:0/3722811167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1640107e30 0x7f164019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 >> 172.21.15.5:0/3722811167 conn(0x7f1640078e00 msgr2=0x7f1640100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 shutdown_connections 2024-05-14T15:27:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:42.316+0000 7f164631d700 1 -- 172.21.15.5:0/3722811167 wait complete. 2024-05-14T15:27:43.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 149010 12.85 MB/sec warmup 64 sec latency 40.836 ms 2024-05-14T15:27:43.203 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:27:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:43.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:43 smithi005 ceph-mon[31559]: pgmap v91: 129 pgs: 129 active+clean; 101 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 471 op/s 2024-05-14T15:27:44.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:43 smithi144 ceph-mon[35650]: pgmap v91: 129 pgs: 129 active+clean; 101 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 471 op/s 2024-05-14T15:27:44.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:43 smithi193 ceph-mon[36001]: pgmap v91: 129 pgs: 129 active+clean; 101 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.0 MiB/s wr, 471 op/s 2024-05-14T15:27:44.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150659 12.76 MB/sec warmup 65 sec latency 16.720 ms 2024-05-14T15:27:45.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 153124 12.78 MB/sec warmup 66 sec latency 12.274 ms 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 -- 172.21.15.5:0/3627703431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf4107df0 msgr2=0x7f8bf4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3627703431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf4107df0 0x7f8bf4108200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8be00099c0 tx=0x7f8be0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 -- 172.21.15.5:0/3627703431 shutdown_connections 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3627703431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf4108740 0x7f8bf41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3627703431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf41034c0 0x7f8bf41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3627703431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf4107df0 0x7f8bf4108200 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 -- 172.21.15.5:0/3627703431 >> 172.21.15.5:0/3627703431 conn(0x7f8bf40fedf0 msgr2=0x7f8bf4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 -- 172.21.15.5:0/3627703431 shutdown_connections 2024-05-14T15:27:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.291+0000 7f8bfad16700 1 -- 172.21.15.5:0/3627703431 wait complete. 2024-05-14T15:27:45.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.292+0000 7f8bfad16700 1 Processor -- start 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.292+0000 7f8bfad16700 1 -- start start 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bfad16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf41034c0 0x7f8bf419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bfad16700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 0x7f8bf419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bfad16700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf4108740 0x7f8bf419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bfad16700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4113350 con 0x7f8bf41034c0 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bfad16700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41131d0 con 0x7f8bf4108740 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bfad16700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41134d0 con 0x7f8bf4107df0 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 0x7f8bf419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 0x7f8bf419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36972/0 (socket says 172.21.15.5:36972) 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 -- 172.21.15.5:0/3788032131 learned_addr learned my addr 172.21.15.5:0/3788032131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf4108740 0x7f8bf419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf41034c0 0x7f8bf419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 -- 172.21.15.5:0/3788032131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf4108740 msgr2=0x7f8bf419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf4108740 0x7f8bf419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 -- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf41034c0 msgr2=0x7f8bf419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf41034c0 0x7f8bf419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.293+0000 7f8bf3fff700 1 -- 172.21.15.5:0/3788032131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf419fce0 con 0x7f8bf4107df0 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.294+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 0x7f8bf419b3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8be0005f50 tx=0x7f8be0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.294+0000 7f8bf1ffb700 1 -- 172.21.15.5:0/3788032131 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0026070 con 0x7f8bf4107df0 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.294+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf419ff70 con 0x7f8bf4107df0 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.294+0000 7f8bf1ffb700 1 -- 172.21.15.5:0/3788032131 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be0003c70 con 0x7f8bf4107df0 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.294+0000 7f8bf1ffb700 1 -- 172.21.15.5:0/3788032131 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be000fad0 con 0x7f8bf4107df0 2024-05-14T15:27:45.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.294+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a78f0 con 0x7f8bf4107df0 2024-05-14T15:27:45.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.298+0000 7f8bf1ffb700 1 -- 172.21.15.5:0/3788032131 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8be0029070 con 0x7f8bf4107df0 2024-05-14T15:27:45.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.298+0000 7f8bf1ffb700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8be40b2a70 0x7f8be40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.299+0000 7f8bf1ffb700 1 -- 172.21.15.5:0/3788032131 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f8be00e31e0 con 0x7f8bf4107df0 2024-05-14T15:27:45.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.299+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8be40b2a70 0x7f8be40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.299+0000 7f8c0972ab80 1 Processor -- start 2024-05-14T15:27:45.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.299+0000 7f8c0972ab80 1 -- start start 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8be40b2a70 0x7f8be40b4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009bf0 tx=0x7f8be8009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505843050 0x564505847320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x564505847c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564505760860 con 0x564505843050 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5645056ff7c0 con 0x564505847860 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56450575c820 con 0x5645056c0e40 2024-05-14T15:27:45.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf92b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x564505847c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505843050 0x564505847320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf92b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x564505847c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42874/0 (socket says 172.21.15.5:42874) 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf92b3700 1 -- 172.21.15.5:0/520034374 learned_addr learned my addr 172.21.15.5:0/520034374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf92b3700 1 -- 172.21.15.5:0/520034374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645056c0e40 msgr2=0x564505842b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.300+0000 7f8bf92b3700 1 -- 172.21.15.5:0/520034374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505843050 msgr2=0x564505847320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.301+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505843050 0x564505847320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.301+0000 7f8bf92b3700 1 -- 172.21.15.5:0/520034374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645056ad8d0 con 0x564505847860 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.301+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x564505847c70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bec011680 tx=0x7f8bec009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df81f0d819d3262 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.301+0000 7f8bdeffd700 1 -- 172.21.15.5:0/520034374 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec019030 con 0x564505847860 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.301+0000 7f8bdeffd700 1 -- 172.21.15.5:0/520034374 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec0092e0 con 0x564505847860 2024-05-14T15:27:45.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.301+0000 7f8bdeffd700 1 -- 172.21.15.5:0/520034374 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec007500 con 0x564505847860 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 -- 172.21.15.5:0/520034374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 msgr2=0x564505847c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x564505847c70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bec011680 tx=0x7f8bec009920 comp rx=0 tx=0).stop 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 -- 172.21.15.5:0/520034374 shutdown_connections 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x564505847c70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/520034374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505843050 0x564505847320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.302+0000 7f8c0972ab80 1 -- 172.21.15.5:0/520034374 >> 172.21.15.5:0/520034374 conn(0x564505708600 msgr2=0x5645058394b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.303+0000 7f8c0972ab80 1 -- 172.21.15.5:0/520034374 shutdown_connections 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.303+0000 7f8c0972ab80 1 -- 172.21.15.5:0/520034374 wait complete. 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.303+0000 7f8c0972ab80 1 -- start start 2024-05-14T15:27:45.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.303+0000 7f8c0972ab80 1 Processor -- start 2024-05-14T15:27:45.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x56450584c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505843050 0x56450584ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505847860 0x56450584cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56450584f7b0 con 0x5645056c0e40 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5645056c6c30 con 0x564505843050 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56450584f930 con 0x564505847860 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8bf8ab2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x56450584c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8c0972ab80 10 client.0 _ll_register_callbacks cb 0x5645056f8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8bf8ab2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x56450584c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55962/0 (socket says 172.21.15.5:55962) 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8bf8ab2700 1 -- 172.21.15.5:0/1285791619 learned_addr learned my addr 172.21.15.5:0/1285791619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.304+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505847860 0x56450584cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.305+0000 7f8bf92b3700 1 -- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505843050 msgr2=0x56450584ca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.305+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505843050 0x56450584ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.305+0000 7f8bf92b3700 1 -- 172.21.15.5:0/1285791619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 msgr2=0x56450584c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.305+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x56450584c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.305+0000 7f8bf92b3700 1 -- 172.21.15.5:0/1285791619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645056fc690 con 0x564505847860 2024-05-14T15:27:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.305+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x56450584c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.306+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505847860 0x56450584cf90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bec011680 tx=0x7f8bec003f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4801bb71b5dc7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8bd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1285791619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004cd0 con 0x564505847860 2024-05-14T15:27:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec0092e0 con 0x564505847860 2024-05-14T15:27:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1285791619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f8bec007500 con 0x564505847860 2024-05-14T15:27:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1285791619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bec02f050 con 0x564505847860 2024-05-14T15:27:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8c0972ab80 10 client.25747 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.307+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56450583f940 con 0x564505847860 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1285791619 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec0092e0 con 0x564505847860 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1285791619 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8bec0053c0 con 0x564505847860 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8c0972ab80 10 client.25747 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8c0972ab80 10 client.25747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8c0972ab80 20 client.25747 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x564505841e70 0x564505842280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8c0972ab80 4 client.25747 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56450576e240 con 0x564505841e70 2024-05-14T15:27:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.309+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x564505841e70 0x564505842280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.311+0000 7f8bd3fff700 10 client.25747 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.311+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x564505841e70 0x564505842280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a7ba0df4b0ff075 server_cookie=d8fc826e6fbfae81 in_seq=0 out_seq=0 2024-05-14T15:27:45.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.312+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1285791619 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56450576e240 con 0x564505841e70 2024-05-14T15:27:45.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.312+0000 7f8bd3fff700 10 client.25747 handle_command_reply: tid=0 2024-05-14T15:27:45.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.312+0000 7f8c0972ab80 1 client.25747 shutdown 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505847860 msgr2=0x56450584cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505847860 0x56450584cf90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bec011680 tx=0x7f8bec003f70 comp rx=0 tx=0).stop 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 shutdown_connections 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x564505841e70 0x564505842280 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505847860 0x56450584cf90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x56450584c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1285791619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505843050 0x56450584ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 >> 172.21.15.5:0/1285791619 conn(0x564505708600 msgr2=0x56450583d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8bd3fff700 0 client.25747 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 shutdown_connections 2024-05-14T15:27:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.313+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1285791619 wait complete. 2024-05-14T15:27:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.314+0000 7f8c0972ab80 20 client.25747 trim_cache size 0 max 16384 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 Processor -- start 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 -- start start 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505847150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x56450584ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41b0ce0 con 0x5645056c0e40 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ae570 con 0x564505847860 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ac3c0 con 0x564505843050 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55974/0 (socket says 172.21.15.5:55974) 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 -- 172.21.15.5:0/4282329766 learned_addr learned my addr 172.21.15.5:0/4282329766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x56450584ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505847150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 -- 172.21.15.5:0/4282329766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 msgr2=0x564505847150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505847150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 -- 172.21.15.5:0/4282329766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 msgr2=0x56450584ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x56450584ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf8ab2700 1 -- 172.21.15.5:0/4282329766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56450584ef90 con 0x5645056c0e40 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.338+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505847150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.339+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x564505842b10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8be80128b0 tx=0x7f8be8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7126df5099847b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.339+0000 7f8bde7fc700 1 -- 172.21.15.5:0/4282329766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be802a670 con 0x5645056c0e40 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.339+0000 7f8bde7fc700 1 -- 172.21.15.5:0/4282329766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be802a810 con 0x5645056c0e40 2024-05-14T15:27:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.339+0000 7f8bde7fc700 1 -- 172.21.15.5:0/4282329766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be801f680 con 0x5645056c0e40 2024-05-14T15:27:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 -- 172.21.15.5:0/4282329766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 msgr2=0x564505842b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x564505842b10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8be80128b0 tx=0x7f8be8014000 comp rx=0 tx=0).stop 2024-05-14T15:27:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 -- 172.21.15.5:0/4282329766 shutdown_connections 2024-05-14T15:27:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564505847860 0x56450584ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645056c0e40 0x564505842b10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/4282329766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505847150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 -- 172.21.15.5:0/4282329766 >> 172.21.15.5:0/4282329766 conn(0x564505708600 msgr2=0x564505846a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 -- 172.21.15.5:0/4282329766 shutdown_connections 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 -- 172.21.15.5:0/4282329766 wait complete. 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.340+0000 7f8c0972ab80 1 -- start start 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 Processor -- start 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645056c0e40 0x564505852b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505853070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505847860 0x5645058d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41aca30 con 0x564505847860 2024-05-14T15:27:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41adea0 con 0x5645056c0e40 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8c0972ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41ad100 con 0x564505843050 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505853070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.341+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505853070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37018/0 (socket says 172.21.15.5:37018) 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 -- 172.21.15.5:0/1061127707 learned_addr learned my addr 172.21.15.5:0/1061127707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645056c0e40 0x564505852b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf92b3700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505847860 0x5645058d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 -- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645056c0e40 msgr2=0x564505852b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645056c0e40 0x564505852b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 -- 172.21.15.5:0/1061127707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505847860 msgr2=0x5645058d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505847860 0x5645058d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 -- 172.21.15.5:0/1061127707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564505853920 con 0x564505843050 2024-05-14T15:27:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645056c0e40 0x564505852b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8c0972ab80 10 client.0 _ll_register_callbacks cb 0x5645056f8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bf3fff700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505853070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8be00e0620 tx=0x7f8be00aa6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3281c42f0118b64 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be802a810 con 0x564505843050 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1061127707 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be00a9900 con 0x564505843050 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.342+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1061127707 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f8be00039d0 con 0x564505843050 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1061127707 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8be0009e60 con 0x564505843050 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 10 client.25753 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5645058d28f0 con 0x564505843050 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1061127707 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8be00ae160 con 0x564505843050 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 10 client.25753 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 10 client.25753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 20 client.25753 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5645058d2ef0 0x5645058d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 4 client.25753 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.343+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8be802a810 con 0x5645058d2ef0 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.344+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5645058d2ef0 0x5645058d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.344+0000 7f8bd3fff700 10 client.25753 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.344+0000 7f8bf8ab2700 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5645058d2ef0 0x5645058d5330 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21d397380d8c042f server_cookie=8a356dc0ca9d9474 in_seq=0 out_seq=0 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.344+0000 7f8bd3fff700 1 -- 172.21.15.5:0/1061127707 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8be802a810 con 0x5645058d2ef0 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.344+0000 7f8bd3fff700 10 client.25753 handle_command_reply: tid=0 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.344+0000 7f8c0972ab80 1 client.25753 shutdown 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 msgr2=0x564505853070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505853070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8be00e0620 tx=0x7f8be00aa6d0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 shutdown_connections 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5645058d2ef0 0x5645058d5330 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564505847860 0x5645058d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8bd3fff700 0 client.25753 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645056c0e40 0x564505852b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 --2- 172.21.15.5:0/1061127707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564505843050 0x564505853070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 >> 172.21.15.5:0/1061127707 conn(0x564505708600 msgr2=0x564505838be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 shutdown_connections 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 1 -- 172.21.15.5:0/1061127707 wait complete. 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.345+0000 7f8c0972ab80 20 client.25753 trim_cache size 0 max 16384 2024-05-14T15:27:45.350 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8be40b2a70 msgr2=0x7f8be40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8be40b2a70 0x7f8be40b4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009bf0 tx=0x7f8be8009260 comp rx=0 tx=0).stop 2024-05-14T15:27:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 msgr2=0x7f8bf419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 0x7f8bf419b3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8be0005f50 tx=0x7f8be0004270 comp rx=0 tx=0).stop 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 shutdown_connections 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8be40b2a70 0x7f8be40b4f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf4108740 0x7f8bf419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf41034c0 0x7f8bf419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 --2- 172.21.15.5:0/3788032131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf4107df0 0x7f8bf419b3b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 >> 172.21.15.5:0/3788032131 conn(0x7f8bf40fedf0 msgr2=0x7f8bf4075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 shutdown_connections 2024-05-14T15:27:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:45.346+0000 7f8bfad16700 1 -- 172.21.15.5:0/3788032131 wait complete. 2024-05-14T15:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 -- 172.21.15.5:0/2217843634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 msgr2=0x7f250c1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 --2- 172.21.15.5:0/2217843634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c1106d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2504009ca0 tx=0x7f250402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 -- 172.21.15.5:0/2217843634 shutdown_connections 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 --2- 172.21.15.5:0/2217843634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c1106d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 --2- 172.21.15.5:0/2217843634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f250c10b7f0 0x7f250c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 --2- 172.21.15.5:0/2217843634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f250c104d90 0x7f250c1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 -- 172.21.15.5:0/2217843634 >> 172.21.15.5:0/2217843634 conn(0x7f250c0fe390 msgr2=0x7f250c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 -- 172.21.15.5:0/2217843634 shutdown_connections 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.794+0000 7f2512ab6640 1 -- 172.21.15.5:0/2217843634 wait complete. 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 Processor -- start 2024-05-14T15:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 -- start start 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f250c104d90 0x7f250c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f250c10b7f0 0x7f250c1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f250c113e40 con 0x7f250c10e1e0 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f250c113cc0 con 0x7f250c104d90 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2512ab6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f250c113fc0 con 0x7f250c10b7f0 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40140/0 (socket says 172.21.15.5:40140) 2024-05-14T15:27:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 -- 172.21.15.5:0/815952281 learned_addr learned my addr 172.21.15.5:0/815952281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 -- 172.21.15.5:0/815952281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f250c10b7f0 msgr2=0x7f250c1a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f250c10b7f0 0x7f250c1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 -- 172.21.15.5:0/815952281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f250c104d90 msgr2=0x7f250c1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.795+0000 7f2511053640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f250c104d90 0x7f250c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2511053640 1 -- 172.21.15.5:0/815952281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c19ae30 con 0x7f250c10e1e0 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2511053640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c19a700 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f25040098d0 tx=0x7f25040317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2509ffb640 1 -- 172.21.15.5:0/815952281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504003b70 con 0x7f250c10e1e0 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2509ffb640 1 -- 172.21.15.5:0/815952281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504003d10 con 0x7f250c10e1e0 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c19b060 con 0x7f250c10e1e0 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2509ffb640 1 -- 172.21.15.5:0/815952281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250402fa20 con 0x7f250c10e1e0 2024-05-14T15:27:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.796+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1a7320 con 0x7f250c10e1e0 2024-05-14T15:27:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.799+0000 7f251996d740 1 Processor -- start 2024-05-14T15:27:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.799+0000 7f2509ffb640 1 -- 172.21.15.5:0/815952281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f250402fc40 con 0x7f250c10e1e0 2024-05-14T15:27:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.799+0000 7f251996d740 1 -- start start 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f251996d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db86be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f251996d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7bd2f0 0x55645db87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f251996d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7c54d0 0x55645db87660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f251996d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55645daec5d0 con 0x55645d7e61b0 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f251996d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55645daa5890 con 0x55645d7bd2f0 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f251996d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55645dae8590 con 0x55645d7c54d0 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db86be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db86be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40142/0 (socket says 172.21.15.5:40142) 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 -- 172.21.15.5:0/339469991 learned_addr learned my addr 172.21.15.5:0/339469991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f2510852640 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7bd2f0 0x55645db87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7c54d0 0x55645db87660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f2509ffb640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f24fc0b2950 0x7f24fc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 -- 172.21.15.5:0/339469991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7c54d0 msgr2=0x55645db87660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7c54d0 0x55645db87660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 -- 172.21.15.5:0/339469991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7bd2f0 msgr2=0x55645db87120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7bd2f0 0x55645db87120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 -- 172.21.15.5:0/339469991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645d7fe210 con 0x55645d7e61b0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f2511053640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f24fc0b2950 0x7f24fc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f250bfff640 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db86be0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005bc0 tx=0x7f24f80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0dd45b05acaff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.800+0000 7f2509ffb640 1 -- 172.21.15.5:0/815952281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f2504103ee0 con 0x7f250c10e1e0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.801+0000 7f24f6ffd640 1 -- 172.21.15.5:0/339469991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8018030 con 0x55645d7e61b0 2024-05-14T15:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.801+0000 7f24f6ffd640 1 -- 172.21.15.5:0/339469991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f8010e40 con 0x55645d7e61b0 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.801+0000 7f24f6ffd640 1 -- 172.21.15.5:0/339469991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8012450 con 0x55645d7e61b0 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 -- 172.21.15.5:0/339469991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 msgr2=0x55645db86be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f2511053640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f24fc0b2950 0x7f24fc0b4e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2504101430 tx=0x7f2504098ae0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db86be0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005bc0 tx=0x7f24f80096c0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 -- 172.21.15.5:0/339469991 shutdown_connections 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7c54d0 0x55645db87660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7bd2f0 0x55645db87120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 --2- 172.21.15.5:0/339469991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db86be0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 -- 172.21.15.5:0/339469991 >> 172.21.15.5:0/339469991 conn(0x55645d7da300 msgr2=0x55645db83340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 -- 172.21.15.5:0/339469991 shutdown_connections 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 -- 172.21.15.5:0/339469991 wait complete. 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 -- start start 2024-05-14T15:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.802+0000 7f251996d740 1 Processor -- start 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645db916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db9d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55645db93d10 con 0x55645d7e61b0 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55645daa9370 con 0x55645d7c54d0 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55645db93e90 con 0x55645d7bd2f0 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f251996d740 10 client.0 _ll_register_callbacks cb 0x55645d7fda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db9d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db9d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40146/0 (socket says 172.21.15.5:40146) 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f250bfff640 1 -- 172.21.15.5:0/363987119 learned_addr learned my addr 172.21.15.5:0/363987119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f250bfff640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f2510852640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645db916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f2510852640 1 -- 172.21.15.5:0/363987119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 msgr2=0x55645db91180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f2510852640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db91180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f2510852640 1 -- 172.21.15.5:0/363987119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 msgr2=0x55645db9d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f2510852640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db9d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f2510852640 1 -- 172.21.15.5:0/363987119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645d8ca620 con 0x55645d7c54d0 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.803+0000 7f250bfff640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f2510852640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645db916c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55645db84320 tx=0x7f2500003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d19ac548c9cb5ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f24ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f24ebfff640 1 -- 172.21.15.5:0/363987119 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250001a470 con 0x55645d7c54d0 2024-05-14T15:27:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f8010e40 con 0x55645d7c54d0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f24ebfff640 1 -- 172.21.15.5:0/363987119 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f250001a610 con 0x55645d7c54d0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f24ebfff640 1 -- 172.21.15.5:0/363987119 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2500023050 con 0x55645d7c54d0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 10 client.25757 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55645dba00c0 con 0x55645d7c54d0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f24ebfff640 1 -- 172.21.15.5:0/363987119 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f8010e40 con 0x55645d7c54d0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f24ebfff640 1 -- 172.21.15.5:0/363987119 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f25000117e0 con 0x55645d7c54d0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 10 client.25757 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 20 client.25757 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7a4d80 0x55645dba3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.804+0000 7f251996d740 4 client.25757 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.805+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55645dafa1e0 con 0x55645d7a4d80 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.805+0000 7f250bfff640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7a4d80 0x55645dba3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.805+0000 7f24ebfff640 10 client.25757 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.805+0000 7f250bfff640 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7a4d80 0x55645dba3540 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3711fdffa02de624 server_cookie=51739972b2f14cbe in_seq=0 out_seq=0 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f24ebfff640 1 -- 172.21.15.5:0/363987119 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55645dafa1e0 con 0x55645d7a4d80 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f24ebfff640 10 client.25757 handle_command_reply: tid=0 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 client.25757 shutdown 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 msgr2=0x55645db916c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645db916c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55645db84320 tx=0x7f2500003c40 comp rx=0 tx=0).stop 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 shutdown_connections 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7a4d80 0x55645dba3540 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7e61b0 0x55645db9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f24ebfff640 0 client.25757 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645db916c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 --2- 172.21.15.5:0/363987119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db91180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 >> 172.21.15.5:0/363987119 conn(0x55645d7da300 msgr2=0x55645db86a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 shutdown_connections 2024-05-14T15:27:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 1 -- 172.21.15.5:0/363987119 wait complete. 2024-05-14T15:27:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.806+0000 7f251996d740 20 client.25757 trim_cache size 0 max 16384 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.821+0000 7f251996d740 1 Processor -- start 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.821+0000 7f251996d740 1 -- start start 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f251996d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7a4d80 0x55645db87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f251996d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645dba8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f251996d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 0x55645dba9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f251996d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1b3c10 con 0x55645d7c54d0 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f251996d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1b14e0 con 0x55645d7a4d80 2024-05-14T15:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f251996d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1af330 con 0x55645d7bd2f0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 0x55645dba9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 0x55645dba9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40160/0 (socket says 172.21.15.5:40160) 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 -- 172.21.15.5:0/1703394710 learned_addr learned my addr 172.21.15.5:0/1703394710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f2510852640 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645dba8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7a4d80 0x55645db87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 -- 172.21.15.5:0/1703394710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 msgr2=0x55645dba8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645dba8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 -- 172.21.15.5:0/1703394710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7a4d80 msgr2=0x55645db87080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7a4d80 0x55645db87080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 -- 172.21.15.5:0/1703394710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55645db93810 con 0x55645d7c54d0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f2510852640 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645dba8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.822+0000 7f250bfff640 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 0x55645dba9260 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f24f8003ad0 tx=0x7f24f801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2239c8c6f6724348 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.823+0000 7f24f67fc640 1 -- 172.21.15.5:0/1703394710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8018040 con 0x55645d7c54d0 2024-05-14T15:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.823+0000 7f24f67fc640 1 -- 172.21.15.5:0/1703394710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f80042e0 con 0x55645d7c54d0 2024-05-14T15:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.823+0000 7f24f67fc640 1 -- 172.21.15.5:0/1703394710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8026040 con 0x55645d7c54d0 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 -- 172.21.15.5:0/1703394710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 msgr2=0x55645dba9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 0x55645dba9260 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f24f8003ad0 tx=0x7f24f801f040 comp rx=0 tx=0).stop 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 -- 172.21.15.5:0/1703394710 shutdown_connections 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645dba8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7c54d0 0x55645dba9260 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 --2- 172.21.15.5:0/1703394710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7a4d80 0x55645db87080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 -- 172.21.15.5:0/1703394710 >> 172.21.15.5:0/1703394710 conn(0x55645d7da300 msgr2=0x55645db86970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 -- 172.21.15.5:0/1703394710 shutdown_connections 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 -- 172.21.15.5:0/1703394710 wait complete. 2024-05-14T15:27:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 -- start start 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.824+0000 7f251996d740 1 Processor -- start 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7a4d80 0x55645db98be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db99120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645dc06c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1af9a0 con 0x55645d7a4d80 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1b0e10 con 0x55645d7c54d0 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1b0070 con 0x55645d7bd2f0 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f251996d740 10 client.0 _ll_register_callbacks cb 0x7f2500004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7a4d80 0x55645db98be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db99120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f250bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7a4d80 0x55645db98be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40172/0 (socket says 172.21.15.5:40172) 2024-05-14T15:27:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f250bfff640 1 -- 172.21.15.5:0/1533192996 learned_addr learned my addr 172.21.15.5:0/1533192996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f250bfff640 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645dc06c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 -- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 msgr2=0x55645dc06c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645dc06c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 -- 172.21.15.5:0/1533192996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7a4d80 msgr2=0x55645db98be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7a4d80 0x55645db98be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 -- 172.21.15.5:0/1533192996 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645dc071f0 con 0x55645d7bd2f0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f2510852640 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db99120 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2500000c00 tx=0x7f250000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42f34a61d99e62e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.825+0000 7f24ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.826+0000 7f24ebfff640 1 -- 172.21.15.5:0/1533192996 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500005850 con 0x55645d7bd2f0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.826+0000 7f24ebfff640 1 -- 172.21.15.5:0/1533192996 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f2500011650 con 0x55645d7bd2f0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.826+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f80042e0 con 0x55645d7bd2f0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.826+0000 7f24ebfff640 1 -- 172.21.15.5:0/1533192996 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2500037050 con 0x55645d7bd2f0 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.826+0000 7f251996d740 10 client.25768 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.826+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55645dc0f760 con 0x55645d7bd2f0 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f24ebfff640 1 -- 172.21.15.5:0/1533192996 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f80042e0 con 0x55645d7bd2f0 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f24ebfff640 1 -- 172.21.15.5:0/1533192996 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2500030060 con 0x55645d7bd2f0 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f251996d740 10 client.25768 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f251996d740 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f251996d740 20 client.25768 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f251996d740 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7e61b0 0x55645dc11fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f251996d740 4 client.25768 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55645daf4050 con 0x55645d7e61b0 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f250bfff640 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7e61b0 0x55645dc11fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.827+0000 7f24ebfff640 10 client.25768 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.828+0000 7f250bfff640 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7e61b0 0x55645dc11fa0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4ad40556238f9f server_cookie=7c17c13286bc77a1 in_seq=0 out_seq=0 2024-05-14T15:27:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f24ebfff640 1 -- 172.21.15.5:0/1533192996 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55645daf4050 con 0x55645d7e61b0 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f24ebfff640 10 client.25768 handle_command_reply: tid=0 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 client.25768 shutdown 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 msgr2=0x55645db99120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db99120 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2500000c00 tx=0x7f250000bf60 comp rx=0 tx=0).stop 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 shutdown_connections 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55645d7e61b0 0x55645dc11fa0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55645d7bd2f0 0x55645db99120 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645d7c54d0 0x55645dc06c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f24ebfff640 0 client.25768 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 --2- 172.21.15.5:0/1533192996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55645d7a4d80 0x55645db98be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 >> 172.21.15.5:0/1533192996 conn(0x55645d7da300 msgr2=0x55645db92430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 shutdown_connections 2024-05-14T15:27:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 1 -- 172.21.15.5:0/1533192996 wait complete. 2024-05-14T15:27:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.950+0000 7f251996d740 20 client.25768 trim_cache size 0 max 16384 2024-05-14T15:27:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:27:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:27:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:27:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f24fc0b2950 msgr2=0x7f24fc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f24fc0b2950 0x7f24fc0b4e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2504101430 tx=0x7f2504098ae0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 msgr2=0x7f250c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c19a700 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f25040098d0 tx=0x7f25040317b0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 shutdown_connections 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f24fc0b2950 0x7f24fc0b4e60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f250c10b7f0 0x7f250c1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c10e1e0 0x7f250c19a700 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 --2- 172.21.15.5:0/815952281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f250c104d90 0x7f250c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 >> 172.21.15.5:0/815952281 conn(0x7f250c0fe390 msgr2=0x7f250c10e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 shutdown_connections 2024-05-14T15:27:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:45.951+0000 7f2512ab6640 1 -- 172.21.15.5:0/815952281 wait complete. 2024-05-14T15:27:46.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:45 smithi005 ceph-mon[31559]: pgmap v92: 129 pgs: 129 active+clean; 101 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 431 op/s 2024-05-14T15:27:46.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:45 smithi144 ceph-mon[35650]: pgmap v92: 129 pgs: 129 active+clean; 101 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 431 op/s 2024-05-14T15:27:46.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:45 smithi193 ceph-mon[36001]: pgmap v92: 129 pgs: 129 active+clean; 101 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 431 op/s 2024-05-14T15:27:46.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 155750 12.78 MB/sec warmup 67 sec latency 44.809 ms 2024-05-14T15:27:46.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:27:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:47.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 157683 12.76 MB/sec warmup 68 sec latency 20.539 ms 2024-05-14T15:27:47.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:47 smithi005 ceph-mon[31559]: pgmap v93: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 686 op/s 2024-05-14T15:27:48.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:47 smithi144 ceph-mon[35650]: pgmap v93: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 686 op/s 2024-05-14T15:27:48.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:47 smithi193 ceph-mon[36001]: pgmap v93: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.3 MiB/s wr, 686 op/s 2024-05-14T15:27:48.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 160248 12.78 MB/sec warmup 69 sec latency 19.300 ms 2024-05-14T15:27:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 -- 172.21.15.5:0/2093109454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c108740 msgr2=0x7f279c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 --2- 172.21.15.5:0/2093109454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c108740 0x7f279c1100a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f279400b0d0 tx=0x7f279400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 -- 172.21.15.5:0/2093109454 shutdown_connections 2024-05-14T15:27:48.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 --2- 172.21.15.5:0/2093109454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c108740 0x7f279c1100a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 --2- 172.21.15.5:0/2093109454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c1034c0 0x7f279c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 --2- 172.21.15.5:0/2093109454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c107df0 0x7f279c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 -- 172.21.15.5:0/2093109454 >> 172.21.15.5:0/2093109454 conn(0x7f279c0fedc0 msgr2=0x7f279c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:48.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 -- 172.21.15.5:0/2093109454 shutdown_connections 2024-05-14T15:27:48.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.149+0000 7f27a1942700 1 -- 172.21.15.5:0/2093109454 wait complete. 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.150+0000 7f27a1942700 1 Processor -- start 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.150+0000 7f27a1942700 1 -- start start 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a1942700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c1034c0 0x7f279c0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a1942700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 0x7f279c0ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a1942700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c108740 0x7f279c0ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a1942700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f279c113370 con 0x7f279c107df0 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a1942700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1131f0 con 0x7f279c1034c0 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a1942700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1134f0 con 0x7f279c108740 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 0x7f279c0ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f279affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c1034c0 0x7f279c0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 0x7f279c0ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40186/0 (socket says 172.21.15.5:40186) 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 -- 172.21.15.5:0/241214211 learned_addr learned my addr 172.21.15.5:0/241214211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f279b7fe700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c108740 0x7f279c0ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 -- 172.21.15.5:0/241214211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c108740 msgr2=0x7f279c0ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c108740 0x7f279c0ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 -- 172.21.15.5:0/241214211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c1034c0 msgr2=0x7f279c0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c1034c0 0x7f279c0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 -- 172.21.15.5:0/241214211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c1a9170 con 0x7f279c107df0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f279b7fe700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c108740 0x7f279c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27927fc700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 0x7f279c0ff7e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f278400c8f0 tx=0x7f278400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.151+0000 7f27a0940700 1 -- 172.21.15.5:0/241214211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784007940 con 0x7f279c107df0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.152+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c1a9460 con 0x7f279c107df0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.152+0000 7f27a0940700 1 -- 172.21.15.5:0/241214211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278400f450 con 0x7f279c107df0 2024-05-14T15:27:48.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.152+0000 7f27a0940700 1 -- 172.21.15.5:0/241214211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784018660 con 0x7f279c107df0 2024-05-14T15:27:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.152+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c1a99f0 con 0x7f279c107df0 2024-05-14T15:27:48.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.154+0000 7f27a0940700 1 -- 172.21.15.5:0/241214211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2784018800 con 0x7f279c107df0 2024-05-14T15:27:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.155+0000 7f27a0940700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f27880b29f0 0x7f27880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.155+0000 7f279affd700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f27880b29f0 0x7f27880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.155+0000 7f27a0940700 1 -- 172.21.15.5:0/241214211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f27840e2850 con 0x7f279c107df0 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f27880b29f0 0x7f27880b4f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f278c009c60 tx=0x7f278c009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 Processor -- start 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 -- start start 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe06720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe0af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5634afd28860 con 0x5634afe06c60 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634afcc77c0 con 0x5634afc88e40 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27b0356b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634afd24820 con 0x5634afe0b470 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe0af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe06720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe0af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40200/0 (socket says 172.21.15.5:40200) 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f27927fc700 1 -- 172.21.15.5:0/1327793326 learned_addr learned my addr 172.21.15.5:0/1327793326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279b7fe700 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 -- 172.21.15.5:0/1327793326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 msgr2=0x5634afe0b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 -- 172.21.15.5:0/1327793326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 msgr2=0x5634afe0af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe0af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.156+0000 7f279affd700 1 -- 172.21.15.5:0/1327793326 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634afc758d0 con 0x5634afc88e40 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.157+0000 7f279affd700 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe06720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e3b0 tx=0x7f278c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=808a5bdff5043c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.157+0000 7f2791ffb700 1 -- 172.21.15.5:0/1327793326 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00ed50 con 0x5634afc88e40 2024-05-14T15:27:48.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.157+0000 7f2791ffb700 1 -- 172.21.15.5:0/1327793326 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c0044e0 con 0x5634afc88e40 2024-05-14T15:27:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.157+0000 7f2791ffb700 1 -- 172.21.15.5:0/1327793326 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0266d0 con 0x5634afc88e40 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 -- 172.21.15.5:0/1327793326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 msgr2=0x5634afe06720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe06720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e3b0 tx=0x7f278c014000 comp rx=0 tx=0).stop 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 -- 172.21.15.5:0/1327793326 shutdown_connections 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe06720 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1327793326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe0af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 -- 172.21.15.5:0/1327793326 >> 172.21.15.5:0/1327793326 conn(0x5634afcd0600 msgr2=0x5634afe014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:48.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 -- 172.21.15.5:0/1327793326 shutdown_connections 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 -- 172.21.15.5:0/1327793326 wait complete. 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.158+0000 7f27b0356b80 1 -- start start 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 Processor -- start 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe10100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe10640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5634afe13420 con 0x5634afe06c60 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634afc8ec30 con 0x5634afc88e40 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634afe135a0 con 0x5634afe0b470 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27b0356b80 10 client.0 _ll_register_callbacks cb 0x5634afcc0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe10640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe10640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40202/0 (socket says 172.21.15.5:40202) 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27927fc700 1 -- 172.21.15.5:0/1312987688 learned_addr learned my addr 172.21.15.5:0/1312987688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279affd700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279b7fe700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279affd700 1 -- 172.21.15.5:0/1312987688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 msgr2=0x5634afe10b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279affd700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe10b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279affd700 1 -- 172.21.15.5:0/1312987688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 msgr2=0x5634afe10640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279affd700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe10640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279affd700 1 -- 172.21.15.5:0/1312987688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634afcc4690 con 0x5634afc88e40 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f27927fc700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe10640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.159+0000 7f279b7fe700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f279affd700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe10100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f278c0097c0 tx=0x7f278c026540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ce3d9fd7280edd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f277affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f277affd700 1 -- 172.21.15.5:0/1312987688 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c034060 con 0x5634afc88e40 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f277affd700 1 -- 172.21.15.5:0/1312987688 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f278c026de0 con 0x5634afc88e40 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278c0044e0 con 0x5634afc88e40 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f277affd700 1 -- 172.21.15.5:0/1312987688 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f278c03b050 con 0x5634afc88e40 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f27b0356b80 10 client.25775 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:48.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5634afe1dfa0 con 0x5634afc88e40 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.160+0000 7f277affd700 1 -- 172.21.15.5:0/1312987688 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c0044e0 con 0x5634afc88e40 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f277affd700 1 -- 172.21.15.5:0/1312987688 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f278c01d410 con 0x5634afc88e40 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27b0356b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27b0356b80 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27b0356b80 20 client.25775 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe20520 0x5634afe22940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27b0356b80 4 client.25775 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634afd36240 con 0x5634afe20520 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27927fc700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe20520 0x5634afe22940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f27927fc700 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe20520 0x5634afe22940 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363963dd0ab1afcd server_cookie=4099445b9a49ddcf in_seq=0 out_seq=0 2024-05-14T15:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.161+0000 7f277affd700 10 client.25775 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f277affd700 1 -- 172.21.15.5:0/1312987688 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634afd36240 con 0x5634afe20520 2024-05-14T15:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f277affd700 10 client.25775 handle_command_reply: tid=0 2024-05-14T15:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 client.25775 shutdown 2024-05-14T15:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 msgr2=0x5634afe10100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe10100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f278c0097c0 tx=0x7f278c026540 comp rx=0 tx=0).stop 2024-05-14T15:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 shutdown_connections 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe20520 0x5634afe22940 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe10100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1312987688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe10640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f277affd700 0 client.25775 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 >> 172.21.15.5:0/1312987688 conn(0x5634afcd0600 msgr2=0x5634afe05450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 shutdown_connections 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 1 -- 172.21.15.5:0/1312987688 wait complete. 2024-05-14T15:27:48.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.163+0000 7f27b0356b80 20 client.25775 trim_cache size 0 max 16384 2024-05-14T15:27:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.182+0000 7f27b0356b80 1 Processor -- start 2024-05-14T15:27:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.182+0000 7f27b0356b80 1 -- start start 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe0f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe1c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27b0356b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1b5100 con 0x5634afe06c60 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27b0356b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1b2990 con 0x5634afc88e40 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27b0356b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1b07e0 con 0x5634afe0b470 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe0f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe1c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe0f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:56742/0 (socket says 172.21.15.5:56742) 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 -- 172.21.15.5:0/1174658826 learned_addr learned my addr 172.21.15.5:0/1174658826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279b7fe700 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 -- 172.21.15.5:0/1174658826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 msgr2=0x5634afe0adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 -- 172.21.15.5:0/1174658826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 msgr2=0x5634afe1c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe1c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.183+0000 7f279affd700 1 -- 172.21.15.5:0/1174658826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634afe0e690 con 0x5634afc88e40 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.184+0000 7f279affd700 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe0f440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f278c01d7e0 tx=0x7f278c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2631d05e8807414b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.184+0000 7f27917fa700 1 -- 172.21.15.5:0/1174658826 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c044070 con 0x5634afc88e40 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.184+0000 7f27917fa700 1 -- 172.21.15.5:0/1174658826 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c00e9a0 con 0x5634afc88e40 2024-05-14T15:27:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.184+0000 7f27917fa700 1 -- 172.21.15.5:0/1174658826 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c034050 con 0x5634afc88e40 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 -- 172.21.15.5:0/1174658826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 msgr2=0x5634afe0f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe0f440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f278c01d7e0 tx=0x7f278c009750 comp rx=0 tx=0).stop 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 -- 172.21.15.5:0/1174658826 shutdown_connections 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe0adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe0f440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1174658826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe1c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 -- 172.21.15.5:0/1174658826 >> 172.21.15.5:0/1174658826 conn(0x5634afcd0600 msgr2=0x5634afe00be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 -- 172.21.15.5:0/1174658826 shutdown_connections 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.185+0000 7f27b0356b80 1 -- 172.21.15.5:0/1174658826 wait complete. 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 -- start start 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 Processor -- start 2024-05-14T15:27:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe25820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe25d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1b0e50 con 0x5634afe06c60 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1b22c0 con 0x5634afc88e40 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1b1520 con 0x5634afe0b470 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.186+0000 7f27b0356b80 10 client.0 _ll_register_callbacks cb 0x5634afcc0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe25d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe25820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f27927fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe25d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40212/0 (socket says 172.21.15.5:40212) 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f27927fc700 1 -- 172.21.15.5:0/1908085556 learned_addr learned my addr 172.21.15.5:0/1908085556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279b7fe700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 -- 172.21.15.5:0/1908085556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 msgr2=0x5634afe22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 -- 172.21.15.5:0/1908085556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 msgr2=0x5634afe25d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe25d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 -- 172.21.15.5:0/1908085556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634afe233f0 con 0x5634afc88e40 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279b7fe700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f277affd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.187+0000 7f279affd700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe25820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f278c042040 tx=0x7f278c009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef60ecc5cb4549e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f277affd700 1 -- 172.21.15.5:0/1908085556 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c044030 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f277affd700 1 -- 172.21.15.5:0/1908085556 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f278c01d410 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278c00e9a0 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f277affd700 1 -- 172.21.15.5:0/1908085556 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f278c034040 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f27b0356b80 10 client.25781 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5634afe8a3d0 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.188+0000 7f277affd700 1 -- 172.21.15.5:0/1908085556 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c00e9a0 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f277affd700 1 -- 172.21.15.5:0/1908085556 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f278c003710 con 0x5634afc88e40 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27b0356b80 10 client.25781 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27b0356b80 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27b0356b80 20 client.25781 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe8a830 0x5634afe8cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27b0356b80 4 client.25781 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:27:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634afd302e0 con 0x5634afe8a830 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27927fc700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe8a830 0x5634afe8cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f277affd700 10 client.25781 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.189+0000 7f27927fc700 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe8a830 0x5634afe8cc50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beea7926abc8cb4c server_cookie=5b7feae0682c2ad9 in_seq=0 out_seq=0 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f277affd700 1 -- 172.21.15.5:0/1908085556 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634afd302e0 con 0x5634afe8a830 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f277affd700 10 client.25781 handle_command_reply: tid=0 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 client.25781 shutdown 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 msgr2=0x5634afe25820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe25820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f278c042040 tx=0x7f278c009fa0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 shutdown_connections 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5634afe8a830 0x5634afe8cc50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634afe0b470 0x5634afe22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634afc88e40 0x5634afe25820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 --2- 172.21.15.5:0/1908085556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634afe06c60 0x5634afe25d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 >> 172.21.15.5:0/1908085556 conn(0x5634afcd0600 msgr2=0x5634afe10a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f277affd700 0 client.25781 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 shutdown_connections 2024-05-14T15:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.190+0000 7f27b0356b80 1 -- 172.21.15.5:0/1908085556 wait complete. 2024-05-14T15:27:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27b0356b80 20 client.25781 trim_cache size 0 max 16384 2024-05-14T15:27:48.195 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f27880b29f0 msgr2=0x7f27880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f27880b29f0 0x7f27880b4f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f278c009c60 tx=0x7f278c009260 comp rx=0 tx=0).stop 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 msgr2=0x7f279c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 0x7f279c0ff7e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f278400c8f0 tx=0x7f278400cc00 comp rx=0 tx=0).stop 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 shutdown_connections 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f27880b29f0 0x7f27880b4f00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f279c108740 0x7f279c0ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f279c1034c0 0x7f279c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 --2- 172.21.15.5:0/241214211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f279c107df0 0x7f279c0ff7e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 >> 172.21.15.5:0/241214211 conn(0x7f279c0fedc0 msgr2=0x7f279c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.191+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 shutdown_connections 2024-05-14T15:27:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:48.192+0000 7f27a1942700 1 -- 172.21.15.5:0/241214211 wait complete. 2024-05-14T15:27:49.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 163132 12.84 MB/sec warmup 70 sec latency 18.272 ms 2024-05-14T15:27:50.015 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:27:50.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:27:50.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:49 smithi005 ceph-mon[31559]: pgmap v94: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 467 op/s 2024-05-14T15:27:50.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:49 smithi144 ceph-mon[35650]: pgmap v94: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 467 op/s 2024-05-14T15:27:50.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:49 smithi193 ceph-mon[36001]: pgmap v94: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 467 op/s 2024-05-14T15:27:50.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 165026 12.76 MB/sec warmup 71 sec latency 15.706 ms 2024-05-14T15:27:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:51.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 167397 12.78 MB/sec warmup 72 sec latency 11.658 ms 2024-05-14T15:27:52.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:51 smithi005 ceph-mon[31559]: pgmap v95: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 467 op/s 2024-05-14T15:27:52.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:51 smithi144 ceph-mon[35650]: pgmap v95: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 467 op/s 2024-05-14T15:27:52.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:51 smithi193 ceph-mon[36001]: pgmap v95: 129 pgs: 129 active+clean; 26 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.6 MiB/s wr, 467 op/s 2024-05-14T15:27:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 -- 172.21.15.5:0/2110256617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c108760 msgr2=0x7fc60c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 --2- 172.21.15.5:0/2110256617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c108760 0x7fc60c1100c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc60400b0d0 tx=0x7fc60400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 -- 172.21.15.5:0/2110256617 shutdown_connections 2024-05-14T15:27:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 --2- 172.21.15.5:0/2110256617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c108760 0x7fc60c1100c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 --2- 172.21.15.5:0/2110256617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c1034e0 0x7fc60c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 --2- 172.21.15.5:0/2110256617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 -- 172.21.15.5:0/2110256617 >> 172.21.15.5:0/2110256617 conn(0x7fc60c0fef30 msgr2=0x7fc60c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 -- 172.21.15.5:0/2110256617 shutdown_connections 2024-05-14T15:27:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.062+0000 7fc611431700 1 -- 172.21.15.5:0/2110256617 wait complete. 2024-05-14T15:27:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 Processor -- start 2024-05-14T15:27:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 -- start start 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c1034e0 0x7fc60c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c108760 0x7fc60c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c113390 con 0x7fc60c1034e0 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c113210 con 0x7fc60c108760 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc611431700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c113510 con 0x7fc60c107e10 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc60a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc60affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c1034e0 0x7fc60c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc60a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c19c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39706/0 (socket says 172.21.15.5:39706) 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc60a7fc700 1 -- 172.21.15.5:0/3582874612 learned_addr learned my addr 172.21.15.5:0/3582874612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.063+0000 7fc60b7fe700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c108760 0x7fc60c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc60a7fc700 1 -- 172.21.15.5:0/3582874612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c108760 msgr2=0x7fc60c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c108760 0x7fc60c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc60a7fc700 1 -- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c1034e0 msgr2=0x7fc60c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c1034e0 0x7fc60c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc60a7fc700 1 -- 172.21.15.5:0/3582874612 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c1a7040 con 0x7fc60c107e10 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c19c230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e970 tx=0x7fc60000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc5f3fff700 1 -- 172.21.15.5:0/3582874612 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600004a30 con 0x7fc60c107e10 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c199b40 con 0x7fc60c107e10 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc5f3fff700 1 -- 172.21.15.5:0/3582874612 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60000cc90 con 0x7fc60c107e10 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc5f3fff700 1 -- 172.21.15.5:0/3582874612 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600005610 con 0x7fc60c107e10 2024-05-14T15:27:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.064+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c19a080 con 0x7fc60c107e10 2024-05-14T15:27:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.067+0000 7fc5f3fff700 1 -- 172.21.15.5:0/3582874612 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc600021440 con 0x7fc60c107e10 2024-05-14T15:27:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.068+0000 7fc5f3fff700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5f40b29f0 0x7fc5f40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.068+0000 7fc5f3fff700 1 -- 172.21.15.5:0/3582874612 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc6000e2b10 con 0x7fc60c107e10 2024-05-14T15:27:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.068+0000 7fc60affd700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5f40b29f0 0x7fc5f40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60affd700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5f40b29f0 0x7fc5f40b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc010d80 tx=0x7fc5fc012040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 Processor -- start 2024-05-14T15:27:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 -- start start 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d61678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d6161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d61622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2d60839c0 con 0x55b2d6161e90 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2d5fe8e20 con 0x55b2d6167e00 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b2d607f980 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d61622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d6161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d61678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d6161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:56760/0 (socket says 172.21.15.5:56760) 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60a7fc700 1 -- 172.21.15.5:0/2973905472 learned_addr learned my addr 172.21.15.5:0/2973905472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.069+0000 7fc60affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d61678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39714/0 (socket says 172.21.15.5:39714) 2024-05-14T15:27:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc60affd700 1 -- 172.21.15.5:0/2973905472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 msgr2=0x55b2d6161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc60affd700 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d6161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc60affd700 1 -- 172.21.15.5:0/2973905472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 msgr2=0x55b2d61622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc60affd700 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d61622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc60affd700 1 -- 172.21.15.5:0/2973905472 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d5fd0950 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc60affd700 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d61678c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009c30 tx=0x7fc5fc018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6c5371c406467f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc5f17fa700 1 -- 172.21.15.5:0/2973905472 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc02b070 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc5f17fa700 1 -- 172.21.15.5:0/2973905472 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5fc0044e0 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.070+0000 7fc5f17fa700 1 -- 172.21.15.5:0/2973905472 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc022040 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2973905472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 msgr2=0x55b2d61678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d61678c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009c30 tx=0x7fc5fc018000 comp rx=0 tx=0).stop 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2973905472 shutdown_connections 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d61622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d61678c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2973905472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d6161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2973905472 >> 172.21.15.5:0/2973905472 conn(0x55b2d602b6b0 msgr2=0x55b2d615ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2973905472 shutdown_connections 2024-05-14T15:27:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.071+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2973905472 wait complete. 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 -- start start 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 Processor -- start 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d5fe3ec0 0x55b2d616b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6161e90 0x55b2d616ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d616bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2d616e8a0 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2d603cf50 con 0x55b2d6167e00 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b2d616ea20 con 0x55b2d6161e90 2024-05-14T15:27:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.072+0000 7fc61fe45b80 10 client.0 _ll_register_callbacks cb 0x55b2d601b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d616bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d616bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:56762/0 (socket says 172.21.15.5:56762) 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60b7fe700 1 -- 172.21.15.5:0/675814806 learned_addr learned my addr 172.21.15.5:0/675814806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6161e90 0x55b2d616ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60affd700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d5fe3ec0 0x55b2d616b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 -- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 msgr2=0x55b2d616bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d616bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 -- 172.21.15.5:0/675814806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d5fe3ec0 msgr2=0x55b2d616b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d5fe3ec0 0x55b2d616b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 -- 172.21.15.5:0/675814806 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d603a7e0 con 0x55b2d6161e90 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6161e90 0x55b2d616ba60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc600009230 tx=0x7fc600004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d4020082524071 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc5eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.073+0000 7fc5eeff5700 1 -- 172.21.15.5:0/675814806 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000c3060 con 0x55b2d6161e90 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5fc0044e0 con 0x55b2d6161e90 2024-05-14T15:27:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc5eeff5700 1 -- 172.21.15.5:0/675814806 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc6000b2610 con 0x55b2d6161e90 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc5eeff5700 1 -- 172.21.15.5:0/675814806 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6000a5040 con 0x55b2d6161e90 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 10 client.25789 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2d61794e0 con 0x55b2d6161e90 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc5eeff5700 1 -- 172.21.15.5:0/675814806 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5fc0044e0 con 0x55b2d6161e90 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc5eeff5700 1 -- 172.21.15.5:0/675814806 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc6000cace0 con 0x55b2d6161e90 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 20 client.25789 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d617bb20 0x55b2d617df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 4 client.25789 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2d60913a0 con 0x55b2d617bb20 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.074+0000 7fc60affd700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d617bb20 0x55b2d617df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.075+0000 7fc60affd700 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d617bb20 0x55b2d617df40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb43e68bc0e0fb server_cookie=de4b0492a50a0000 in_seq=0 out_seq=0 2024-05-14T15:27:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.075+0000 7fc5eeff5700 10 client.25789 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc5eeff5700 1 -- 172.21.15.5:0/675814806 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2d60913a0 con 0x55b2d617bb20 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc5eeff5700 10 client.25789 handle_command_reply: tid=0 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 client.25789 shutdown 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6161e90 msgr2=0x55b2d616ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6161e90 0x55b2d616ba60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc600009230 tx=0x7fc600004e70 comp rx=0 tx=0).stop 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 shutdown_connections 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d617bb20 0x55b2d617df40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6167e00 0x55b2d616bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d5fe3ec0 0x55b2d616b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/675814806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6161e90 0x55b2d616ba60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc5eeff5700 0 client.25789 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 >> 172.21.15.5:0/675814806 conn(0x55b2d602b6b0 msgr2=0x55b2d6160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 shutdown_connections 2024-05-14T15:27:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.077+0000 7fc61fe45b80 1 -- 172.21.15.5:0/675814806 wait complete. 2024-05-14T15:27:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.078+0000 7fc61fe45b80 20 client.25789 trim_cache size 0 max 16384 2024-05-14T15:27:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 Processor -- start 2024-05-14T15:27:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 -- start start 2024-05-14T15:27:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d616d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6167e00 0x55b2d6177930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1b0db0 con 0x55b2d6161e90 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1ae640 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1ac490 con 0x55b2d6167e00 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc60a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d616d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc60affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc60affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:56772/0 (socket says 172.21.15.5:56772) 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc60affd700 1 -- 172.21.15.5:0/974479267 learned_addr learned my addr 172.21.15.5:0/974479267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.101+0000 7fc60b7fe700 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6167e00 0x55b2d6177930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60a7fc700 1 -- 172.21.15.5:0/974479267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6167e00 msgr2=0x55b2d6177930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6167e00 0x55b2d6177930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60a7fc700 1 -- 172.21.15.5:0/974479267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 msgr2=0x55b2d6161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60a7fc700 1 -- 172.21.15.5:0/974479267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d6167410 con 0x55b2d6161e90 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60affd700 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d616d2e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc6000a9730 tx=0x7fc6000a5310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e415096c31eab94f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc5f0ff9700 1 -- 172.21.15.5:0/974479267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000c9070 con 0x55b2d6161e90 2024-05-14T15:27:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc5f0ff9700 1 -- 172.21.15.5:0/974479267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6000b21d0 con 0x55b2d6161e90 2024-05-14T15:27:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.102+0000 7fc5f0ff9700 1 -- 172.21.15.5:0/974479267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6000aa130 con 0x55b2d6161e90 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 -- 172.21.15.5:0/974479267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 msgr2=0x55b2d616d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d616d2e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc6000a9730 tx=0x7fc6000a5310 comp rx=0 tx=0).stop 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 -- 172.21.15.5:0/974479267 shutdown_connections 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d6167e00 0x55b2d6177930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/974479267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6161e90 0x55b2d616d2e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 -- 172.21.15.5:0/974479267 >> 172.21.15.5:0/974479267 conn(0x55b2d602b6b0 msgr2=0x55b2d616cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.103+0000 7fc61fe45b80 1 -- 172.21.15.5:0/974479267 shutdown_connections 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.104+0000 7fc61fe45b80 1 -- 172.21.15.5:0/974479267 wait complete. 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.104+0000 7fc61fe45b80 1 -- start start 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.104+0000 7fc61fe45b80 1 Processor -- start 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6173eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6161e90 0x55b2d6170c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 0x55b2d61711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1acb00 con 0x55b2d6167e00 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1adf70 con 0x55b2d6161e90 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1ad1d0 con 0x55b2d5fe3ec0 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 0x55b2d61711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 0x55b2d61711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40276/0 (socket says 172.21.15.5:40276) 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 -- 172.21.15.5:0/2008842761 learned_addr learned my addr 172.21.15.5:0/2008842761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc61fe45b80 10 client.0 _ll_register_callbacks cb 0x55b2d601b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60affd700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6173eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60a7fc700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6161e90 0x55b2d6170c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 -- 172.21.15.5:0/2008842761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 msgr2=0x55b2d6173eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6173eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 -- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6161e90 msgr2=0x55b2d6170c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6161e90 0x55b2d6170c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.105+0000 7fc60b7fe700 1 -- 172.21.15.5:0/2008842761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d61746b0 con 0x55b2d6167e00 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc60b7fe700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 0x55b2d61711c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc60400b0d0 tx=0x7fc604009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76581e464cf89584 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc5eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc5eeff5700 1 -- 172.21.15.5:0/2008842761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60402c060 con 0x55b2d6167e00 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc5eeff5700 1 -- 172.21.15.5:0/2008842761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc6040075c0 con 0x55b2d6167e00 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b2d608b440 con 0x55b2d6167e00 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc5eeff5700 1 -- 172.21.15.5:0/2008842761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc604033050 con 0x55b2d6167e00 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc61fe45b80 10 client.16170 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2d61e6c00 con 0x55b2d6167e00 2024-05-14T15:27:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc5eeff5700 1 -- 172.21.15.5:0/2008842761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b2d608b440 con 0x55b2d6167e00 2024-05-14T15:27:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.106+0000 7fc5eeff5700 1 -- 172.21.15.5:0/2008842761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc604011070 con 0x55b2d6167e00 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc61fe45b80 10 client.16170 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc61fe45b80 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc61fe45b80 20 client.16170 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d61e7340 0x55b2d61e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc61fe45b80 4 client.16170 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b2d5f032d0 con 0x55b2d61e7340 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.107+0000 7fc60affd700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d61e7340 0x55b2d61e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.108+0000 7fc5eeff5700 10 client.16170 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.108+0000 7fc60affd700 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d61e7340 0x55b2d61e9790 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0b76950301552c server_cookie=79e72f0709e5f24f in_seq=0 out_seq=0 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc5eeff5700 1 -- 172.21.15.5:0/2008842761 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b2d5f032d0 con 0x55b2d61e7340 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc5eeff5700 10 client.16170 handle_command_reply: tid=0 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 client.16170 shutdown 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 msgr2=0x55b2d61711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 0x55b2d61711c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc60400b0d0 tx=0x7fc604009550 comp rx=0 tx=0).stop 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 shutdown_connections 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b2d61e7340 0x55b2d61e9790 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d6167e00 0x55b2d61711c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2d5fe3ec0 0x55b2d6173eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 --2- 172.21.15.5:0/2008842761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d6161e90 0x55b2d6170c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 >> 172.21.15.5:0/2008842761 conn(0x55b2d602b6b0 msgr2=0x55b2d6156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc5eeff5700 0 client.16170 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 shutdown_connections 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 1 -- 172.21.15.5:0/2008842761 wait complete. 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.110+0000 7fc61fe45b80 20 client.16170 trim_cache size 0 max 16384 2024-05-14T15:27:52.113 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:27:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:27:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "384929bb-0d48-4786-92a3-2a3ab00e94c5", 2024-05-14T15:27:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:27:52.114 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:27:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5f40b29f0 msgr2=0x7fc5f40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5f40b29f0 0x7fc5f40b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc010d80 tx=0x7fc5fc012040 comp rx=0 tx=0).stop 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 msgr2=0x7fc60c19c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c19c230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e970 tx=0x7fc60000ec80 comp rx=0 tx=0).stop 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 shutdown_connections 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5f40b29f0 0x7fc5f40b4f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c108760 0x7fc60c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c1034e0 0x7fc60c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 --2- 172.21.15.5:0/3582874612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c107e10 0x7fc60c19c230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 >> 172.21.15.5:0/3582874612 conn(0x7fc60c0fef30 msgr2=0x7fc60c107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 shutdown_connections 2024-05-14T15:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:52.111+0000 7fc611431700 1 -- 172.21.15.5:0/3582874612 wait complete. 2024-05-14T15:27:52.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 170275 12.84 MB/sec warmup 73 sec latency 53.831 ms 2024-05-14T15:27:52.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:27:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:52 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:27:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:52 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:53.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:52 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:27:53.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:52 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:52 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:27:53.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:52 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:52 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:27:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:52 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:52 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:27:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:52 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:53.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:52 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:27:53.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:52 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:27:53.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 171456 12.72 MB/sec warmup 74 sec latency 208.336 ms 2024-05-14T15:27:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:27:54.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:53 smithi144 ceph-mon[35650]: pgmap v96: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-05-14T15:27:54.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:53 smithi193 ceph-mon[36001]: pgmap v96: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-05-14T15:27:54.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:53 smithi005 ceph-mon[31559]: pgmap v96: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 7.3 MiB/s wr, 553 op/s 2024-05-14T15:27:54.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 174239 12.77 MB/sec warmup 75 sec latency 13.346 ms 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3263491571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f0107df0 msgr2=0x7f98f0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3263491571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f0107df0 0x7f98f0108200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f98e40099c0 tx=0x7f98e4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3263491571 shutdown_connections 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3263491571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0108740 0x7f98f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3263491571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f01034c0 0x7f98f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3263491571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f0107df0 0x7f98f0108200 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.943+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3263491571 >> 172.21.15.5:0/3263491571 conn(0x7f98f00fedc0 msgr2=0x7f98f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.944+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3263491571 shutdown_connections 2024-05-14T15:27:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.944+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3263491571 wait complete. 2024-05-14T15:27:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 Processor -- start 2024-05-14T15:27:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 -- start start 2024-05-14T15:27:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 0x7f98f019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0107df0 0x7f98f019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f0108740 0x7f98f019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0113350 con 0x7f98f01034c0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01131d0 con 0x7f98f0108740 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98f4b0b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01134d0 con 0x7f98f0107df0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 0x7f98f019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 0x7f98f019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40294/0 (socket says 172.21.15.5:40294) 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98ee59c700 1 -- 172.21.15.5:0/3596115166 learned_addr learned my addr 172.21.15.5:0/3596115166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.945+0000 7f98eed9d700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f0108740 0x7f98f019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98edd9b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0107df0 0x7f98f019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98ee59c700 1 -- 172.21.15.5:0/3596115166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0107df0 msgr2=0x7f98f019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98ee59c700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0107df0 0x7f98f019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98ee59c700 1 -- 172.21.15.5:0/3596115166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f0108740 msgr2=0x7f98f019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98ee59c700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f0108740 0x7f98f019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98ee59c700 1 -- 172.21.15.5:0/3596115166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f019fce0 con 0x7f98f01034c0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98ee59c700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 0x7f98f019ae10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d980 tx=0x7f98e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98df7fe700 1 -- 172.21.15.5:0/3596115166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0016070 con 0x7f98f01034c0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f019ffd0 con 0x7f98f01034c0 2024-05-14T15:27:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98df7fe700 1 -- 172.21.15.5:0/3596115166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e0004830 con 0x7f98f01034c0 2024-05-14T15:27:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98df7fe700 1 -- 172.21.15.5:0/3596115166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0005540 con 0x7f98f01034c0 2024-05-14T15:27:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.946+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f00ff1d0 con 0x7f98f01034c0 2024-05-14T15:27:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.949+0000 7f98df7fe700 1 -- 172.21.15.5:0/3596115166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f98e000f690 con 0x7f98f01034c0 2024-05-14T15:27:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.950+0000 7f98df7fe700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98d80b29f0 0x7f98d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.950+0000 7f990351fb80 1 Processor -- start 2024-05-14T15:27:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.951+0000 7f98df7fe700 1 -- 172.21.15.5:0/3596115166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f98e00e27b0 con 0x7f98f01034c0 2024-05-14T15:27:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.951+0000 7f98edd9b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98d80b29f0 0x7f98d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.951+0000 7f990351fb80 1 -- start start 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.951+0000 7f98edd9b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98d80b29f0 0x7f98d80b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98e4000c00 tx=0x7f98e401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f990351fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fa2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f990351fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 0x55f868fa6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f990351fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa7470 0x55f868fa7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f990351fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f868ec4860 con 0x55f868fa7470 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f990351fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f868e637c0 con 0x55f868e24e40 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f990351fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f868ec0820 con 0x55f868fa2c60 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa7470 0x55f868fa7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 0x55f868fa6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 0x55f868fa6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39768/0 (socket says 172.21.15.5:39768) 2024-05-14T15:27:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa7470 0x55f868fa7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40300/0 (socket says 172.21.15.5:40300) 2024-05-14T15:27:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 -- 172.21.15.5:0/4029853409 learned_addr learned my addr 172.21.15.5:0/4029853409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98ee59c700 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fa2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 -- 172.21.15.5:0/4029853409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 msgr2=0x55f868fa2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fa2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 -- 172.21.15.5:0/4029853409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa7470 msgr2=0x55f868fa7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa7470 0x55f868fa7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 -- 172.21.15.5:0/4029853409 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f868e118d0 con 0x55f868fa2c60 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.952+0000 7f98edd9b700 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 0x55f868fa6f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e4015000 tx=0x7f98e4009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ae3ce67b88e77c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.953+0000 7f98dcff9700 1 -- 172.21.15.5:0/4029853409 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40056e0 con 0x55f868fa2c60 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.953+0000 7f98dcff9700 1 -- 172.21.15.5:0/4029853409 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e4003e50 con 0x55f868fa2c60 2024-05-14T15:27:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.953+0000 7f98dcff9700 1 -- 172.21.15.5:0/4029853409 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4025be0 con 0x55f868fa2c60 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 -- 172.21.15.5:0/4029853409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 msgr2=0x55f868fa6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 0x55f868fa6f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98e4015000 tx=0x7f98e4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 -- 172.21.15.5:0/4029853409 shutdown_connections 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa7470 0x55f868fa7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fa2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 --2- 172.21.15.5:0/4029853409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa2c60 0x55f868fa6f30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 -- 172.21.15.5:0/4029853409 >> 172.21.15.5:0/4029853409 conn(0x55f868e6c600 msgr2=0x55f868f9d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 -- 172.21.15.5:0/4029853409 shutdown_connections 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 -- 172.21.15.5:0/4029853409 wait complete. 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 -- start start 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.954+0000 7f990351fb80 1 Processor -- start 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 0x55f868facbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f868faf3c0 con 0x55f868fa2c60 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f868e2ac30 con 0x55f868e24e40 2024-05-14T15:27:54.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f868faf540 con 0x55f868fa7470 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f990351fb80 10 client.0 _ll_register_callbacks cb 0x55f868e5c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40302/0 (socket says 172.21.15.5:40302) 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 -- 172.21.15.5:0/4049960332 learned_addr learned my addr 172.21.15.5:0/4049960332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 -- 172.21.15.5:0/4049960332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 msgr2=0x55f868facbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 0x55f868facbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 -- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 msgr2=0x55f868fac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 -- 172.21.15.5:0/4049960332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f868e60690 con 0x55f868fa2c60 2024-05-14T15:27:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98edd9b700 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac670 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f98e4025800 tx=0x7f98e4025f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951b0203ebd905a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.955+0000 7f98d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f98d5ffb700 1 -- 172.21.15.5:0/4049960332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4035040 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f98d5ffb700 1 -- 172.21.15.5:0/4049960332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f98e400b790 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98e4003e50 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f98d5ffb700 1 -- 172.21.15.5:0/4049960332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98e4047050 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 10 client.16182 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f868faf3c0 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f98d5ffb700 1 -- 172.21.15.5:0/4049960332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e4003e50 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f98d5ffb700 1 -- 172.21.15.5:0/4049960332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f98e4038070 con 0x55f868fa2c60 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 20 client.16182 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f868fbc270 0x55f868fbe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 4 client.16182 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f868ed2240 con 0x55f868fbc270 2024-05-14T15:27:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.956+0000 7f98ee59c700 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f868fbc270 0x55f868fbe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.957+0000 7f98d5ffb700 10 client.16182 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.957+0000 7f98ee59c700 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f868fbc270 0x55f868fbe690 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db1dbc6f8e2b969 server_cookie=b47811729ba568e7 in_seq=0 out_seq=0 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f98d5ffb700 1 -- 172.21.15.5:0/4049960332 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f868ed2240 con 0x55f868fbc270 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f98d5ffb700 10 client.16182 handle_command_reply: tid=0 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 client.16182 shutdown 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 msgr2=0x55f868fac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac670 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f98e4025800 tx=0x7f98e4025f80 comp rx=0 tx=0).stop 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 shutdown_connections 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f868fbc270 0x55f868fbe690 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 0x55f868facbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 --2- 172.21.15.5:0/4049960332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac670 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 >> 172.21.15.5:0/4049960332 conn(0x55f868e6c600 msgr2=0x55f868fa1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f98d5ffb700 0 client.16182 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:54.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 shutdown_connections 2024-05-14T15:27:54.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 1 -- 172.21.15.5:0/4049960332 wait complete. 2024-05-14T15:27:54.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.958+0000 7f990351fb80 20 client.16182 trim_cache size 0 max 16384 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 Processor -- start 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 -- start start 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868e24e40 0x55f868fa1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868fa7470 0x55f868fa8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01b4f00 con 0x55f868fa2c60 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01b2790 con 0x55f868fa7470 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f990351fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01b0600 con 0x55f868e24e40 2024-05-14T15:27:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40304/0 (socket says 172.21.15.5:40304) 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f98edd9b700 1 -- 172.21.15.5:0/2384157759 learned_addr learned my addr 172.21.15.5:0/2384157759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f98eed9d700 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868fa7470 0x55f868fa8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.980+0000 7f98ee59c700 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868e24e40 0x55f868fa1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98edd9b700 1 -- 172.21.15.5:0/2384157759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868e24e40 msgr2=0x55f868fa1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98edd9b700 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868e24e40 0x55f868fa1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98edd9b700 1 -- 172.21.15.5:0/2384157759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868fa7470 msgr2=0x55f868fa8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98edd9b700 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868fa7470 0x55f868fa8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98edd9b700 1 -- 172.21.15.5:0/2384157759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f868f99ae0 con 0x55f868fa2c60 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98ee59c700 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868e24e40 0x55f868fa1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98edd9b700 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac9f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f98e4025f50 tx=0x7f98e4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eccba432a7ec342 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:54.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98d7fff700 1 -- 172.21.15.5:0/2384157759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40056e0 con 0x55f868fa2c60 2024-05-14T15:27:54.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98d7fff700 1 -- 172.21.15.5:0/2384157759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e403e070 con 0x55f868fa2c60 2024-05-14T15:27:54.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.981+0000 7f98d7fff700 1 -- 172.21.15.5:0/2384157759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e400f460 con 0x55f868fa2c60 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 -- 172.21.15.5:0/2384157759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 msgr2=0x55f868fac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac9f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f98e4025f50 tx=0x7f98e4009310 comp rx=0 tx=0).stop 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 -- 172.21.15.5:0/2384157759 shutdown_connections 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868fa7470 0x55f868fa8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868e24e40 0x55f868fa1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 --2- 172.21.15.5:0/2384157759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fac9f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 -- 172.21.15.5:0/2384157759 >> 172.21.15.5:0/2384157759 conn(0x55f868e6c600 msgr2=0x55f868f9d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 -- 172.21.15.5:0/2384157759 shutdown_connections 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.982+0000 7f990351fb80 1 -- 172.21.15.5:0/2384157759 wait complete. 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 -- start start 2024-05-14T15:27:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 Processor -- start 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fb4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 0x55f868fafb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01b0c50 con 0x55f868fa2c60 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01b20c0 con 0x55f868e24e40 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01b1320 con 0x55f868fa7470 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f990351fb80 10 client.0 _ll_register_callbacks cb 0x55f868e5c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fb4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f98edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fb4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40306/0 (socket says 172.21.15.5:40306) 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.983+0000 7f98edd9b700 1 -- 172.21.15.5:0/2307490232 learned_addr learned my addr 172.21.15.5:0/2307490232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98ee59c700 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98edd9b700 1 -- 172.21.15.5:0/2307490232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 msgr2=0x55f868fafb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98edd9b700 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 0x55f868fafb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98edd9b700 1 -- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 msgr2=0x55f868fb4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:54.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98edd9b700 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fb4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98edd9b700 1 -- 172.21.15.5:0/2307490232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f868fb0280 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98edd9b700 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fb4bc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f98e4003f20 tx=0x7f98e4003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae5dc1aa6143085 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98d5ffb700 1 -- 172.21.15.5:0/2307490232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e400f460 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98e403e070 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98d5ffb700 1 -- 172.21.15.5:0/2307490232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f98e400f600 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f98d5ffb700 1 -- 172.21.15.5:0/2307490232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98e405b050 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f990351fb80 10 client.16188 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.984+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f8690261f0 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f98d5ffb700 1 -- 172.21.15.5:0/2307490232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e403e070 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f98d5ffb700 1 -- 172.21.15.5:0/2307490232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f98e4038030 con 0x55f868fa2c60 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f990351fb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:54.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f990351fb80 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:54.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f990351fb80 20 client.16188 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:54.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f990351fb80 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8690268d0 0x55f869028cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:54.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f990351fb80 4 client.16188 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:27:54.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.985+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f868ecc2e0 con 0x55f8690268d0 2024-05-14T15:27:54.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.987+0000 7f98ee59c700 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8690268d0 0x55f869028cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:54.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f98d5ffb700 10 client.16188 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f98ee59c700 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8690268d0 0x55f869028cf0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a87bc59a09edce2 server_cookie=d994efe19bd16d01 in_seq=0 out_seq=0 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f98d5ffb700 1 -- 172.21.15.5:0/2307490232 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f868ecc2e0 con 0x55f8690268d0 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f98d5ffb700 10 client.16188 handle_command_reply: tid=0 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 client.16188 shutdown 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 msgr2=0x55f868fb4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fb4bc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f98e4003f20 tx=0x7f98e4003f50 comp rx=0 tx=0).stop 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 shutdown_connections 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8690268d0 0x55f869028cf0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f868fa7470 0x55f868fafb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f868e24e40 0x55f868fb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 --2- 172.21.15.5:0/2307490232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f868fa2c60 0x55f868fb4bc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.988+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 >> 172.21.15.5:0/2307490232 conn(0x55f868e6c600 msgr2=0x55f868fa9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f98d5ffb700 0 client.16188 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 shutdown_connections 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f990351fb80 1 -- 172.21.15.5:0/2307490232 wait complete. 2024-05-14T15:27:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f990351fb80 20 client.16188 trim_cache size 0 max 16384 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "384929bb-0d48-4786-92a3-2a3ab00e94c5": { 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "384929bb-0d48-4786-92a3-2a3ab00e94c5", 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:27:54.993 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:27:54.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98d80b29f0 msgr2=0x7f98d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98d80b29f0 0x7f98d80b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98e4000c00 tx=0x7f98e401a040 comp rx=0 tx=0).stop 2024-05-14T15:27:54.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 msgr2=0x7f98f019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:54.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.989+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 0x7f98f019ae10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d980 tx=0x7f98e000dc90 comp rx=0 tx=0).stop 2024-05-14T15:27:54.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 shutdown_connections 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f98d80b29f0 0x7f98d80b4f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98f0108740 0x7f98f019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98f01034c0 0x7f98f019ae10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 --2- 172.21.15.5:0/3596115166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f0107df0 0x7f98f019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 >> 172.21.15.5:0/3596115166 conn(0x7f98f00fedc0 msgr2=0x7f98f006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 shutdown_connections 2024-05-14T15:27:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:27:54.990+0000 7f98f4b0b700 1 -- 172.21.15.5:0/3596115166 wait complete. 2024-05-14T15:27:55.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 176750 12.77 MB/sec warmup 76 sec latency 21.988 ms 2024-05-14T15:27:55.804 INFO:tasks.cephfs.filesystem:scrub status for tag:384929bb-0d48-4786-92a3-2a3ab00e94c5 - {'path': '/', 'tag': '384929bb-0d48-4786-92a3-2a3ab00e94c5', 'options': 'recursive,force'} 2024-05-14T15:27:56.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:55 smithi144 ceph-mon[35650]: pgmap v97: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-05-14T15:27:56.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:55 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:27:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:55 smithi193 ceph-mon[36001]: pgmap v97: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-05-14T15:27:56.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:55 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:27:56.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:55 smithi005 ceph-mon[31559]: pgmap v97: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-05-14T15:27:56.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:55 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:27:56.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 178087 12.69 MB/sec warmup 77 sec latency 207.472 ms 2024-05-14T15:27:57.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181018 12.75 MB/sec warmup 78 sec latency 11.308 ms 2024-05-14T15:27:58.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:57 smithi144 ceph-mon[35650]: pgmap v98: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 460 op/s 2024-05-14T15:27:58.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:57 smithi193 ceph-mon[36001]: pgmap v98: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 460 op/s 2024-05-14T15:27:58.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:57 smithi005 ceph-mon[31559]: pgmap v98: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 460 op/s 2024-05-14T15:27:58.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 183251 12.70 MB/sec warmup 79 sec latency 15.904 ms 2024-05-14T15:27:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 -- 172.21.15.5:0/3461120731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44101032e0 msgr2=0x7f441010f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 --2- 172.21.15.5:0/3461120731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44101032e0 0x7f441010f6b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4408009ca0 tx=0x7f440802a8c0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 -- 172.21.15.5:0/3461120731 shutdown_connections 2024-05-14T15:27:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 --2- 172.21.15.5:0/3461120731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44101032e0 0x7f441010f6b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 --2- 172.21.15.5:0/3461120731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410109030 0x7f4410102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 --2- 172.21.15.5:0/3461120731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4410108b90 0x7f4410102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.057+0000 7f4415cb6640 1 -- 172.21.15.5:0/3461120731 >> 172.21.15.5:0/3461120731 conn(0x7f44100fe390 msgr2=0x7f44101007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.058+0000 7f4415cb6640 1 -- 172.21.15.5:0/3461120731 shutdown_connections 2024-05-14T15:27:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.058+0000 7f4415cb6640 1 -- 172.21.15.5:0/3461120731 wait complete. 2024-05-14T15:27:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.058+0000 7f4415cb6640 1 Processor -- start 2024-05-14T15:27:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.058+0000 7f4415cb6640 1 -- start start 2024-05-14T15:27:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4415cb6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44101032e0 0x7f44101997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4415cb6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410108b90 0x7f4410199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4415cb6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 0x7f441019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4415cb6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4410112d80 con 0x7f4410109030 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4415cb6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4410112c00 con 0x7f4410108b90 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4415cb6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4410112f00 con 0x7f44101032e0 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 0x7f441019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410108b90 0x7f4410199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 0x7f441019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60944/0 (socket says 172.21.15.5:60944) 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 -- 172.21.15.5:0/1174911301 learned_addr learned my addr 172.21.15.5:0/1174911301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440f7fe640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44101032e0 0x7f44101997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 -- 172.21.15.5:0/1174911301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44101032e0 msgr2=0x7f44101997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44101032e0 0x7f44101997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 -- 172.21.15.5:0/1174911301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410108b90 msgr2=0x7f4410199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410108b90 0x7f4410199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f440ffff640 1 -- 172.21.15.5:0/1174911301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441019e5b0 con 0x7f4410109030 2024-05-14T15:27:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.059+0000 7f4406ffd640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410108b90 0x7f4410199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.060+0000 7f440ffff640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 0x7f441019df70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f44080098d0 tx=0x7f44080317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.060+0000 7f440d7fa640 1 -- 172.21.15.5:0/1174911301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408003b70 con 0x7f4410109030 2024-05-14T15:27:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.060+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441019e840 con 0x7f4410109030 2024-05-14T15:27:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.060+0000 7f440d7fa640 1 -- 172.21.15.5:0/1174911301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4408003d10 con 0x7f4410109030 2024-05-14T15:27:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.060+0000 7f440d7fa640 1 -- 172.21.15.5:0/1174911301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440802fa20 con 0x7f4410109030 2024-05-14T15:27:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.060+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441019ed90 con 0x7f4410109030 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.063+0000 7f441cb6c740 1 Processor -- start 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.063+0000 7f440d7fa640 1 -- 172.21.15.5:0/1174911301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f440802fc40 con 0x7f4410109030 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.063+0000 7f441cb6c740 1 -- start start 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d21a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f441cb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564701c875d0 con 0x5647019582f0 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f441cb6c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564701c40890 con 0x5647019604d0 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f441cb6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564701c83590 con 0x5647019811b0 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d21f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60948/0 (socket says 172.21.15.5:60948) 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d21a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440f7fe640 1 -- 172.21.15.5:0/2024968965 learned_addr learned my addr 172.21.15.5:0/2024968965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 -- 172.21.15.5:0/2024968965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 msgr2=0x564701d224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440d7fa640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43fc0b2950 0x7f43fc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d224d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 -- 172.21.15.5:0/2024968965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 msgr2=0x564701d21f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d21f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 -- 172.21.15.5:0/2024968965 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564701999210 con 0x5647019811b0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440f7fe640 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d21f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440ffff640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43fc0b2950 0x7f43fc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f440d7fa640 1 -- 172.21.15.5:0/1174911301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4408103e20 con 0x7f4410109030 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.064+0000 7f4406ffd640 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d21a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4410106cd0 tx=0x7f43f80096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9586225edc5ef161 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f4405ffb640 1 -- 172.21.15.5:0/2024968965 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8011050 con 0x5647019811b0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f4405ffb640 1 -- 172.21.15.5:0/2024968965 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f80111f0 con 0x5647019811b0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f4405ffb640 1 -- 172.21.15.5:0/2024968965 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f80035f0 con 0x5647019811b0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 -- 172.21.15.5:0/2024968965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 msgr2=0x564701d21a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d21a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4410106cd0 tx=0x7f43f80096c0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f440ffff640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43fc0b2950 0x7f43fc0b4e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4408101370 tx=0x7f4408078050 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 -- 172.21.15.5:0/2024968965 shutdown_connections 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d21f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 --2- 172.21.15.5:0/2024968965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d21a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.065+0000 7f441cb6c740 1 -- 172.21.15.5:0/2024968965 >> 172.21.15.5:0/2024968965 conn(0x564701975300 msgr2=0x564701d1e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 -- 172.21.15.5:0/2024968965 shutdown_connections 2024-05-14T15:27:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 -- 172.21.15.5:0/2024968965 wait complete. 2024-05-14T15:27:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 -- start start 2024-05-14T15:27:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 Processor -- start 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d2e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d3a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564701d30f20 con 0x5647019582f0 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564701c44370 con 0x5647019604d0 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.066+0000 7f441cb6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564701d310a0 con 0x5647019811b0 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f441cb6c740 10 client.0 _ll_register_callbacks cb 0x564701998a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d2e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d2e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60958/0 (socket says 172.21.15.5:60958) 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f4406ffd640 1 -- 172.21.15.5:0/4127698559 learned_addr learned my addr 172.21.15.5:0/4127698559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f4406ffd640 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d3a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 -- 172.21.15.5:0/4127698559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 msgr2=0x564701d3a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d3a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 -- 172.21.15.5:0/4127698559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 msgr2=0x564701d2e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d2e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 -- 172.21.15.5:0/4127698559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564701a65620 con 0x5647019604d0 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f440f7fe640 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d2e910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4400005fd0 tx=0x7f4400012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d17aa86051a3c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f43eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:27:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f43eeffd640 1 -- 172.21.15.5:0/4127698559 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000108a0 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f80111f0 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.067+0000 7f43eeffd640 1 -- 172.21.15.5:0/4127698559 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4400010a40 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f43eeffd640 1 -- 172.21.15.5:0/4127698559 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4400010c40 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 10 client.25826 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564701d3d300 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f43eeffd640 1 -- 172.21.15.5:0/4127698559 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f80111f0 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f43eeffd640 1 -- 172.21.15.5:0/4127698559 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4400019070 con 0x5647019604d0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 10 client.25826 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 20 client.25826 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56470193fd80 0x564701d407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 4 client.25826 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564701c951e0 con 0x56470193fd80 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.068+0000 7f4406ffd640 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56470193fd80 0x564701d407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.069+0000 7f43eeffd640 10 client.25826 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.069+0000 7f4406ffd640 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56470193fd80 0x564701d407c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627cc83a52893c08 server_cookie=c2f77b76ffbaede0 in_seq=0 out_seq=0 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f43eeffd640 1 -- 172.21.15.5:0/4127698559 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564701c951e0 con 0x56470193fd80 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f43eeffd640 10 client.25826 handle_command_reply: tid=0 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 client.25826 shutdown 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 msgr2=0x564701d2e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d2e910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4400005fd0 tx=0x7f4400012820 comp rx=0 tx=0).stop 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 shutdown_connections 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56470193fd80 0x564701d407c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d3a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d2e910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 --2- 172.21.15.5:0/4127698559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d2e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f43eeffd640 0 client.25826 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 >> 172.21.15.5:0/4127698559 conn(0x564701975300 msgr2=0x564701d223a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 shutdown_connections 2024-05-14T15:27:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 1 -- 172.21.15.5:0/4127698559 wait complete. 2024-05-14T15:27:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.070+0000 7f441cb6c740 20 client.25826 trim_cache size 0 max 16384 2024-05-14T15:27:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.085+0000 7f441cb6c740 1 Processor -- start 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.085+0000 7f441cb6c740 1 -- start start 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.085+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d27c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d45fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f441cb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b1a30 con 0x5647019582f0 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f441cb6c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44101af300 con 0x5647019604d0 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f441cb6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44101ad150 con 0x5647019811b0 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d239a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60974/0 (socket says 172.21.15.5:60974) 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 -- 172.21.15.5:0/3488256444 learned_addr learned my addr 172.21.15.5:0/3488256444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f440f7fe640 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d27c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d45fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 -- 172.21.15.5:0/3488256444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 msgr2=0x564701d45fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d45fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 -- 172.21.15.5:0/3488256444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 msgr2=0x564701d27c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d27c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 -- 172.21.15.5:0/3488256444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564701d19a80 con 0x5647019582f0 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f4406ffd640 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d239a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f43f8006010 tx=0x7f43f8009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f961882d0068b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f44057fa640 1 -- 172.21.15.5:0/3488256444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8028040 con 0x5647019582f0 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f44057fa640 1 -- 172.21.15.5:0/3488256444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f8029040 con 0x5647019582f0 2024-05-14T15:27:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.086+0000 7f44057fa640 1 -- 172.21.15.5:0/3488256444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8003b80 con 0x5647019582f0 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 -- 172.21.15.5:0/3488256444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 msgr2=0x564701d239a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d239a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f43f8006010 tx=0x7f43f8009ce0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 -- 172.21.15.5:0/3488256444 shutdown_connections 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d45fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019604d0 0x564701d27c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 --2- 172.21.15.5:0/3488256444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019582f0 0x564701d239a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 -- 172.21.15.5:0/3488256444 >> 172.21.15.5:0/3488256444 conn(0x564701975300 msgr2=0x564701d223a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 -- 172.21.15.5:0/3488256444 shutdown_connections 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.087+0000 7f441cb6c740 1 -- 172.21.15.5:0/3488256444 wait complete. 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 -- start start 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 Processor -- start 2024-05-14T15:27:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019582f0 0x564701d395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 0x564701d39b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44101ad7c0 con 0x5647019604d0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44101aec30 con 0x5647019582f0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44101ade90 con 0x5647019811b0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f441cb6c740 10 client.0 _ll_register_callbacks cb 0x7f440000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 0x564701d39b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 0x564701d39b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60986/0 (socket says 172.21.15.5:60986) 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f440f7fe640 1 -- 172.21.15.5:0/1562089905 learned_addr learned my addr 172.21.15.5:0/1562089905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.088+0000 7f4406ffd640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019582f0 0x564701d395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f4406ffd640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f440f7fe640 1 -- 172.21.15.5:0/1562089905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 msgr2=0x564701d35330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f440f7fe640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d35330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f440f7fe640 1 -- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019582f0 msgr2=0x564701d395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f440f7fe640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019582f0 0x564701d395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f440f7fe640 1 -- 172.21.15.5:0/1562089905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564701d3a3c0 con 0x5647019604d0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f4406ffd640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f440f7fe640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 0x564701d39b00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x564701d2f790 tx=0x7f4400005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7f72e372a4e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f43eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f43eeffd640 1 -- 172.21.15.5:0/1562089905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440002d070 con 0x5647019604d0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f43eeffd640 1 -- 172.21.15.5:0/1562089905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4400004440 con 0x5647019604d0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f8029040 con 0x5647019604d0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f43eeffd640 1 -- 172.21.15.5:0/1562089905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4400005040 con 0x5647019604d0 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f441cb6c740 10 client.16206 fetch_fsmap learned FSMap version 30 2024-05-14T15:27:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564701d36990 con 0x5647019604d0 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.089+0000 7f43eeffd640 1 -- 172.21.15.5:0/1562089905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f8029040 con 0x5647019604d0 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f43eeffd640 1 -- 172.21.15.5:0/1562089905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4400019040 con 0x5647019604d0 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f441cb6c740 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f441cb6c740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f441cb6c740 20 client.16206 populate_metadata read hostname 'smithi005' 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f441cb6c740 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5647019151c0 0x564701daad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f441cb6c740 4 client.16206 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564701d24850 con 0x5647019151c0 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f4406ffd640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5647019151c0 0x564701daad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f43eeffd640 10 client.16206 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.090+0000 7f4406ffd640 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5647019151c0 0x564701daad60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d570c03864f7e542 server_cookie=e3b45dcc9e267e51 in_seq=0 out_seq=0 2024-05-14T15:27:59.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 185123 12.69 MB/sec warmup 80 sec latency 16.214 ms 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f43eeffd640 1 -- 172.21.15.5:0/1562089905 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564701d24850 con 0x5647019151c0 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f43eeffd640 10 client.16206 handle_command_reply: tid=0 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 client.16206 shutdown 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 msgr2=0x564701d39b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 0x564701d39b00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x564701d2f790 tx=0x7f4400005cc0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 shutdown_connections 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5647019151c0 0x564701daad60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f43eeffd640 0 client.16206 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647019811b0 0x564701d35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647019604d0 0x564701d39b00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 --2- 172.21.15.5:0/1562089905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647019582f0 0x564701d395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 >> 172.21.15.5:0/1562089905 conn(0x564701975300 msgr2=0x564701d22100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 shutdown_connections 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.347+0000 7f441cb6c740 1 -- 172.21.15.5:0/1562089905 wait complete. 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f441cb6c740 20 client.16206 trim_cache size 0 max 16384 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:27:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43fc0b2950 msgr2=0x7f43fc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43fc0b2950 0x7f43fc0b4e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4408101370 tx=0x7f4408078050 comp rx=0 tx=0).stop 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 msgr2=0x7f441019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 0x7f441019df70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f44080098d0 tx=0x7f44080317b0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 shutdown_connections 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43fc0b2950 0x7f43fc0b4e60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4410109030 0x7f441019df70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.348+0000 7f4415cb6640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4410108b90 0x7f4410199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.349+0000 7f4415cb6640 1 --2- 172.21.15.5:0/1174911301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44101032e0 0x7f44101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:27:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.349+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 >> 172.21.15.5:0/1174911301 conn(0x7f44100fe390 msgr2=0x7f4410105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:27:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.349+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 shutdown_connections 2024-05-14T15:27:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:27:59.349+0000 7f4415cb6640 1 -- 172.21.15.5:0/1174911301 wait complete. 2024-05-14T15:28:00.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:27:59 smithi193 ceph-mon[36001]: pgmap v99: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-14T15:28:00.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:27:59 smithi005 ceph-mon[31559]: pgmap v99: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-14T15:28:00.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 187650 12.70 MB/sec warmup 81 sec latency 69.340 ms 2024-05-14T15:28:00.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:27:59 smithi144 ceph-mon[35650]: pgmap v99: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-14T15:28:01.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 188237 12.61 MB/sec warmup 82 sec latency 85.156 ms 2024-05-14T15:28:02.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:01 smithi193 ceph-mon[36001]: pgmap v100: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-14T15:28:02.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:01 smithi005 ceph-mon[31559]: pgmap v100: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-14T15:28:02.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 190407 12.55 MB/sec warmup 83 sec latency 19.043 ms 2024-05-14T15:28:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:01 smithi144 ceph-mon[35650]: pgmap v100: 129 pgs: 129 active+clean; 38 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-05-14T15:28:03.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 192336 12.55 MB/sec warmup 84 sec latency 20.602 ms 2024-05-14T15:28:04.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:03 smithi005 ceph-mon[31559]: pgmap v101: 129 pgs: 129 active+clean; 36 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 433 op/s 2024-05-14T15:28:04.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 195297 12.62 MB/sec warmup 85 sec latency 20.950 ms 2024-05-14T15:28:04.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:03 smithi144 ceph-mon[35650]: pgmap v101: 129 pgs: 129 active+clean; 36 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 433 op/s 2024-05-14T15:28:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:03 smithi193 ceph-mon[36001]: pgmap v101: 129 pgs: 129 active+clean; 36 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 433 op/s 2024-05-14T15:28:05.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 197654 12.57 MB/sec warmup 86 sec latency 17.928 ms 2024-05-14T15:28:06.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:05 smithi005 ceph-mon[31559]: pgmap v102: 129 pgs: 129 active+clean; 36 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-05-14T15:28:06.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 199656 12.56 MB/sec warmup 87 sec latency 14.230 ms 2024-05-14T15:28:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:05 smithi144 ceph-mon[35650]: pgmap v102: 129 pgs: 129 active+clean; 36 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-05-14T15:28:06.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:05 smithi193 ceph-mon[36001]: pgmap v102: 129 pgs: 129 active+clean; 36 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 347 op/s 2024-05-14T15:28:07.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 202268 12.61 MB/sec warmup 88 sec latency 143.926 ms 2024-05-14T15:28:08.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:07 smithi005 ceph-mon[31559]: pgmap v103: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 431 op/s 2024-05-14T15:28:08.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 202717 12.48 MB/sec warmup 89 sec latency 224.072 ms 2024-05-14T15:28:08.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:07 smithi144 ceph-mon[35650]: pgmap v103: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 431 op/s 2024-05-14T15:28:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:07 smithi193 ceph-mon[36001]: pgmap v103: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 431 op/s 2024-05-14T15:28:09.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:08 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:28:09.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:08 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:28:09.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:08 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:28:09.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:08 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:28:09.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204576 12.42 MB/sec warmup 90 sec latency 40.016 ms 2024-05-14T15:28:09.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:08 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:28:09.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:08 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:28:09.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:08 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:28:09.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:08 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:28:09.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:08 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:28:09.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:08 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:28:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:08 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:28:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:08 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:28:10.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:09 smithi005 ceph-mon[31559]: pgmap v104: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-05-14T15:28:10.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 206535 12.42 MB/sec warmup 91 sec latency 12.146 ms 2024-05-14T15:28:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:09 smithi144 ceph-mon[35650]: pgmap v104: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-05-14T15:28:10.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:09 smithi193 ceph-mon[36001]: pgmap v104: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-05-14T15:28:11.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209518 12.49 MB/sec warmup 92 sec latency 11.624 ms 2024-05-14T15:28:12.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:11 smithi005 ceph-mon[31559]: pgmap v105: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-05-14T15:28:12.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 210937 12.41 MB/sec warmup 93 sec latency 147.985 ms 2024-05-14T15:28:12.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:11 smithi144 ceph-mon[35650]: pgmap v105: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-05-14T15:28:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:11 smithi193 ceph-mon[36001]: pgmap v105: 129 pgs: 129 active+clean; 43 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 -- 172.21.15.5:0/2003645706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 msgr2=0x7f373410dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 --2- 172.21.15.5:0/2003645706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f373410dc20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3728009860 tx=0x7f372802a340 comp rx=0 tx=0).stop 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 -- 172.21.15.5:0/2003645706 shutdown_connections 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 --2- 172.21.15.5:0/2003645706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373410e1f0 0x7f37341106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 --2- 172.21.15.5:0/2003645706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f373410dc20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 --2- 172.21.15.5:0/2003645706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3734104da0 0x7f37341051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 -- 172.21.15.5:0/2003645706 >> 172.21.15.5:0/2003645706 conn(0x7f37340782e0 msgr2=0x7f3734078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 -- 172.21.15.5:0/2003645706 shutdown_connections 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.468+0000 7f373a415640 1 -- 172.21.15.5:0/2003645706 wait complete. 2024-05-14T15:28:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 Processor -- start 2024-05-14T15:28:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 -- start start 2024-05-14T15:28:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 0x7f37341a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f37341a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 0x7f373419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3734113e50 con 0x7f373410e1f0 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3734113cd0 con 0x7f373410b800 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.469+0000 7f373a415640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3734113fd0 con 0x7f3734104da0 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 0x7f373419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f3733fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 0x7f37341a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f37341a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f3733fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 0x7f37341a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:48182/0 (socket says 172.21.15.5:48182) 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 0x7f373419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46232/0 (socket says 172.21.15.5:46232) 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f3733fff640 1 -- 172.21.15.5:0/1531570072 learned_addr learned my addr 172.21.15.5:0/1531570072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 -- 172.21.15.5:0/1531570072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 msgr2=0x7f37341a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 0x7f37341a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 -- 172.21.15.5:0/1531570072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 msgr2=0x7f37341a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f37341a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 -- 172.21.15.5:0/1531570072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373419ae00 con 0x7f373410e1f0 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f3733fff640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 0x7f37341a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37337fe640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f37341a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37389b2640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 0x7f373419a6d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f372c009fd0 tx=0x7f372c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37317fa640 1 -- 172.21.15.5:0/1531570072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c00ce20 con 0x7f373410e1f0 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373419b090 con 0x7f373410e1f0 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37317fa640 1 -- 172.21.15.5:0/1531570072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c005020 con 0x7f373410e1f0 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f37317fa640 1 -- 172.21.15.5:0/1531570072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c010690 con 0x7f373410e1f0 2024-05-14T15:28:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.470+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a72f0 con 0x7f373410e1f0 2024-05-14T15:28:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.473+0000 7f37413a3740 1 Processor -- start 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.473+0000 7f37317fa640 1 -- 172.21.15.5:0/1531570072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f372c0054d0 con 0x7f373410e1f0 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.473+0000 7f37413a3740 1 -- start start 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37413a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 0x5644eeb3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37413a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37413a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee77e4d0 0x5644eeb40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37413a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644eeaa55d0 con 0x5644ee79f1b0 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37413a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5644eea5e890 con 0x5644ee77e4d0 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37413a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644eeaa1590 con 0x5644ee7762f0 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 0x5644eeb3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 0x5644eeb3fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46236/0 (socket says 172.21.15.5:46236) 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 -- 172.21.15.5:0/2534197986 learned_addr learned my addr 172.21.15.5:0/2534197986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f3733fff640 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee77e4d0 0x5644eeb40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 -- 172.21.15.5:0/2534197986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 msgr2=0x5644eeb40120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb40120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 -- 172.21.15.5:0/2534197986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee77e4d0 msgr2=0x5644eeb40660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee77e4d0 0x5644eeb40660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 -- 172.21.15.5:0/2534197986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ee7b7210 con 0x5644ee79f1b0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37317fa640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f371c0b2950 0x7f371c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f37337fe640 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 0x5644eeb3fbe0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3728000c00 tx=0x7f3728005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad234c451c73f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.474+0000 7f3733fff640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f371c0b2950 0x7f371c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.475+0000 7f37317fa640 1 -- 172.21.15.5:0/1531570072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f372c0e3070 con 0x7f373410e1f0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.475+0000 7f371a7fc640 1 -- 172.21.15.5:0/2534197986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728037040 con 0x5644ee79f1b0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.475+0000 7f371a7fc640 1 -- 172.21.15.5:0/2534197986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3728030900 con 0x5644ee79f1b0 2024-05-14T15:28:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.475+0000 7f371a7fc640 1 -- 172.21.15.5:0/2534197986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728032d30 con 0x5644ee79f1b0 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 -- 172.21.15.5:0/2534197986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 msgr2=0x5644eeb3fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 0x5644eeb3fbe0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3728000c00 tx=0x7f3728005d10 comp rx=0 tx=0).stop 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f3733fff640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f371c0b2950 0x7f371c0b4e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5644eeb40f90 tx=0x7f3724009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 -- 172.21.15.5:0/2534197986 shutdown_connections 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb40120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee77e4d0 0x5644eeb40660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 --2- 172.21.15.5:0/2534197986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee79f1b0 0x5644eeb3fbe0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 -- 172.21.15.5:0/2534197986 >> 172.21.15.5:0/2534197986 conn(0x5644ee793300 msgr2=0x5644eeb3c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 -- 172.21.15.5:0/2534197986 shutdown_connections 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 -- 172.21.15.5:0/2534197986 wait complete. 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 -- start start 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.476+0000 7f37413a3740 1 Processor -- start 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee79f1b0 0x5644eeb50140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644eeb4cc50 con 0x5644ee77e4d0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5644eea62370 con 0x5644ee7762f0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644eeb4cdd0 con 0x5644ee79f1b0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37413a3740 10 client.0 _ll_register_callbacks cb 0x5644ee7b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb4a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46240/0 (socket says 172.21.15.5:46240) 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 -- 172.21.15.5:0/2494261640 learned_addr learned my addr 172.21.15.5:0/2494261640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f3733fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37389b2640 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee79f1b0 0x5644eeb50140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 -- 172.21.15.5:0/2494261640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee79f1b0 msgr2=0x5644eeb50140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee79f1b0 0x5644eeb50140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 -- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 msgr2=0x5644eeb4a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb4a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 -- 172.21.15.5:0/2494261640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ee883620 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37337fe640 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb4a840 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f37280057c0 tx=0x7f37280097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e37be0b825d94f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.477+0000 7f37137fe640 1 -- 172.21.15.5:0/2494261640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728037070 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37137fe640 1 -- 172.21.15.5:0/2494261640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f3728032cb0 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3728030900 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37137fe640 1 -- 172.21.15.5:0/2494261640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3728051050 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 10 client.16218 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644eeb59110 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37137fe640 1 -- 172.21.15.5:0/2494261640 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3728030900 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37137fe640 1 -- 172.21.15.5:0/2494261640 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f372803d040 con 0x5644ee77e4d0 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 10 client.16218 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 20 client.16218 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee75dd80 0x5644eeb5c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 4 client.16218 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.478+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644eeab31e0 con 0x5644ee75dd80 2024-05-14T15:28:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.479+0000 7f3733fff640 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee75dd80 0x5644eeb5c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.479+0000 7f37137fe640 10 client.16218 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.479+0000 7f3733fff640 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee75dd80 0x5644eeb5c470 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74599b277c4dcf82 server_cookie=b018a4b6036b806e in_seq=0 out_seq=0 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37137fe640 1 -- 172.21.15.5:0/2494261640 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644eeab31e0 con 0x5644ee75dd80 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37137fe640 10 client.16218 handle_command_reply: tid=0 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 client.16218 shutdown 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 msgr2=0x5644eeb4a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb4a840 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f37280057c0 tx=0x7f37280097f0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 shutdown_connections 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee75dd80 0x5644eeb5c470 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee79f1b0 0x5644eeb50140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb4a840 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37137fe640 0 client.16218 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 --2- 172.21.15.5:0/2494261640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.480+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 >> 172.21.15.5:0/2494261640 conn(0x5644ee793300 msgr2=0x5644eeb3f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.481+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 shutdown_connections 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.481+0000 7f37413a3740 1 -- 172.21.15.5:0/2494261640 wait complete. 2024-05-14T15:28:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.481+0000 7f37413a3740 20 client.16218 trim_cache size 0 max 16384 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.497+0000 7f37413a3740 1 Processor -- start 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.497+0000 7f37413a3740 1 -- start start 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.497+0000 7f37413a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee75dd80 0x5644eeb40080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37413a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37413a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37413a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b3b20 con 0x5644ee77e4d0 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37413a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b13f0 con 0x5644ee75dd80 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37413a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37341af240 con 0x5644ee7762f0 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb3bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46242/0 (socket says 172.21.15.5:46242) 2024-05-14T15:28:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 -- 172.21.15.5:0/3906817004 learned_addr learned my addr 172.21.15.5:0/3906817004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37337fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f3733fff640 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee75dd80 0x5644eeb40080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 -- 172.21.15.5:0/3906817004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 msgr2=0x5644eeb3b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb3b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 -- 172.21.15.5:0/3906817004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee75dd80 msgr2=0x5644eeb40080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee75dd80 0x5644eeb40080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 -- 172.21.15.5:0/3906817004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644eeb45c70 con 0x5644ee77e4d0 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f37389b2640 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb3bec0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f372c0a7200 tx=0x7f372c0aa690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbffa8bbcb421a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f3719ffb640 1 -- 172.21.15.5:0/3906817004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c0c4020 con 0x5644ee77e4d0 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f3719ffb640 1 -- 172.21.15.5:0/3906817004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c0b2df0 con 0x5644ee77e4d0 2024-05-14T15:28:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.498+0000 7f3719ffb640 1 -- 172.21.15.5:0/3906817004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c0b2bc0 con 0x5644ee77e4d0 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 -- 172.21.15.5:0/3906817004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 msgr2=0x5644eeb3bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb3bec0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f372c0a7200 tx=0x7f372c0aa690 comp rx=0 tx=0).stop 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 -- 172.21.15.5:0/3906817004 shutdown_connections 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee7762f0 0x5644eeb3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee77e4d0 0x5644eeb3bec0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 --2- 172.21.15.5:0/3906817004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee75dd80 0x5644eeb40080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 -- 172.21.15.5:0/3906817004 >> 172.21.15.5:0/3906817004 conn(0x5644ee793300 msgr2=0x5644eeb3f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 -- 172.21.15.5:0/3906817004 shutdown_connections 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.499+0000 7f37413a3740 1 -- 172.21.15.5:0/3906817004 wait complete. 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 -- start start 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 Processor -- start 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 0x5644eeb56910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb53c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee77e4d0 0x5644eeb542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37341af8b0 con 0x5644ee75dd80 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b0d20 con 0x5644ee7762f0 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37341aff80 con 0x5644ee77e4d0 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f37413a3740 10 client.0 _ll_register_callbacks cb 0x7f3724009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f3733fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 0x5644eeb56910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f3733fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 0x5644eeb56910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46252/0 (socket says 172.21.15.5:46252) 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.500+0000 7f3733fff640 1 -- 172.21.15.5:0/3364241454 learned_addr learned my addr 172.21.15.5:0/3364241454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37337fe640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb53c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37389b2640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee77e4d0 0x5644eeb542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f3733fff640 1 -- 172.21.15.5:0/3364241454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee77e4d0 msgr2=0x5644eeb542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f3733fff640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee77e4d0 0x5644eeb542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f3733fff640 1 -- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 msgr2=0x5644eeb53c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f3733fff640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb53c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f3733fff640 1 -- 172.21.15.5:0/3364241454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644eeb54a10 con 0x5644ee75dd80 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37337fe640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb53c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37389b2640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee77e4d0 0x5644eeb542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f3733fff640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 0x5644eeb56910 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5644eeb3d320 tx=0x7f37240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90da9d693dc0dd20 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37137fe640 1 -- 172.21.15.5:0/3364241454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400f7c0 con 0x5644ee75dd80 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37137fe640 1 -- 172.21.15.5:0/3364241454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f372400f960 con 0x5644ee75dd80 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c0b2df0 con 0x5644ee75dd80 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37137fe640 1 -- 172.21.15.5:0/3364241454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f372400fb60 con 0x5644ee75dd80 2024-05-14T15:28:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37413a3740 10 client.16224 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.501+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644eebc4620 con 0x5644ee75dd80 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37137fe640 1 -- 172.21.15.5:0/3364241454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c0b2df0 con 0x5644ee75dd80 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37137fe640 1 -- 172.21.15.5:0/3364241454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f372401f420 con 0x5644ee75dd80 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37413a3740 10 client.16224 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37413a3740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37413a3740 20 client.16224 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37413a3740 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee79f1b0 0x5644eebc6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37413a3740 4 client.16224 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644eeaad050 con 0x5644ee79f1b0 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37337fe640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee79f1b0 0x5644eebc6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.502+0000 7f37137fe640 10 client.16224 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.503+0000 7f37337fe640 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee79f1b0 0x5644eebc6d30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479c313fa05b0ae8 server_cookie=1995c63d09bce087 in_seq=0 out_seq=0 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37137fe640 1 -- 172.21.15.5:0/3364241454 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644eeaad050 con 0x5644ee79f1b0 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37137fe640 10 client.16224 handle_command_reply: tid=0 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 client.16224 shutdown 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 msgr2=0x5644eeb56910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 0x5644eeb56910 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5644eeb3d320 tx=0x7f37240046c0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 shutdown_connections 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5644ee79f1b0 0x5644eebc6d30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37137fe640 0 client.16224 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644ee77e4d0 0x5644eeb542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644ee7762f0 0x5644eeb53c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 --2- 172.21.15.5:0/3364241454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644ee75dd80 0x5644eeb56910 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 >> 172.21.15.5:0/3364241454 conn(0x5644ee793300 msgr2=0x5644eeb4b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 shutdown_connections 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.618+0000 7f37413a3740 1 -- 172.21.15.5:0/3364241454 wait complete. 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f37413a3740 20 client.16224 trim_cache size 0 max 16384 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:28:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f371c0b2950 msgr2=0x7f371c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f371c0b2950 0x7f371c0b4e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5644eeb40f90 tx=0x7f3724009260 comp rx=0 tx=0).stop 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 msgr2=0x7f373419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 0x7f373419a6d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f372c009fd0 tx=0x7f372c00ef10 comp rx=0 tx=0).stop 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 shutdown_connections 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f371c0b2950 0x7f371c0b4e60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f373410e1f0 0x7f373419a6d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f373410b800 0x7f37341a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 --2- 172.21.15.5:0/1531570072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3734104da0 0x7f37341a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.619+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 >> 172.21.15.5:0/1531570072 conn(0x7f37340782e0 msgr2=0x7f37341012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.620+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 shutdown_connections 2024-05-14T15:28:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:12.620+0000 7f373a415640 1 -- 172.21.15.5:0/1531570072 wait complete. 2024-05-14T15:28:13.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 213265 12.43 MB/sec warmup 94 sec latency 40.100 ms 2024-05-14T15:28:14.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:13 smithi005 ceph-mon[31559]: pgmap v106: 129 pgs: 129 active+clean; 44 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-05-14T15:28:14.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 215803 12.43 MB/sec warmup 95 sec latency 21.986 ms 2024-05-14T15:28:14.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:13 smithi144 ceph-mon[35650]: pgmap v106: 129 pgs: 129 active+clean; 44 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-05-14T15:28:14.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:13 smithi193 ceph-mon[36001]: pgmap v106: 129 pgs: 129 active+clean; 44 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.9 MiB/s wr, 595 op/s 2024-05-14T15:28:15.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 217826 12.42 MB/sec warmup 96 sec latency 14.046 ms 2024-05-14T15:28:16.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 220522 12.44 MB/sec warmup 97 sec latency 9.189 ms 2024-05-14T15:28:16.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:15 smithi144 ceph-mon[35650]: pgmap v107: 129 pgs: 129 active+clean; 44 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 328 op/s 2024-05-14T15:28:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:15 smithi193 ceph-mon[36001]: pgmap v107: 129 pgs: 129 active+clean; 44 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 328 op/s 2024-05-14T15:28:16.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:15 smithi005 ceph-mon[31559]: pgmap v107: 129 pgs: 129 active+clean; 44 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.2 MiB/s wr, 328 op/s 2024-05-14T15:28:17.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 223339 12.46 MB/sec warmup 98 sec latency 19.765 ms 2024-05-14T15:28:18.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 225604 12.44 MB/sec warmup 99 sec latency 14.395 ms 2024-05-14T15:28:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:17 smithi144 ceph-mon[35650]: pgmap v108: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 469 op/s 2024-05-14T15:28:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:17 smithi193 ceph-mon[36001]: pgmap v108: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 469 op/s 2024-05-14T15:28:18.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:17 smithi005 ceph-mon[31559]: pgmap v108: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 469 op/s 2024-05-14T15:28:19.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 227797 12.45 MB/sec warmup 100 sec latency 13.196 ms 2024-05-14T15:28:20.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230836 12.51 MB/sec warmup 101 sec latency 15.193 ms 2024-05-14T15:28:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:19 smithi144 ceph-mon[35650]: pgmap v109: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-14T15:28:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:19 smithi193 ceph-mon[36001]: pgmap v109: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-14T15:28:20.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:19 smithi005 ceph-mon[31559]: pgmap v109: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-14T15:28:21.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 233055 12.46 MB/sec warmup 102 sec latency 30.263 ms 2024-05-14T15:28:22.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 235122 12.47 MB/sec warmup 103 sec latency 12.915 ms 2024-05-14T15:28:22.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:21 smithi144 ceph-mon[35650]: pgmap v110: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-14T15:28:22.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:21 smithi193 ceph-mon[36001]: pgmap v110: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-14T15:28:22.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:21 smithi005 ceph-mon[31559]: pgmap v110: 129 pgs: 129 active+clean; 35 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 385 op/s 2024-05-14T15:28:23.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237989 12.52 MB/sec warmup 104 sec latency 19.587 ms 2024-05-14T15:28:24.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 240303 12.48 MB/sec warmup 105 sec latency 17.756 ms 2024-05-14T15:28:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:23 smithi144 ceph-mon[35650]: pgmap v111: 129 pgs: 129 active+clean; 42 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 465 op/s 2024-05-14T15:28:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:23 smithi193 ceph-mon[36001]: pgmap v111: 129 pgs: 129 active+clean; 42 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 465 op/s 2024-05-14T15:28:24.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:23 smithi005 ceph-mon[31559]: pgmap v111: 129 pgs: 129 active+clean; 42 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 465 op/s 2024-05-14T15:28:25.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 242331 12.48 MB/sec warmup 106 sec latency 21.014 ms 2024-05-14T15:28:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 -- 172.21.15.5:0/2472396052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 msgr2=0x7f9288102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 --2- 172.21.15.5:0/2472396052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f9288102c80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9270009900 tx=0x7f927002a410 comp rx=0 tx=0).stop 2024-05-14T15:28:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 -- 172.21.15.5:0/2472396052 shutdown_connections 2024-05-14T15:28:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 --2- 172.21.15.5:0/2472396052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92881032f0 0x7f928810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 --2- 172.21.15.5:0/2472396052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f9288102c80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 --2- 172.21.15.5:0/2472396052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9288108ba0 0x7f9288102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 -- 172.21.15.5:0/2472396052 >> 172.21.15.5:0/2472396052 conn(0x7f92880782e0 msgr2=0x7f9288078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 -- 172.21.15.5:0/2472396052 shutdown_connections 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.727+0000 7f928db15640 1 -- 172.21.15.5:0/2472396052 wait complete. 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.728+0000 7f928db15640 1 Processor -- start 2024-05-14T15:28:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.728+0000 7f928db15640 1 -- start start 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f928db15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 0x7f92881997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f928db15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288108ba0 0x7f9288199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f928db15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f928819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f928db15640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112d90 con 0x7f9288109040 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f928db15640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112c10 con 0x7f92881032f0 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f928db15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9288112f10 con 0x7f9288108ba0 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 0x7f92881997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f928819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 0x7f92881997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39614/0 (socket says 172.21.15.5:39614) 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 -- 172.21.15.5:0/281552924 learned_addr learned my addr 172.21.15.5:0/281552924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f928819dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52952/0 (socket says 172.21.15.5:52952) 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f927effd640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288108ba0 0x7f9288199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 -- 172.21.15.5:0/281552924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288108ba0 msgr2=0x7f9288199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288108ba0 0x7f9288199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 -- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 msgr2=0x7f928819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f928819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f92877fe640 1 -- 172.21.15.5:0/281552924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928819e600 con 0x7f92881032f0 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f9287fff640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f928819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.729+0000 7f927effd640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288108ba0 0x7f9288199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.730+0000 7f92877fe640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 0x7f92881997f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f927800e9c0 tx=0x7f927800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.730+0000 7f92857fa640 1 -- 172.21.15.5:0/281552924 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927800cd80 con 0x7f92881032f0 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.730+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928819e8f0 con 0x7f92881032f0 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.730+0000 7f92857fa640 1 -- 172.21.15.5:0/281552924 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92780054d0 con 0x7f92881032f0 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.730+0000 7f92857fa640 1 -- 172.21.15.5:0/281552924 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278010430 con 0x7f92881032f0 2024-05-14T15:28:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.730+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928806a4f0 con 0x7f92881032f0 2024-05-14T15:28:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 Processor -- start 2024-05-14T15:28:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 -- start start 2024-05-14T15:28:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fdfbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fdfc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d5fd615d0 con 0x556d5fa5b1b0 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d5fd1a890 con 0x556d5fa322f0 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92949c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d5fd5d590 con 0x556d5fa3a4d0 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fdfbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fdfbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52962/0 (socket says 172.21.15.5:52962) 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 -- 172.21.15.5:0/4283694145 learned_addr learned my addr 172.21.15.5:0/4283694145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f927effd640 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f92857fa640 1 -- 172.21.15.5:0/281552924 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f92780105d0 con 0x7f92881032f0 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 -- 172.21.15.5:0/4283694145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 msgr2=0x556d5fdfc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 -- 172.21.15.5:0/4283694145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 msgr2=0x556d5fdfc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fdfc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.733+0000 7f9287fff640 1 -- 172.21.15.5:0/4283694145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5fa73210 con 0x556d5fa5b1b0 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.734+0000 7f9287fff640 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fdfbbe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f928000e6f0 tx=0x7f9280009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d123fc6be0597b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.734+0000 7f927dffb640 1 -- 172.21.15.5:0/4283694145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928001c070 con 0x556d5fa5b1b0 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.734+0000 7f927dffb640 1 -- 172.21.15.5:0/4283694145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928000ee50 con 0x556d5fa5b1b0 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.734+0000 7f927dffb640 1 -- 172.21.15.5:0/4283694145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280004370 con 0x556d5fa5b1b0 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.734+0000 7f92857fa640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92740b6d60 0x7f92740b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.734+0000 7f92877fe640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92740b6d60 0x7f92740b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92857fa640 1 -- 172.21.15.5:0/281552924 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f92780e2dc0 con 0x7f92881032f0 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 -- 172.21.15.5:0/4283694145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 msgr2=0x556d5fdfbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fdfbbe0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f928000e6f0 tx=0x7f9280009500 comp rx=0 tx=0).stop 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 -- 172.21.15.5:0/4283694145 shutdown_connections 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fdfc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 --2- 172.21.15.5:0/4283694145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fdfbbe0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 -- 172.21.15.5:0/4283694145 >> 172.21.15.5:0/4283694145 conn(0x556d5fa4f300 msgr2=0x556d5fdf8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 -- 172.21.15.5:0/4283694145 shutdown_connections 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 -- 172.21.15.5:0/4283694145 wait complete. 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92877fe640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92740b6d60 0x7f92740b9270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x556d5fdf96d0 tx=0x7f9278076610 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.735+0000 7f92949c5740 1 -- start start 2024-05-14T15:28:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 Processor -- start 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa322f0 0x556d5fe061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa3a4d0 0x556d5fe06730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d5fe08d10 con 0x556d5fa5b1b0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d5fd1e370 con 0x556d5fa3a4d0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d5fe08e90 con 0x556d5fa322f0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f92949c5740 10 client.0 _ll_register_callbacks cb 0x556d5fa72a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52968/0 (socket says 172.21.15.5:52968) 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 -- 172.21.15.5:0/2804707790 learned_addr learned my addr 172.21.15.5:0/2804707790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f927effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa3a4d0 0x556d5fe06730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa322f0 0x556d5fe061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 -- 172.21.15.5:0/2804707790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa322f0 msgr2=0x556d5fe061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa322f0 0x556d5fe061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 -- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa3a4d0 msgr2=0x556d5fe06730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa3a4d0 0x556d5fe06730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.736+0000 7f9287fff640 1 -- 172.21.15.5:0/2804707790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5fb3f620 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f9287fff640 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe121c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f928000e530 tx=0x7f9280003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f11d6917c2aa5e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f9266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f9266ffd640 1 -- 172.21.15.5:0/2804707790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928002c060 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f9266ffd640 1 -- 172.21.15.5:0/2804707790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9280005180 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928000ee50 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f9266ffd640 1 -- 172.21.15.5:0/2804707790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9280033050 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f92949c5740 10 client.16236 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d5fe15080 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.737+0000 7f9266ffd640 1 -- 172.21.15.5:0/2804707790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928000ee50 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f9266ffd640 1 -- 172.21.15.5:0/2804707790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9280025980 con 0x556d5fa5b1b0 2024-05-14T15:28:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f92949c5740 10 client.16236 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f92949c5740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f92949c5740 20 client.16236 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f92949c5740 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5fa19d80 0x556d5fe18490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f92949c5740 4 client.16236 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d5fd6f1e0 con 0x556d5fa19d80 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f927effd640 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5fa19d80 0x556d5fe18490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f9266ffd640 10 client.16236 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.738+0000 7f927effd640 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5fa19d80 0x556d5fe18490 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdba77a8632533b9 server_cookie=6a71e77a8e624e57 in_seq=0 out_seq=0 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f9266ffd640 1 -- 172.21.15.5:0/2804707790 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d5fd6f1e0 con 0x556d5fa19d80 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f9266ffd640 10 client.16236 handle_command_reply: tid=0 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 client.16236 shutdown 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 msgr2=0x556d5fe121c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe121c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f928000e530 tx=0x7f9280003f50 comp rx=0 tx=0).stop 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 shutdown_connections 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5fa19d80 0x556d5fe18490 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe121c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa3a4d0 0x556d5fe06730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 --2- 172.21.15.5:0/2804707790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa322f0 0x556d5fe061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f9266ffd640 0 client.16236 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 >> 172.21.15.5:0/2804707790 conn(0x556d5fa4f300 msgr2=0x556d5fdfb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 shutdown_connections 2024-05-14T15:28:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 1 -- 172.21.15.5:0/2804707790 wait complete. 2024-05-14T15:28:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.741+0000 7f92949c5740 20 client.16236 trim_cache size 0 max 16384 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 Processor -- start 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 -- start start 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 0x556d5fe008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa5b1b0 0x556d5fe12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92881b1af0 con 0x556d5fa322f0 2024-05-14T15:28:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92881af3c0 con 0x556d5fa5b1b0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f92949c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92881ad210 con 0x556d5fa3a4d0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 0x556d5fe008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 0x556d5fe008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52972/0 (socket says 172.21.15.5:52972) 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f927effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 -- 172.21.15.5:0/2274300344 learned_addr learned my addr 172.21.15.5:0/2274300344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa5b1b0 0x556d5fe12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 -- 172.21.15.5:0/2274300344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 msgr2=0x556d5fdfc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 -- 172.21.15.5:0/2274300344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa5b1b0 msgr2=0x556d5fe12130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa5b1b0 0x556d5fe12130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f9287fff640 1 -- 172.21.15.5:0/2274300344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d5fe05f50 con 0x556d5fa322f0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.759+0000 7f927effd640 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.760+0000 7f9287fff640 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 0x556d5fe008b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9280000f80 tx=0x7f92800021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7783067818310985 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.760+0000 7f927d7fa640 1 -- 172.21.15.5:0/2274300344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928002c040 con 0x556d5fa322f0 2024-05-14T15:28:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.760+0000 7f927d7fa640 1 -- 172.21.15.5:0/2274300344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928000ee50 con 0x556d5fa322f0 2024-05-14T15:28:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.760+0000 7f927d7fa640 1 -- 172.21.15.5:0/2274300344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928001c030 con 0x556d5fa322f0 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 -- 172.21.15.5:0/2274300344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 msgr2=0x556d5fe008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 0x556d5fe008b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9280000f80 tx=0x7f92800021f0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 -- 172.21.15.5:0/2274300344 shutdown_connections 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fdfc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa5b1b0 0x556d5fe12130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 --2- 172.21.15.5:0/2274300344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa322f0 0x556d5fe008b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 -- 172.21.15.5:0/2274300344 >> 172.21.15.5:0/2274300344 conn(0x556d5fa4f300 msgr2=0x556d5fdfb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 -- 172.21.15.5:0/2274300344 shutdown_connections 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 -- 172.21.15.5:0/2274300344 wait complete. 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.761+0000 7f92949c5740 1 -- start start 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 Processor -- start 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fe181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fe18730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe18c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92881ad880 con 0x556d5fa5b1b0 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92881aecf0 con 0x556d5fa322f0 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92881adf50 con 0x556d5fa3a4d0 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f92949c5740 10 client.0 _ll_register_callbacks cb 0x7f928001a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe18c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fe181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fe181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39670/0 (socket says 172.21.15.5:39670) 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 -- 172.21.15.5:0/4143739229 learned_addr learned my addr 172.21.15.5:0/4143739229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f927effd640 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fe18730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 -- 172.21.15.5:0/4143739229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 msgr2=0x556d5fe18730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fe18730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 -- 172.21.15.5:0/4143739229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 msgr2=0x556d5fe18c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe18c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.762+0000 7f9287fff640 1 -- 172.21.15.5:0/4143739229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5fe7e630 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9287fff640 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fe181f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x556d5fce3cf0 tx=0x7f92800021f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f1c6c1bc9c8671 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9266ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9266ffd640 1 -- 172.21.15.5:0/4143739229 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928001c040 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928000ee50 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9266ffd640 1 -- 172.21.15.5:0/4143739229 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9280017760 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9266ffd640 1 -- 172.21.15.5:0/4143739229 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9280003d40 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 10 client.25865 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d5fe7f8a0 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9266ffd640 1 -- 172.21.15.5:0/4143739229 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928000ee50 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f9266ffd640 1 -- 172.21.15.5:0/4143739229 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9280025960 con 0x556d5fa322f0 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 10 client.25865 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 20 client.25865 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5f9ef1c0 0x556d5fe81fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 4 client.25865 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.763+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d5fd69050 con 0x556d5f9ef1c0 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.764+0000 7f927effd640 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5f9ef1c0 0x556d5fe81fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.764+0000 7f9266ffd640 10 client.25865 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:25.764+0000 7f927effd640 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5f9ef1c0 0x556d5fe81fb0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4236d14530f01776 server_cookie=acde6124c400808f in_seq=0 out_seq=0 2024-05-14T15:28:25.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:28:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f9266ffd640 1 -- 172.21.15.5:0/4143739229 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d5fd69050 con 0x556d5f9ef1c0 2024-05-14T15:28:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f9266ffd640 10 client.25865 handle_command_reply: tid=0 2024-05-14T15:28:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 client.25865 shutdown 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 msgr2=0x556d5fe181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fe181f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x556d5fce3cf0 tx=0x7f92800021f0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 shutdown_connections 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556d5f9ef1c0 0x556d5fe81fb0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d5fa3a4d0 0x556d5fe18730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d5fa5b1b0 0x556d5fe18c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 --2- 172.21.15.5:0/4143739229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d5fa322f0 0x556d5fe181f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 >> 172.21.15.5:0/4143739229 conn(0x556d5fa4f300 msgr2=0x556d5fe07690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f9266ffd640 0 client.25865 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 shutdown_connections 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.021+0000 7f92949c5740 1 -- 172.21.15.5:0/4143739229 wait complete. 2024-05-14T15:28:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.022+0000 7f92949c5740 20 client.25865 trim_cache size 0 max 16384 2024-05-14T15:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:28:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:28:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.022+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92740b6d60 msgr2=0x7f92740b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.022+0000 7f928db15640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92740b6d60 0x7f92740b9270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x556d5fdf96d0 tx=0x7f9278076610 comp rx=0 tx=0).stop 2024-05-14T15:28:26.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.022+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 msgr2=0x7f92881997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.022+0000 7f928db15640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 0x7f92881997f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f927800e9c0 tx=0x7f927800ee80 comp rx=0 tx=0).stop 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 shutdown_connections 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92740b6d60 0x7f92740b9270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288108ba0 0x7f9288199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9288109040 0x7f928819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 --2- 172.21.15.5:0/281552924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92881032f0 0x7f92881997f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 >> 172.21.15.5:0/281552924 conn(0x7f92880782e0 msgr2=0x7f92881012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 shutdown_connections 2024-05-14T15:28:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:26.023+0000 7f928db15640 1 -- 172.21.15.5:0/281552924 wait complete. 2024-05-14T15:28:26.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244901 12.51 MB/sec warmup 107 sec latency 89.616 ms 2024-05-14T15:28:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:26.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:25 smithi005 ceph-mon[31559]: pgmap v112: 129 pgs: 129 active+clean; 42 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-05-14T15:28:26.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:25 smithi144 ceph-mon[35650]: pgmap v112: 129 pgs: 129 active+clean; 42 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-05-14T15:28:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:25 smithi193 ceph-mon[36001]: pgmap v112: 129 pgs: 129 active+clean; 42 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-05-14T15:28:27.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 246742 12.46 MB/sec warmup 108 sec latency 72.809 ms 2024-05-14T15:28:28.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:27 smithi005 ceph-mon[31559]: pgmap v113: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.1 MiB/s wr, 454 op/s 2024-05-14T15:28:28.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 249015 12.47 MB/sec warmup 109 sec latency 21.976 ms 2024-05-14T15:28:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 -- 172.21.15.5:0/4055210867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 msgr2=0x7ff034108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 --2- 172.21.15.5:0/4055210867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff034108200 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff0240099c0 tx=0x7ff024009cd0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 -- 172.21.15.5:0/4055210867 shutdown_connections 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 --2- 172.21.15.5:0/4055210867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 0x7ff0341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 --2- 172.21.15.5:0/4055210867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 0x7ff0341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 --2- 172.21.15.5:0/4055210867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff034108200 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 -- 172.21.15.5:0/4055210867 >> 172.21.15.5:0/4055210867 conn(0x7ff0340fedc0 msgr2=0x7ff0341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 -- 172.21.15.5:0/4055210867 shutdown_connections 2024-05-14T15:28:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.198+0000 7ff03b29f700 1 -- 172.21.15.5:0/4055210867 wait complete. 2024-05-14T15:28:28.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.199+0000 7ff03b29f700 1 Processor -- start 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.199+0000 7ff03b29f700 1 -- start start 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03b29f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 0x7ff03419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03b29f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff03419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03b29f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 0x7ff03419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03b29f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff034113350 con 0x7ff034107df0 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03b29f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341131d0 con 0x7ff0341034c0 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03b29f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341134d0 con 0x7ff034108740 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff03419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff03419c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53010/0 (socket says 172.21.15.5:53010) 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 -- 172.21.15.5:0/846656611 learned_addr learned my addr 172.21.15.5:0/846656611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03903b700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 0x7ff03419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03983c700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 0x7ff03419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 -- 172.21.15.5:0/846656611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 msgr2=0x7ff03419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 0x7ff03419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 -- 172.21.15.5:0/846656611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 msgr2=0x7ff03419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.200+0000 7ff03883a700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 0x7ff03419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff03883a700 1 -- 172.21.15.5:0/846656611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0341a6fe0 con 0x7ff034107df0 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff03903b700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 0x7ff03419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff03983c700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 0x7ff03419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff03883a700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff03419c1d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff024005f50 tx=0x7ff024004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff02a7fc700 1 -- 172.21.15.5:0/846656611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024026070 con 0x7ff034107df0 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff02a7fc700 1 -- 172.21.15.5:0/846656611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024003c70 con 0x7ff034107df0 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff02a7fc700 1 -- 172.21.15.5:0/846656611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400f930 con 0x7ff034107df0 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff034199a80 con 0x7ff034107df0 2024-05-14T15:28:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.201+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff034199e50 con 0x7ff034107df0 2024-05-14T15:28:28.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.206+0000 7ff02a7fc700 1 -- 172.21.15.5:0/846656611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff024029070 con 0x7ff034107df0 2024-05-14T15:28:28.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.206+0000 7ff049cb3b80 1 Processor -- start 2024-05-14T15:28:28.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.206+0000 7ff049cb3b80 1 -- start start 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31a60e40 0x55bf31be4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31be4d80 0x55bf31bde8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 0x55bf31bdf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf31b00940 con 0x55bf31a60e40 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf31a65da0 con 0x55bf31be4d80 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf31afc900 con 0x55bf31bdee10 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03883a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 0x55bf31bdf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03983c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31a60e40 0x55bf31be4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03903b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31be4d80 0x55bf31bde8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03983c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31a60e40 0x55bf31be4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53016/0 (socket says 172.21.15.5:53016) 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03903b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31be4d80 0x55bf31bde8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39692/0 (socket says 172.21.15.5:39692) 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03983c700 1 -- 172.21.15.5:0/1995700940 learned_addr learned my addr 172.21.15.5:0/1995700940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.207+0000 7ff03883a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 0x55bf31bdf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59502/0 (socket says 172.21.15.5:59502) 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03883a700 1 -- 172.21.15.5:0/1995700940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31be4d80 msgr2=0x55bf31bde8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03883a700 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31be4d80 0x55bf31bde8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03883a700 1 -- 172.21.15.5:0/1995700940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31a60e40 msgr2=0x55bf31be4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03883a700 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31a60e40 0x55bf31be4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03883a700 1 -- 172.21.15.5:0/1995700940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf31a4d8d0 con 0x55bf31bdee10 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03883a700 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 0x55bf31bdf220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff0240e0490 tx=0x7ff0240eb760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff14b3db541e2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:28.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff02a7fc700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff0200b2a70 0x7ff0200b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff03983c700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff0200b2a70 0x7ff0200b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.208+0000 7ff02a7fc700 1 -- 172.21.15.5:0/846656611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7ff0240e3100 con 0x7ff034107df0 2024-05-14T15:28:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.209+0000 7ff01f7fe700 1 -- 172.21.15.5:0/1995700940 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a5f70 con 0x55bf31bdee10 2024-05-14T15:28:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.209+0000 7ff01f7fe700 1 -- 172.21.15.5:0/1995700940 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0240f0030 con 0x55bf31bdee10 2024-05-14T15:28:28.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.209+0000 7ff01f7fe700 1 -- 172.21.15.5:0/1995700940 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a9820 con 0x55bf31bdee10 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1995700940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 msgr2=0x55bf31bdf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff03983c700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff0200b2a70 0x7ff0200b4f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff02c010990 tx=0x7ff02c012040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 0x55bf31bdf220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff0240e0490 tx=0x7ff0240eb760 comp rx=0 tx=0).stop 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1995700940 shutdown_connections 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31bdee10 0x55bf31bdf220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31a60e40 0x55bf31be4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1995700940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31be4d80 0x55bf31bde8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1995700940 >> 172.21.15.5:0/1995700940 conn(0x55bf31aa8630 msgr2=0x55bf31bd9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1995700940 shutdown_connections 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1995700940 wait complete. 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 -- start start 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.210+0000 7ff049cb3b80 1 Processor -- start 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31be84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31be8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf31beb5e0 con 0x55bf31be4d80 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf31ab9ed0 con 0x55bf31bdee10 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf31beb760 con 0x55bf31a60e40 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff049cb3b80 10 client.0 _ll_register_callbacks cb 0x55bf31a98950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff03883a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff03883a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39704/0 (socket says 172.21.15.5:39704) 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff03883a700 1 -- 172.21.15.5:0/1040226950 learned_addr learned my addr 172.21.15.5:0/1040226950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff03903b700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31be84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.211+0000 7ff03983c700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31be8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff03883a700 1 -- 172.21.15.5:0/1040226950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 msgr2=0x55bf31be84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff03883a700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31be84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff03883a700 1 -- 172.21.15.5:0/1040226950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 msgr2=0x55bf31be8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff03883a700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31be8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff03883a700 1 -- 172.21.15.5:0/1040226950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf31ab7760 con 0x55bf31bdee10 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff03883a700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be8a20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff0240e0540 tx=0x7ff024009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471935bff3cc1907 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:28.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff01cff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff01cff9700 1 -- 172.21.15.5:0/1040226950 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a9a00 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0240f0030 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff01cff9700 1 -- 172.21.15.5:0/1040226950 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7ff0240a6340 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff01cff9700 1 -- 172.21.15.5:0/1040226950 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0240a9e80 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff049cb3b80 10 client.25877 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf31beb5e0 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.212+0000 7ff01cff9700 1 -- 172.21.15.5:0/1040226950 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0240f0030 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff01cff9700 1 -- 172.21.15.5:0/1040226950 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff0240ae0c0 con 0x55bf31bdee10 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff049cb3b80 10 client.25877 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff049cb3b80 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff049cb3b80 20 client.25877 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31bfa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff049cb3b80 4 client.25877 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf31b0e320 con 0x55bf31bf8510 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff03903b700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31bfa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.213+0000 7ff01cff9700 10 client.25877 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.214+0000 7ff03903b700 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31bfa950 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713eeba3eec78ad server_cookie=1672df53457ca151 in_seq=0 out_seq=0 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff01cff9700 1 -- 172.21.15.5:0/1040226950 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf31b0e320 con 0x55bf31bf8510 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff01cff9700 10 client.25877 handle_command_reply: tid=0 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 client.25877 shutdown 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 msgr2=0x55bf31be8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be8a20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff0240e0540 tx=0x7ff024009310 comp rx=0 tx=0).stop 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 shutdown_connections 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31bfa950 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31be8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff01cff9700 0 client.25877 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31be84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/1040226950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be8a20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 >> 172.21.15.5:0/1040226950 conn(0x55bf31aa8630 msgr2=0x55bf31bdd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 shutdown_connections 2024-05-14T15:28:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.215+0000 7ff049cb3b80 1 -- 172.21.15.5:0/1040226950 wait complete. 2024-05-14T15:28:28.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.216+0000 7ff049cb3b80 20 client.25877 trim_cache size 0 max 16384 2024-05-14T15:28:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.263+0000 7ff049cb3b80 1 Processor -- start 2024-05-14T15:28:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.263+0000 7ff049cb3b80 1 -- start start 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.263+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31bde8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.263+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.263+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31bd3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341b0bd0 con 0x55bf31be4d80 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341ae460 con 0x55bf31bdee10 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341ac2b0 con 0x55bf31a60e40 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31bde8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03983c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31bd3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03883a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31bde8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59534/0 (socket says 172.21.15.5:59534) 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03883a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39716/0 (socket says 172.21.15.5:39716) 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03983c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31bd3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53028/0 (socket says 172.21.15.5:53028) 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 -- 172.21.15.5:0/953248147 learned_addr learned my addr 172.21.15.5:0/953248147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 -- 172.21.15.5:0/953248147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 msgr2=0x55bf31be3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 -- 172.21.15.5:0/953248147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 msgr2=0x55bf31bd3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31bd3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 -- 172.21.15.5:0/953248147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf31bdb020 con 0x55bf31a60e40 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff03903b700 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31bde8d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff030012a20 tx=0x7ff030009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c90ff29d48fca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff01effd700 1 -- 172.21.15.5:0/953248147 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030026020 con 0x55bf31a60e40 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff01effd700 1 -- 172.21.15.5:0/953248147 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff030004ce0 con 0x55bf31a60e40 2024-05-14T15:28:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.264+0000 7ff01effd700 1 -- 172.21.15.5:0/953248147 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03000f7d0 con 0x55bf31a60e40 2024-05-14T15:28:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 -- 172.21.15.5:0/953248147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 msgr2=0x55bf31bde8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31bde8d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff030012a20 tx=0x7ff030009260 comp rx=0 tx=0).stop 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 -- 172.21.15.5:0/953248147 shutdown_connections 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31be4d80 0x55bf31bd3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31a60e40 0x55bf31bde8d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/953248147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31bdee10 0x55bf31be3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.265+0000 7ff049cb3b80 1 -- 172.21.15.5:0/953248147 >> 172.21.15.5:0/953248147 conn(0x55bf31aa8630 msgr2=0x55bf31bd3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.266+0000 7ff049cb3b80 1 -- 172.21.15.5:0/953248147 shutdown_connections 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.266+0000 7ff049cb3b80 1 -- 172.21.15.5:0/953248147 wait complete. 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.266+0000 7ff049cb3b80 1 -- start start 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.266+0000 7ff049cb3b80 1 Processor -- start 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31a60e40 0x55bf31bf0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 0x55bf31bff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31be4d80 0x55bf31bffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341ac920 con 0x55bf31bdee10 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341add90 con 0x55bf31a60e40 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0341acff0 con 0x55bf31be4d80 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff049cb3b80 10 client.0 _ll_register_callbacks cb 0x55bf31a98950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 0x55bf31bff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03903b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31a60e40 0x55bf31bf0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 0x55bf31bff500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53040/0 (socket says 172.21.15.5:53040) 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 -- 172.21.15.5:0/4105287723 learned_addr learned my addr 172.21.15.5:0/4105287723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03983c700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31be4d80 0x55bf31bffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 -- 172.21.15.5:0/4105287723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31be4d80 msgr2=0x55bf31bffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31be4d80 0x55bf31bffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 -- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31a60e40 msgr2=0x55bf31bf0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31a60e40 0x55bf31bf0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 -- 172.21.15.5:0/4105287723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf31c00030 con 0x55bf31bdee10 2024-05-14T15:28:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03903b700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31a60e40 0x55bf31bf0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff03883a700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 0x55bf31bff500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff0240a60e0 tx=0x7ff0240e0570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad34ed5c26f6ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.267+0000 7ff01cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff01cff9700 1 -- 172.21.15.5:0/4105287723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240a9a00 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff01cff9700 1 -- 172.21.15.5:0/4105287723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7ff0240ebab0 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff030004ce0 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff01cff9700 1 -- 172.21.15.5:0/4105287723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff024003e20 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff049cb3b80 10 client.16260 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf31c00030 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff01cff9700 1 -- 172.21.15.5:0/4105287723 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff030004ce0 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff01cff9700 1 -- 172.21.15.5:0/4105287723 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ff0240a6340 con 0x55bf31bdee10 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff049cb3b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff049cb3b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.268+0000 7ff049cb3b80 20 client.16260 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31c64cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff049cb3b80 4 client.16260 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf31b083c0 con 0x55bf31bf8510 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff03903b700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31c64cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff01cff9700 10 client.16260 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff03903b700 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31c64cc0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=951fa9fa4f25be server_cookie=ab6e1c3966485228 in_seq=0 out_seq=0 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff01cff9700 1 -- 172.21.15.5:0/4105287723 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bf31b083c0 con 0x55bf31bf8510 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff01cff9700 10 client.16260 handle_command_reply: tid=0 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.269+0000 7ff049cb3b80 1 client.16260 shutdown 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 msgr2=0x55bf31bff500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 0x55bf31bff500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff0240a60e0 tx=0x7ff0240e0570 comp rx=0 tx=0).stop 2024-05-14T15:28:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 shutdown_connections 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf31bf8510 0x55bf31c64cc0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf31be4d80 0x55bf31bffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf31a60e40 0x55bf31bf0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 --2- 172.21.15.5:0/4105287723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf31bdee10 0x55bf31bff500 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 >> 172.21.15.5:0/4105287723 conn(0x55bf31aa8630 msgr2=0x55bf31bd3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff01cff9700 0 client.16260 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 shutdown_connections 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 1 -- 172.21.15.5:0/4105287723 wait complete. 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.270+0000 7ff049cb3b80 20 client.16260 trim_cache size 0 max 16384 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:28:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:28:28.277 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:28:28.277 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:28:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff0200b2a70 msgr2=0x7ff0200b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff0200b2a70 0x7ff0200b4f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff02c010990 tx=0x7ff02c012040 comp rx=0 tx=0).stop 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 msgr2=0x7ff03419c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff03419c1d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff024005f50 tx=0x7ff024004270 comp rx=0 tx=0).stop 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 shutdown_connections 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff0200b2a70 0x7ff0200b4f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff034108740 0x7ff03419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0341034c0 0x7ff03419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 --2- 172.21.15.5:0/846656611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff034107df0 0x7ff03419c1d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 >> 172.21.15.5:0/846656611 conn(0x7ff0340fedc0 msgr2=0x7ff0341096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 shutdown_connections 2024-05-14T15:28:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:28.271+0000 7ff03b29f700 1 -- 172.21.15.5:0/846656611 wait complete. 2024-05-14T15:28:28.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:27 smithi144 ceph-mon[35650]: pgmap v113: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.1 MiB/s wr, 454 op/s 2024-05-14T15:28:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:27 smithi193 ceph-mon[36001]: pgmap v113: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.1 MiB/s wr, 454 op/s 2024-05-14T15:28:29.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251970 12.52 MB/sec warmup 110 sec latency 22.858 ms 2024-05-14T15:28:29.170 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:28:29.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:28:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:30.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:29 smithi005 ceph-mon[31559]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2024-05-14T15:28:30.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:29 smithi144 ceph-mon[35650]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2024-05-14T15:28:30.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:29 smithi193 ceph-mon[36001]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2024-05-14T15:28:30.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 254269 12.48 MB/sec warmup 111 sec latency 13.810 ms 2024-05-14T15:28:31.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 256206 12.48 MB/sec warmup 112 sec latency 13.793 ms 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 -- 172.21.15.5:0/1302635472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70110a60 msgr2=0x7f0b70112e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 --2- 172.21.15.5:0/1302635472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70110a60 0x7f0b70112e80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800b0a0 tx=0x7f0b6800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 -- 172.21.15.5:0/1302635472 shutdown_connections 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 --2- 172.21.15.5:0/1302635472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70110a60 0x7f0b70112e80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 --2- 172.21.15.5:0/1302635472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70102eb0 0x7f0b701032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 --2- 172.21.15.5:0/1302635472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70103890 0x7f0b70110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.241+0000 7f0b76e81700 1 -- 172.21.15.5:0/1302635472 >> 172.21.15.5:0/1302635472 conn(0x7f0b70078b10 msgr2=0x7f0b70078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.242+0000 7f0b76e81700 1 -- 172.21.15.5:0/1302635472 shutdown_connections 2024-05-14T15:28:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.242+0000 7f0b76e81700 1 -- 172.21.15.5:0/1302635472 wait complete. 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.242+0000 7f0b76e81700 1 Processor -- start 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.242+0000 7f0b76e81700 1 -- start start 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76e81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 0x7f0b7019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76e81700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70103890 0x7f0b7019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76e81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70110a60 0x7f0b701a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76e81700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701164b0 con 0x7f0b70102eb0 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76e81700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70116330 con 0x7f0b70110a60 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76e81700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70116630 con 0x7f0b70103890 2024-05-14T15:28:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 0x7f0b7019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 0x7f0b7019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53054/0 (socket says 172.21.15.5:53054) 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 -- 172.21.15.5:0/25831094 learned_addr learned my addr 172.21.15.5:0/25831094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b7567e700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70103890 0x7f0b7019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76680700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70110a60 0x7f0b701a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 -- 172.21.15.5:0/25831094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70103890 msgr2=0x7f0b7019d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70103890 0x7f0b7019d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 -- 172.21.15.5:0/25831094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70110a60 msgr2=0x7f0b701a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70110a60 0x7f0b701a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 -- 172.21.15.5:0/25831094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b701a1c60 con 0x7f0b70102eb0 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b7567e700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70103890 0x7f0b7019d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b76680700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70110a60 0x7f0b701a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.243+0000 7f0b75e7f700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 0x7f0b7019cde0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00e9a0 tx=0x7f0b6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.244+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00ca10 con 0x7f0b70102eb0 2024-05-14T15:28:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.244+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b701a1f50 con 0x7f0b70102eb0 2024-05-14T15:28:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.244+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b6c00cbb0 con 0x7f0b70102eb0 2024-05-14T15:28:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.244+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c018680 con 0x7f0b70102eb0 2024-05-14T15:28:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.244+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b701a9870 con 0x7f0b70102eb0 2024-05-14T15:28:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.245+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b700512c0 con 0x7f0b70102eb0 2024-05-14T15:28:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.246+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0b6c010430 con 0x7f0b70102eb0 2024-05-14T15:28:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.247+0000 7f0b66ffd700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0b5c0b29f0 0x7f0b5c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.247+0000 7f0b7567e700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0b5c0b29f0 0x7f0b5c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.247+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0b6c0e2e90 con 0x7f0b70102eb0 2024-05-14T15:28:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.248+0000 7f0b7567e700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0b5c0b29f0 0x7f0b5c0b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b70104850 tx=0x7f0b60009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.251+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b6c0aa4b0 con 0x7f0b70102eb0 2024-05-14T15:28:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.553+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b7002e5f0 con 0x7f0b70102eb0 2024-05-14T15:28:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.553+0000 7f0b66ffd700 1 -- 172.21.15.5:0/25831094 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11864 (secure 0 0 0) 0x7f0b6c0a9db0 con 0x7f0b70102eb0 2024-05-14T15:28:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:28:31.555 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:27:33.754380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25691":{"gid":25691,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2401236249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2401236249},{"type":"v1","addr":"172.21.15.144:6835","nonce":2401236249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:28:31.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0b5c0b29f0 msgr2=0x7f0b5c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:31.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0b5c0b29f0 0x7f0b5c0b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b70104850 tx=0x7f0b60009260 comp rx=0 tx=0).stop 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 msgr2=0x7f0b7019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 0x7f0b7019cde0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00e9a0 tx=0x7f0b6c00ecb0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 shutdown_connections 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0b5c0b29f0 0x7f0b5c0b4f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b70110a60 0x7f0b701a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b70102eb0 0x7f0b7019cde0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 --2- 172.21.15.5:0/25831094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b70103890 0x7f0b7019d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 >> 172.21.15.5:0/25831094 conn(0x7f0b70078b10 msgr2=0x7f0b70111f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 shutdown_connections 2024-05-14T15:28:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:31.557+0000 7f0b76e81700 1 -- 172.21.15.5:0/25831094 wait complete. 2024-05-14T15:28:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-05-14T15:28:32.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259215 12.54 MB/sec warmup 113 sec latency 26.810 ms 2024-05-14T15:28:32.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:31 smithi005 ceph-mon[31559]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2024-05-14T15:28:32.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:31 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/25831094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:28:32.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:31 smithi144 ceph-mon[35650]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2024-05-14T15:28:32.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:31 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/25831094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:28:32.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:31 smithi193 ceph-mon[36001]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 314 op/s 2024-05-14T15:28:32.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:31 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/25831094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:28:32.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:28:32.578 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:33.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 261527 12.50 MB/sec warmup 114 sec latency 15.792 ms 2024-05-14T15:28:34.130 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263491 12.49 MB/sec warmup 115 sec latency 14.434 ms 2024-05-14T15:28:34.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:33 smithi005 ceph-mon[31559]: pgmap v116: 129 pgs: 129 active+clean; 44 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 397 op/s 2024-05-14T15:28:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:33 smithi144 ceph-mon[35650]: pgmap v116: 129 pgs: 129 active+clean; 44 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 397 op/s 2024-05-14T15:28:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:33 smithi193 ceph-mon[36001]: pgmap v116: 129 pgs: 129 active+clean; 44 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 397 op/s 2024-05-14T15:28:35.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266313 12.54 MB/sec warmup 116 sec latency 52.165 ms 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 -- 172.21.15.5:0/33083749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1238102790 msgr2=0x7f1238102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 --2- 172.21.15.5:0/33083749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1238102790 0x7f1238102ba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f122c009990 tx=0x7f122c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 -- 172.21.15.5:0/33083749 shutdown_connections 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 --2- 172.21.15.5:0/33083749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12381030e0 0x7f1238111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 --2- 172.21.15.5:0/33083749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238109300 0x7f1238109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 --2- 172.21.15.5:0/33083749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1238102790 0x7f1238102ba0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 -- 172.21.15.5:0/33083749 >> 172.21.15.5:0/33083749 conn(0x7f12380fe540 msgr2=0x7f1238100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 -- 172.21.15.5:0/33083749 shutdown_connections 2024-05-14T15:28:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.355+0000 7f123daca700 1 -- 172.21.15.5:0/33083749 wait complete. 2024-05-14T15:28:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.356+0000 7f123daca700 1 Processor -- start 2024-05-14T15:28:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.356+0000 7f123daca700 1 -- start start 2024-05-14T15:28:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.356+0000 7f123daca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 0x7f123819d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.356+0000 7f123daca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 0x7f123819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.356+0000 7f123daca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1238109300 0x7f12381a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f123daca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12381145a0 con 0x7f12381030e0 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f123daca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1238114420 con 0x7f1238102790 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f123daca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1238114720 con 0x7f1238109300 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f12377fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 0x7f123819d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 0x7f123819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1237fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1238109300 0x7f12381a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f12377fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 0x7f123819d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39776/0 (socket says 172.21.15.5:39776) 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 0x7f123819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53070/0 (socket says 172.21.15.5:53070) 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f12377fe700 1 -- 172.21.15.5:0/95644626 learned_addr learned my addr 172.21.15.5:0/95644626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 -- 172.21.15.5:0/95644626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1238109300 msgr2=0x7f12381a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1238109300 0x7f12381a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 -- 172.21.15.5:0/95644626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 msgr2=0x7f123819d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 0x7f123819d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 -- 172.21.15.5:0/95644626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381a1fc0 con 0x7f12381030e0 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1237fff700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1238109300 0x7f12381a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f12377fe700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 0x7f123819d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1236ffd700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 0x7f123819d690 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098e0 tx=0x7f122c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.357+0000 7f1234ff9700 1 -- 172.21.15.5:0/95644626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c027070 con 0x7f12381030e0 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.358+0000 7f1234ff9700 1 -- 172.21.15.5:0/95644626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c022470 con 0x7f12381030e0 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.358+0000 7f1234ff9700 1 -- 172.21.15.5:0/95644626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c017650 con 0x7f12381030e0 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.358+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381a2250 con 0x7f12381030e0 2024-05-14T15:28:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.358+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a26b0 con 0x7f12381030e0 2024-05-14T15:28:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.360+0000 7f1234ff9700 1 -- 172.21.15.5:0/95644626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f122c02a070 con 0x7f12381030e0 2024-05-14T15:28:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.361+0000 7f1234ff9700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12200b29f0 0x7f12200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.362+0000 7f12377fe700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12200b29f0 0x7f12200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.362+0000 7f1234ff9700 1 -- 172.21.15.5:0/95644626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f122c0e3710 con 0x7f12381030e0 2024-05-14T15:28:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.362+0000 7f12377fe700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12200b29f0 0x7f12200b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1228011550 tx=0x7f1228009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 Processor -- start 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 -- start start 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af84840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af7f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55778aea0940 con 0x55778af84d80 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55778ae05da0 con 0x55778af7ee10 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f124c4deb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55778ae9c900 con 0x55778ae00e40 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af7e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53078/0 (socket says 172.21.15.5:53078) 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f1236ffd700 1 -- 172.21.15.5:0/202661049 learned_addr learned my addr 172.21.15.5:0/202661049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f1237fff700 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af7f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.363+0000 7f12377fe700 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af84840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f1236ffd700 1 -- 172.21.15.5:0/202661049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 msgr2=0x55778af84840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f1236ffd700 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af84840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f1236ffd700 1 -- 172.21.15.5:0/202661049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 msgr2=0x55778af7f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f1236ffd700 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af7f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f1236ffd700 1 -- 172.21.15.5:0/202661049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778aded8d0 con 0x55778af84d80 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f1236ffd700 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af7e8d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f122c0aa3a0 tx=0x7f122c0aa280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2b6563376fa813 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f121dffb700 1 -- 172.21.15.5:0/202661049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c0aea50 con 0x55778af84d80 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f121dffb700 1 -- 172.21.15.5:0/202661049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c0b3470 con 0x55778af84d80 2024-05-14T15:28:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.364+0000 7f121dffb700 1 -- 172.21.15.5:0/202661049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c022940 con 0x55778af84d80 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 -- 172.21.15.5:0/202661049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 msgr2=0x55778af7e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af7e8d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f122c0aa3a0 tx=0x7f122c0aa280 comp rx=0 tx=0).stop 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 -- 172.21.15.5:0/202661049 shutdown_connections 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af7f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af84840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 --2- 172.21.15.5:0/202661049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af7e8d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 -- 172.21.15.5:0/202661049 >> 172.21.15.5:0/202661049 conn(0x55778ae48630 msgr2=0x55778af79a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 -- 172.21.15.5:0/202661049 shutdown_connections 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.365+0000 7f124c4deb80 1 -- 172.21.15.5:0/202661049 wait complete. 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.366+0000 7f124c4deb80 1 -- start start 2024-05-14T15:28:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.366+0000 7f124c4deb80 1 Processor -- start 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.366+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af88530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.366+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 0x55778af88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af84d80 0x55778af88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f124c4deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55778af8b5e0 con 0x55778af7ee10 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f124c4deb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55778ae59ed0 con 0x55778af84d80 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f124c4deb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55778af8b760 con 0x55778ae00e40 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f124c4deb80 10 client.0 _ll_register_callbacks cb 0x55778ae38950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 0x55778af88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f12377fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af88530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 0x55778af88a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53082/0 (socket says 172.21.15.5:53082) 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 -- 172.21.15.5:0/1438760787 learned_addr learned my addr 172.21.15.5:0/1438760787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1237fff700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af84d80 0x55778af88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 -- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 msgr2=0x55778af88530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af88530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 -- 172.21.15.5:0/1438760787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af84d80 msgr2=0x55778af88fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af84d80 0x55778af88fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 -- 172.21.15.5:0/1438760787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778ae57760 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f12377fe700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af88530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1237fff700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af84d80 0x55778af88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1236ffd700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 0x55778af88a70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f122c000c00 tx=0x7f122c092360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f965692d7c26819b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.367+0000 7f1216ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f1216ffd700 1 -- 172.21.15.5:0/1438760787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c0aea50 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f1216ffd700 1 -- 172.21.15.5:0/1438760787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f122c0a6c90 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122c0b3470 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f1216ffd700 1 -- 172.21.15.5:0/1438760787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f122c0afbe0 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 10 client.16278 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55778af8b5e0 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f1216ffd700 1 -- 172.21.15.5:0/1438760787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c0b3470 con 0x55778af7ee10 2024-05-14T15:28:35.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f1216ffd700 1 -- 172.21.15.5:0/1438760787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f122c0aa9e0 con 0x55778af7ee10 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 10 client.16278 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 10 client.16278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 20 client.16278 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778af984f0 0x55778af9a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 4 client.16278 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.368+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55778aeae320 con 0x55778af984f0 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.369+0000 7f12377fe700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778af984f0 0x55778af9a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.369+0000 7f12377fe700 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778af984f0 0x55778af9a910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8c73f213ee8dcda server_cookie=938008e7a7936c42 in_seq=0 out_seq=0 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.369+0000 7f1216ffd700 10 client.16278 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f1216ffd700 1 -- 172.21.15.5:0/1438760787 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55778aeae320 con 0x55778af984f0 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f1216ffd700 10 client.16278 handle_command_reply: tid=0 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 client.16278 shutdown 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 msgr2=0x55778af88a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 0x55778af88a70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f122c000c00 tx=0x7f122c092360 comp rx=0 tx=0).stop 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 shutdown_connections 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778af984f0 0x55778af9a910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af84d80 0x55778af88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af88530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 --2- 172.21.15.5:0/1438760787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af7ee10 0x55778af88a70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 >> 172.21.15.5:0/1438760787 conn(0x55778ae48630 msgr2=0x55778af7d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f1216ffd700 0 client.16278 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.370+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 shutdown_connections 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.371+0000 7f124c4deb80 1 -- 172.21.15.5:0/1438760787 wait complete. 2024-05-14T15:28:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.371+0000 7f124c4deb80 20 client.16278 trim_cache size 0 max 16384 2024-05-14T15:28:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 Processor -- start 2024-05-14T15:28:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 -- start start 2024-05-14T15:28:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 0x55778af7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778af84d80 0x55778af87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b5030 con 0x55778ae00e40 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b28c0 con 0x55778af7ee10 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f124c4deb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b0710 con 0x55778af84d80 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f12377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 0x55778af7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f12377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 0x55778af7e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53092/0 (socket says 172.21.15.5:53092) 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.386+0000 7f12377fe700 1 -- 172.21.15.5:0/2576648345 learned_addr learned my addr 172.21.15.5:0/2576648345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f1236ffd700 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f1237fff700 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778af84d80 0x55778af87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f12377fe700 1 -- 172.21.15.5:0/2576648345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778af84d80 msgr2=0x55778af87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f12377fe700 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778af84d80 0x55778af87e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f12377fe700 1 -- 172.21.15.5:0/2576648345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 msgr2=0x55778af861a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f12377fe700 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af861a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f12377fe700 1 -- 172.21.15.5:0/2576648345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778af7f740 con 0x55778ae00e40 2024-05-14T15:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f1237fff700 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778af84d80 0x55778af87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f12377fe700 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 0x55778af7e8d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1228011ca0 tx=0x7f1228014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5395d3c4d4ae6248 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f121d7fa700 1 -- 172.21.15.5:0/2576648345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122801db60 con 0x55778ae00e40 2024-05-14T15:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f121d7fa700 1 -- 172.21.15.5:0/2576648345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1228030070 con 0x55778ae00e40 2024-05-14T15:28:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.387+0000 7f121d7fa700 1 -- 172.21.15.5:0/2576648345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12280053a0 con 0x55778ae00e40 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 -- 172.21.15.5:0/2576648345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 msgr2=0x55778af7e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 0x55778af7e8d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1228011ca0 tx=0x7f1228014000 comp rx=0 tx=0).stop 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 -- 172.21.15.5:0/2576648345 shutdown_connections 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778af84d80 0x55778af87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778ae00e40 0x55778af7e8d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2576648345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 -- 172.21.15.5:0/2576648345 >> 172.21.15.5:0/2576648345 conn(0x55778ae48630 msgr2=0x55778af73580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 -- 172.21.15.5:0/2576648345 shutdown_connections 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.388+0000 7f124c4deb80 1 -- 172.21.15.5:0/2576648345 wait complete. 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 -- start start 2024-05-14T15:28:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 Processor -- start 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af90bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af9f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b0d80 con 0x55778af84d80 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b21f0 con 0x55778af7ee10 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b1450 con 0x55778ae00e40 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f124c4deb80 10 client.0 _ll_register_callbacks cb 0x55778ae38950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f12377fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af90bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f1236ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af9f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f12377fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af90bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59622/0 (socket says 172.21.15.5:59622) 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f12377fe700 1 -- 172.21.15.5:0/2997486646 learned_addr learned my addr 172.21.15.5:0/2997486646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.389+0000 7f1237fff700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f12377fe700 1 -- 172.21.15.5:0/2997486646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 msgr2=0x55778af9f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f12377fe700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af9f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f12377fe700 1 -- 172.21.15.5:0/2997486646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 msgr2=0x55778af9f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f12377fe700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af9f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f12377fe700 1 -- 172.21.15.5:0/2997486646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778af91330 con 0x55778ae00e40 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f1237fff700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f12377fe700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af90bb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1228009e20 tx=0x7f122801dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05c29aa23f8cb3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f1216ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f1216ffd700 1 -- 172.21.15.5:0/2997486646 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228004ba0 con 0x55778ae00e40 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f1216ffd700 1 -- 172.21.15.5:0/2997486646 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f1228004f90 con 0x55778ae00e40 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1228030070 con 0x55778ae00e40 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f1216ffd700 1 -- 172.21.15.5:0/2997486646 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1228045050 con 0x55778ae00e40 2024-05-14T15:28:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.390+0000 7f124c4deb80 10 client.25900 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55778afa0a30 con 0x55778ae00e40 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f1216ffd700 1 -- 172.21.15.5:0/2997486646 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1228030070 con 0x55778ae00e40 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f1216ffd700 1 -- 172.21.15.5:0/2997486646 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f122802c6e0 con 0x55778ae00e40 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 10 client.25900 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 10 client.25900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 20 client.25900 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778b002030 0x55778b004450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 4 client.25900 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55778aea83c0 con 0x55778b002030 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.391+0000 7f1236ffd700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778b002030 0x55778b004450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f1236ffd700 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778b002030 0x55778b004450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=caadda9d8d232a6a server_cookie=7cf6221b129bb6e0 in_seq=0 out_seq=0 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f1216ffd700 10 client.25900 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f1216ffd700 1 -- 172.21.15.5:0/2997486646 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55778aea83c0 con 0x55778b002030 2024-05-14T15:28:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f1216ffd700 10 client.25900 handle_command_reply: tid=0 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 client.25900 shutdown 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 msgr2=0x55778af90bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af90bb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1228009e20 tx=0x7f122801dac0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 shutdown_connections 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55778af84d80 0x55778af9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55778ae00e40 0x55778af90bb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55778b002030 0x55778b004450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 --2- 172.21.15.5:0/2997486646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55778af7ee10 0x55778af9f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f1216ffd700 0 client.25900 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 >> 172.21.15.5:0/2997486646 conn(0x55778ae48630 msgr2=0x55778af73580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 shutdown_connections 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 1 -- 172.21.15.5:0/2997486646 wait complete. 2024-05-14T15:28:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.392+0000 7f124c4deb80 20 client.25900 trim_cache size 0 max 16384 2024-05-14T15:28:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:28:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12200b29f0 msgr2=0x7f12200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12200b29f0 0x7f12200b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1228011550 tx=0x7f1228009260 comp rx=0 tx=0).stop 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 msgr2=0x7f123819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 0x7f123819d690 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098e0 tx=0x7f122c00b670 comp rx=0 tx=0).stop 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 shutdown_connections 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f12200b29f0 0x7f12200b4f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1238109300 0x7f12381a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1238102790 0x7f123819d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 --2- 172.21.15.5:0/95644626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12381030e0 0x7f123819d690 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 >> 172.21.15.5:0/95644626 conn(0x7f12380fe540 msgr2=0x7f1238107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 shutdown_connections 2024-05-14T15:28:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:35.393+0000 7f123daca700 1 -- 172.21.15.5:0/95644626 wait complete. 2024-05-14T15:28:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:35 smithi005 ceph-mon[31559]: pgmap v117: 129 pgs: 129 active+clean; 44 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-05-14T15:28:36.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 268539 12.50 MB/sec warmup 117 sec latency 20.324 ms 2024-05-14T15:28:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:35 smithi144 ceph-mon[35650]: pgmap v117: 129 pgs: 129 active+clean; 44 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-05-14T15:28:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:35 smithi193 ceph-mon[36001]: pgmap v117: 129 pgs: 129 active+clean; 44 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 317 op/s 2024-05-14T15:28:36.761 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:28:37.035 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:37.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 270515 12.50 MB/sec warmup 118 sec latency 32.171 ms 2024-05-14T15:28:38.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273426 12.55 MB/sec warmup 119 sec latency 16.881 ms 2024-05-14T15:28:38.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:37 smithi005 ceph-mon[31559]: pgmap v118: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 396 op/s 2024-05-14T15:28:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:37 smithi144 ceph-mon[35650]: pgmap v118: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 396 op/s 2024-05-14T15:28:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:37 smithi193 ceph-mon[36001]: pgmap v118: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 396 op/s 2024-05-14T15:28:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 -- 172.21.15.5:0/1182810118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae0105dd0 msgr2=0x7fbae010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 --2- 172.21.15.5:0/1182810118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae0105dd0 0x7fbae010e320 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009990 tx=0x7fbad4009ca0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 -- 172.21.15.5:0/1182810118 shutdown_connections 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 --2- 172.21.15.5:0/1182810118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae010e8f0 0x7fbae0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 --2- 172.21.15.5:0/1182810118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 0x7fbae0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 --2- 172.21.15.5:0/1182810118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae0105dd0 0x7fbae010e320 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 -- 172.21.15.5:0/1182810118 >> 172.21.15.5:0/1182810118 conn(0x7fbae00fef30 msgr2=0x7fbae0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 -- 172.21.15.5:0/1182810118 shutdown_connections 2024-05-14T15:28:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.074+0000 7fbae780f700 1 -- 172.21.15.5:0/1182810118 wait complete. 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 Processor -- start 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 -- start start 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 0x7fbae019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae0105dd0 0x7fbae019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 0x7fbae019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0114450 con 0x7fbae010e8f0 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01142d0 con 0x7fbae0105dd0 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae780f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01145d0 con 0x7fbae0105480 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae5dac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 0x7fbae019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae5dac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 0x7fbae019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41418/0 (socket says 172.21.15.5:41418) 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.075+0000 7fbae5dac700 1 -- 172.21.15.5:0/3856289277 learned_addr learned my addr 172.21.15.5:0/3856289277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae4daa700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae0105dd0 0x7fbae019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae55ab700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 0x7fbae019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae5dac700 1 -- 172.21.15.5:0/3856289277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 msgr2=0x7fbae019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae5dac700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 0x7fbae019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae5dac700 1 -- 172.21.15.5:0/3856289277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae0105dd0 msgr2=0x7fbae019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae5dac700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae0105dd0 0x7fbae019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae5dac700 1 -- 172.21.15.5:0/3856289277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae019fca0 con 0x7fbae010e8f0 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.076+0000 7fbae55ab700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 0x7fbae019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.077+0000 7fbae5dac700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 0x7fbae019f600 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbad800ef30 tx=0x7fbad800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.077+0000 7fbad27fc700 1 -- 172.21.15.5:0/3856289277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad80092e0 con 0x7fbae010e8f0 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.077+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae019ff30 con 0x7fbae010e8f0 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.077+0000 7fbad27fc700 1 -- 172.21.15.5:0/3856289277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad8004bb0 con 0x7fbae010e8f0 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.077+0000 7fbad27fc700 1 -- 172.21.15.5:0/3856289277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004280 con 0x7fbae010e8f0 2024-05-14T15:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.077+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae01a0440 con 0x7fbae010e8f0 2024-05-14T15:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.080+0000 7fbad27fc700 1 -- 172.21.15.5:0/3856289277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbad801b030 con 0x7fbae010e8f0 2024-05-14T15:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbad27fc700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbacc0b2a70 0x7fbacc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbaf6223b80 1 Processor -- start 2024-05-14T15:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbaf6223b80 1 -- start start 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbad27fc700 1 -- 172.21.15.5:0/3856289277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fbad80e24a0 con 0x7fbae010e8f0 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbae55ab700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbacc0b2a70 0x7fbacc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.081+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bff75e9860 con 0x55bff7549e40 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bff75887c0 con 0x55bff76cc070 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bff75e5820 con 0x55bff76d0890 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae4daa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae5dac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae5dac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33280/0 (socket says 172.21.15.5:33280) 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae5dac700 1 -- 172.21.15.5:0/1575394709 learned_addr learned my addr 172.21.15.5:0/1575394709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae55ab700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbacc0b2a70 0x7fbacc0b4f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005fd0 tx=0x7fbadc005ee0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae55ab700 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae4daa700 1 -- 172.21.15.5:0/1575394709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 msgr2=0x55bff76d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae4daa700 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae4daa700 1 -- 172.21.15.5:0/1575394709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 msgr2=0x55bff76cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae4daa700 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.082+0000 7fbae4daa700 1 -- 172.21.15.5:0/1575394709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff75368d0 con 0x55bff76cc070 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.083+0000 7fbae4daa700 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d0350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbad40059d0 tx=0x7fbad4005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff286e3172540f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.083+0000 7fbae5dac700 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.083+0000 7fbacb7fe700 1 -- 172.21.15.5:0/1575394709 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4027070 con 0x55bff76cc070 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.083+0000 7fbacb7fe700 1 -- 172.21.15.5:0/1575394709 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad400bbd0 con 0x55bff76cc070 2024-05-14T15:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.083+0000 7fbacb7fe700 1 -- 172.21.15.5:0/1575394709 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4003cf0 con 0x55bff76cc070 2024-05-14T15:28:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 -- 172.21.15.5:0/1575394709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 msgr2=0x55bff76d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d0350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbad40059d0 tx=0x7fbad4005ec0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 -- 172.21.15.5:0/1575394709 shutdown_connections 2024-05-14T15:28:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/1575394709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d0350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 -- 172.21.15.5:0/1575394709 >> 172.21.15.5:0/1575394709 conn(0x55bff7591600 msgr2=0x55bff76c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 -- 172.21.15.5:0/1575394709 shutdown_connections 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.084+0000 7fbaf6223b80 1 -- 172.21.15.5:0/1575394709 wait complete. 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 -- start start 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 Processor -- start 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff7549e40 0x55bff76d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76d0890 0x55bff76d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bff76d87e0 con 0x55bff76d0890 2024-05-14T15:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bff754fc30 con 0x55bff76cc070 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bff76d8960 con 0x55bff7549e40 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbaf6223b80 10 client.0 _ll_register_callbacks cb 0x55bff7581920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbae4daa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbae5dac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76d0890 0x55bff76d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbae4daa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:45250/0 (socket says 172.21.15.5:45250) 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.085+0000 7fbae4daa700 1 -- 172.21.15.5:0/358812242 learned_addr learned my addr 172.21.15.5:0/358812242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae55ab700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff7549e40 0x55bff76d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae4daa700 1 -- 172.21.15.5:0/358812242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff7549e40 msgr2=0x55bff76d5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae4daa700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff7549e40 0x55bff76d5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae4daa700 1 -- 172.21.15.5:0/358812242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76d0890 msgr2=0x55bff76d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae4daa700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76d0890 0x55bff76d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae4daa700 1 -- 172.21.15.5:0/358812242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff7585690 con 0x55bff76cc070 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae5dac700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76d0890 0x55bff76d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae55ab700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff7549e40 0x55bff76d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbae4daa700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59dc925fd8aad286 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbac8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbac8ff9700 1 -- 172.21.15.5:0/358812242 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4031060 con 0x55bff76cc070 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.086+0000 7fbac8ff9700 1 -- 172.21.15.5:0/358812242 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fbad400b920 con 0x55bff76cc070 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.087+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad400bbd0 con 0x55bff76cc070 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.088+0000 7fbac8ff9700 1 -- 172.21.15.5:0/358812242 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbad4027070 con 0x55bff76cc070 2024-05-14T15:28:39.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.088+0000 7fbaf6223b80 10 client.25919 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:39.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.088+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bff76e3390 con 0x55bff76cc070 2024-05-14T15:28:39.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.097+0000 7fbac8ff9700 1 -- 172.21.15.5:0/358812242 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad400bbd0 con 0x55bff76cc070 2024-05-14T15:28:39.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.097+0000 7fbac8ff9700 1 -- 172.21.15.5:0/358812242 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbad400fa20 con 0x55bff76cc070 2024-05-14T15:28:39.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.097+0000 7fbaf6223b80 10 client.25919 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:39.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.097+0000 7fbaf6223b80 10 client.25919 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:28:39.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.098+0000 7fbaf6223b80 20 client.25919 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:39.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.098+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff76e5860 0x55bff76e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.098+0000 7fbaf6223b80 4 client.25919 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:39.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.098+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bff75f7240 con 0x55bff76e5860 2024-05-14T15:28:39.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.101+0000 7fbae55ab700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff76e5860 0x55bff76e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.102+0000 7fbae55ab700 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff76e5860 0x55bff76e7c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc97a2021bbeeadd server_cookie=2c94207baf714739 in_seq=0 out_seq=0 2024-05-14T15:28:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.102+0000 7fbac8ff9700 10 client.25919 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:28:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.105+0000 7fbac8ff9700 1 -- 172.21.15.5:0/358812242 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bff75f7240 con 0x55bff76e5860 2024-05-14T15:28:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.105+0000 7fbac8ff9700 10 client.25919 handle_command_reply: tid=0 2024-05-14T15:28:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.105+0000 7fbaf6223b80 1 client.25919 shutdown 2024-05-14T15:28:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.105+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 msgr2=0x55bff76d5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.105+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbad4000c00 tx=0x7fbad400b420 comp rx=0 tx=0).stop 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 shutdown_connections 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff76e5860 0x55bff76e7c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76d0890 0x55bff76d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff7549e40 0x55bff76d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/358812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5a90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 >> 172.21.15.5:0/358812242 conn(0x55bff7591600 msgr2=0x55bff76c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.106+0000 7fbac8ff9700 0 client.25919 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:28:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.107+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 shutdown_connections 2024-05-14T15:28:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.107+0000 7fbaf6223b80 1 -- 172.21.15.5:0/358812242 wait complete. 2024-05-14T15:28:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.107+0000 7fbaf6223b80 20 client.25919 trim_cache size 0 max 16384 2024-05-14T15:28:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.129+0000 7fbaf6223b80 1 Processor -- start 2024-05-14T15:28:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.129+0000 7fbaf6223b80 1 -- start start 2024-05-14T15:28:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01b0cd0 con 0x55bff7549e40 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01ae560 con 0x55bff76cc070 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbaf6223b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01ac3b0 con 0x55bff76d0890 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41446/0 (socket says 172.21.15.5:41446) 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 -- 172.21.15.5:0/408666793 learned_addr learned my addr 172.21.15.5:0/408666793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae4daa700 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae5dac700 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 -- 172.21.15.5:0/408666793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 msgr2=0x55bff76c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 -- 172.21.15.5:0/408666793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 msgr2=0x55bff76d5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.130+0000 7fbae55ab700 1 -- 172.21.15.5:0/408666793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bff76beae0 con 0x55bff7549e40 2024-05-14T15:28:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.131+0000 7fbae55ab700 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76c2f80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x55bff76cca80 tx=0x7fbadc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e262f92416ad08 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.131+0000 7fbacaffd700 1 -- 172.21.15.5:0/408666793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004020 con 0x55bff7549e40 2024-05-14T15:28:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.131+0000 7fbacaffd700 1 -- 172.21.15.5:0/408666793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbadc017040 con 0x55bff7549e40 2024-05-14T15:28:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.131+0000 7fbacaffd700 1 -- 172.21.15.5:0/408666793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0038e0 con 0x55bff7549e40 2024-05-14T15:28:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 -- 172.21.15.5:0/408666793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 msgr2=0x55bff76c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76c2f80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x55bff76cca80 tx=0x7fbadc013000 comp rx=0 tx=0).stop 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 -- 172.21.15.5:0/408666793 shutdown_connections 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff7549e40 0x55bff76c2f80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/408666793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff76cc070 0x55bff76d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 -- 172.21.15.5:0/408666793 >> 172.21.15.5:0/408666793 conn(0x55bff7591600 msgr2=0x55bff76c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 -- 172.21.15.5:0/408666793 shutdown_connections 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 -- 172.21.15.5:0/408666793 wait complete. 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 -- start start 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.132+0000 7fbaf6223b80 1 Processor -- start 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 0x55bff76eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76cc070 0x55bff76e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbae55ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 0x55bff76eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbae55ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 0x55bff76eaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:45272/0 (socket says 172.21.15.5:45272) 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbae55ab700 1 -- 172.21.15.5:0/2149692208 learned_addr learned my addr 172.21.15.5:0/2149692208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01aca20 con 0x55bff76cc070 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbae4daa700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76cc070 0x55bff76e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01ade90 con 0x55bff7549e40 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01ad0f0 con 0x55bff76d0890 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbaf6223b80 10 client.0 _ll_register_callbacks cb 0x55bff7581920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.133+0000 7fbae5dac700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae55ab700 1 -- 172.21.15.5:0/2149692208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 msgr2=0x55bff76e93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae55ab700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76e93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae55ab700 1 -- 172.21.15.5:0/2149692208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76cc070 msgr2=0x55bff76e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae55ab700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76cc070 0x55bff76e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae55ab700 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bff76e9900 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae5dac700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76e93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbae55ab700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 0x55bff76eaaf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005d90 tx=0x7fbadc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60f134cf73f56e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbac8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbac8ff9700 1 -- 172.21.15.5:0/2149692208 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc035070 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbadc017040 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.134+0000 7fbac8ff9700 1 -- 172.21.15.5:0/2149692208 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fbadc028540 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbac8ff9700 1 -- 172.21.15.5:0/2149692208 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbadc028a90 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 10 client.25925 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bff774e9f0 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbac8ff9700 1 -- 172.21.15.5:0/2149692208 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbadc017040 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbac8ff9700 1 -- 172.21.15.5:0/2149692208 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbadc00f040 con 0x55bff7549e40 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 10 client.25925 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 10 client.25925 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:28:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 20 client.25925 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:39.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff774f0d0 0x55bff77514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 4 client.25925 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:28:39.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.135+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bff75f12e0 con 0x55bff774f0d0 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.136+0000 7fbae4daa700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff774f0d0 0x55bff77514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.136+0000 7fbac8ff9700 10 client.25925 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.136+0000 7fbae4daa700 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff774f0d0 0x55bff77514f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fab39eb4622a2b6 server_cookie=28060097a40a5ea5 in_seq=0 out_seq=0 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.136+0000 7fbac8ff9700 1 -- 172.21.15.5:0/2149692208 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bff75f12e0 con 0x55bff774f0d0 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.136+0000 7fbac8ff9700 10 client.25925 handle_command_reply: tid=0 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 client.25925 shutdown 2024-05-14T15:28:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 msgr2=0x55bff76eaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 0x55bff76eaaf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005d90 tx=0x7fbadc005850 comp rx=0 tx=0).stop 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 shutdown_connections 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55bff774f0d0 0x55bff77514f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bff76d0890 0x55bff76e93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bff7549e40 0x55bff76eaaf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 --2- 172.21.15.5:0/2149692208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bff76cc070 0x55bff76e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbac8ff9700 0 client.25925 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 >> 172.21.15.5:0/2149692208 conn(0x55bff7591600 msgr2=0x55bff76d5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 shutdown_connections 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 1 -- 172.21.15.5:0/2149692208 wait complete. 2024-05-14T15:28:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.137+0000 7fbaf6223b80 20 client.25925 trim_cache size 0 max 16384 2024-05-14T15:28:39.143 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:28:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbacc0b2a70 msgr2=0x7fbacc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbacc0b2a70 0x7fbacc0b4f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005fd0 tx=0x7fbadc005ee0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 msgr2=0x7fbae019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 0x7fbae019f600 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbad800ef30 tx=0x7fbad800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 shutdown_connections 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbacc0b2a70 0x7fbacc0b4f80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae010e8f0 0x7fbae019f600 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbae0105480 0x7fbae019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 --2- 172.21.15.5:0/3856289277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbae0105dd0 0x7fbae019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 >> 172.21.15.5:0/3856289277 conn(0x7fbae00fef30 msgr2=0x7fbae00ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 shutdown_connections 2024-05-14T15:28:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:39.138+0000 7fbae780f700 1 -- 172.21.15.5:0/3856289277 wait complete. 2024-05-14T15:28:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/3959847463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 msgr2=0x7fc1c410f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/3959847463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c410f6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009be0 tx=0x7fc1bc02a6c0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/3959847463 shutdown_connections 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/3959847463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c4109050 0x7fc1c4102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/3959847463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c410f6d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/3959847463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4108bb0 0x7fc1c4102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/3959847463 >> 172.21.15.5:0/3959847463 conn(0x7fc1c4078280 msgr2=0x7fc1c40786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/3959847463 shutdown_connections 2024-05-14T15:28:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.149+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/3959847463 wait complete. 2024-05-14T15:28:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.150+0000 7fc1c8a6e640 1 Processor -- start 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.150+0000 7fc1c8a6e640 1 -- start start 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.150+0000 7fc1c8a6e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.150+0000 7fc1c8a6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c4108bb0 0x7fc1c419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.150+0000 7fc1c8a6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 0x7fc1c419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c8a6e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4112dc0 con 0x7fc1c4109050 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c8a6e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4112c40 con 0x7fc1c4103300 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c8a6e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4112f40 con 0x7fc1c4108bb0 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 0x7fc1c419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 0x7fc1c419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41462/0 (socket says 172.21.15.5:41462) 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:45286/0 (socket says 172.21.15.5:45286) 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c4108bb0 0x7fc1c419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1742986580 learned_addr learned my addr 172.21.15.5:0/1742986580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1742986580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c4108bb0 msgr2=0x7fc1c419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c4108bb0 0x7fc1c419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1742986580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 msgr2=0x7fc1c419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1742986580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c41a6a70 con 0x7fc1c4109050 2024-05-14T15:28:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.151+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 0x7fc1c419c1e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009b30 tx=0x7fc1bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.152+0000 7fc1b37fe640 1 -- 172.21.15.5:0/1742986580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc040020 con 0x7fc1c4109050 2024-05-14T15:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.152+0000 7fc1b37fe640 1 -- 172.21.15.5:0/1742986580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc0318f0 con 0x7fc1c4109050 2024-05-14T15:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.152+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c41a6d00 con 0x7fc1c4109050 2024-05-14T15:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.153+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c41a71a0 con 0x7fc1c4109050 2024-05-14T15:28:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.152+0000 7fc1b37fe640 1 -- 172.21.15.5:0/1742986580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc036040 con 0x7fc1c4109050 2024-05-14T15:28:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.155+0000 7fc1b37fe640 1 -- 172.21.15.5:0/1742986580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc1bc043020 con 0x7fc1c4109050 2024-05-14T15:28:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.156+0000 7fc1b37fe640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc1ac0b2aa0 0x7fc1ac0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.156+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc1ac0b2aa0 0x7fc1ac0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.156+0000 7fc1b37fe640 1 -- 172.21.15.5:0/1742986580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc1bc103490 con 0x7fc1c4109050 2024-05-14T15:28:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 Processor -- start 2024-05-14T15:28:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc1ac0b2aa0 0x7fc1ac0b4fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4103890 tx=0x7fc1b4009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 -- start start 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 0x55eb1fffbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb1fffc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 0x55eb1fffc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff615d0 con 0x55eb1fc5b1b0 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff1a890 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.157+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff5d590 con 0x55eb1fc322f0 2024-05-14T15:28:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 0x55eb1fffbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 0x55eb1fffc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 0x55eb1fffbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41464/0 (socket says 172.21.15.5:41464) 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 0x55eb1fffc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:45296/0 (socket says 172.21.15.5:45296) 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c259c640 1 -- 172.21.15.5:0/1935194925 learned_addr learned my addr 172.21.15.5:0/1935194925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb1fffc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1935194925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 msgr2=0x55eb1fffc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb1fffc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1935194925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 msgr2=0x55eb1fffbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 0x55eb1fffbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1935194925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1fc73210 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 0x55eb1fffbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 0x55eb1fffc660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc100820 tx=0x7fc1bc0ca000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad88e2c1359f9e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.158+0000 7fc1b0ff9640 1 -- 172.21.15.5:0/1935194925 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0d3210 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.159+0000 7fc1b0ff9640 1 -- 172.21.15.5:0/1935194925 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc100aa0 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.159+0000 7fc1b0ff9640 1 -- 172.21.15.5:0/1935194925 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc030c00 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.159+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1935194925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 msgr2=0x55eb1fffc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.159+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 0x55eb1fffc660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc100820 tx=0x7fc1bc0ca000 comp rx=0 tx=0).stop 2024-05-14T15:28:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.159+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1935194925 shutdown_connections 2024-05-14T15:28:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb1fffc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc3a4d0 0x55eb1fffc660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1935194925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc5b1b0 0x55eb1fffbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1935194925 >> 172.21.15.5:0/1935194925 conn(0x55eb1fc4f300 msgr2=0x55eb1fff8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1935194925 shutdown_connections 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1935194925 wait complete. 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 -- start start 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.160+0000 7fc1cfa03740 1 Processor -- start 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20006230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20006770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc5b1b0 0x55eb200102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20006230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20006230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33320/0 (socket says 172.21.15.5:33320) 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1c259c640 1 -- 172.21.15.5:0/2799475832 learned_addr learned my addr 172.21.15.5:0/2799475832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb2000cee0 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ff1e370 con 0x55eb1fc5b1b0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20006770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb2000d060 con 0x55eb1fc322f0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1cfa03740 10 client.0 _ll_register_callbacks cb 0x55eb1fc72a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.161+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc5b1b0 0x55eb200102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1c259c640 1 -- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc5b1b0 msgr2=0x55eb200102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1c259c640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc5b1b0 0x55eb200102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1c259c640 1 -- 172.21.15.5:0/2799475832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 msgr2=0x55eb20006770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1c259c640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20006770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1c259c640 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1fd3f620 con 0x55eb1fc322f0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1c259c640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20006230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4003a30 tx=0x7fc1b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d1066904b44fda server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/2799475832 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b401fdc0 con 0x55eb1fc322f0 2024-05-14T15:28:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/2799475832 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc1b40273e0 con 0x55eb1fc322f0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1bc100aa0 con 0x55eb1fc322f0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.162+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/2799475832 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1b40275e0 con 0x55eb1fc322f0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 10 client.25927 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb20019280 con 0x55eb1fc322f0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/2799475832 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc100aa0 con 0x55eb1fc322f0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/2799475832 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc1b4026070 con 0x55eb1fc322f0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 10 client.25927 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 20 client.25927 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc19d80 0x55eb2001c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 4 client.25927 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1ff6f1e0 con 0x55eb1fc19d80 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.163+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc19d80 0x55eb2001c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.164+0000 7fc1a9ffb640 10 client.25927 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.164+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc19d80 0x55eb2001c8d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a9cc3a3c9a10fb server_cookie=8cce563749c56cdf in_seq=0 out_seq=0 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/2799475832 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb1ff6f1e0 con 0x55eb1fc19d80 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1a9ffb640 10 client.25927 handle_command_reply: tid=0 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 client.25927 shutdown 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 msgr2=0x55eb20006230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20006230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4003a30 tx=0x7fc1b4004730 comp rx=0 tx=0).stop 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 shutdown_connections 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc19d80 0x55eb2001c8d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc5b1b0 0x55eb200102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20006770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/2799475832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20006230 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1a9ffb640 0 client.25927 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 >> 172.21.15.5:0/2799475832 conn(0x55eb1fc4f300 msgr2=0x55eb1fffd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 shutdown_connections 2024-05-14T15:28:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.165+0000 7fc1cfa03740 1 -- 172.21.15.5:0/2799475832 wait complete. 2024-05-14T15:28:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.166+0000 7fc1cfa03740 20 client.25927 trim_cache size 0 max 16384 2024-05-14T15:28:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 Processor -- start 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 -- start start 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 0x55eb20005340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc322f0 0x55eb20007e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc3a4d0 0x55eb20022090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41b19e0 con 0x55eb1fc19d80 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41af2b0 con 0x55eb1fc322f0 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ad100 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 0x55eb20005340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc322f0 0x55eb20007e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 0x55eb20005340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41476/0 (socket says 172.21.15.5:41476) 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.183+0000 7fc1c259c640 1 -- 172.21.15.5:0/1559784935 learned_addr learned my addr 172.21.15.5:0/1559784935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc3a4d0 0x55eb20022090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c259c640 1 -- 172.21.15.5:0/1559784935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc3a4d0 msgr2=0x55eb20022090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc3a4d0 0x55eb20022090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c259c640 1 -- 172.21.15.5:0/1559784935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc322f0 msgr2=0x55eb20007e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc322f0 0x55eb20007e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c259c640 1 -- 172.21.15.5:0/1559784935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1fff2530 con 0x55eb1fc19d80 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc322f0 0x55eb20007e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 0x55eb20005340 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4004050 tx=0x7fc1b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482ddf75d69d0919 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1abfff640 1 -- 172.21.15.5:0/1559784935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b403a020 con 0x55eb1fc19d80 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1abfff640 1 -- 172.21.15.5:0/1559784935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b401fbc0 con 0x55eb1fc19d80 2024-05-14T15:28:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.184+0000 7fc1abfff640 1 -- 172.21.15.5:0/1559784935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4005040 con 0x55eb1fc19d80 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1559784935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 msgr2=0x55eb20005340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 0x55eb20005340 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4004050 tx=0x7fc1b40046c0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1559784935 shutdown_connections 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc3a4d0 0x55eb20022090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc322f0 0x55eb20007e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1559784935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc19d80 0x55eb20005340 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1559784935 >> 172.21.15.5:0/1559784935 conn(0x55eb1fc4f300 msgr2=0x55eb20007760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1559784935 shutdown_connections 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.185+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1559784935 wait complete. 2024-05-14T15:28:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 -- start start 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 Processor -- start 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc19d80 0x55eb20013bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20014130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20082c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ad770 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41aebe0 con 0x55eb1fc19d80 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ade40 con 0x55eb1fc322f0 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1cfa03740 10 client.0 _ll_register_callbacks cb 0x7fc1b4004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20082c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1c2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20082c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41478/0 (socket says 172.21.15.5:41478) 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1690412047 learned_addr learned my addr 172.21.15.5:0/1690412047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc19d80 0x55eb20013bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.186+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20014130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1690412047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 msgr2=0x55eb20014130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20014130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc19d80 msgr2=0x55eb20013bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc19d80 0x55eb20013bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c2d9d640 1 -- 172.21.15.5:0/1690412047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb20083180 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc19d80 0x55eb20013bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c1d9b640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20014130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1c2d9d640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20082c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc003fd0 tx=0x7fc1bc0c9bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e63c4b7d696aee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/1690412047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0e9060 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/1690412047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc1bc0c69a0 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.187+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b401fbc0 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/1690412047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc0c6270 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 10 client.16320 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb20084340 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/1690412047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b401fbc0 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/1690412047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc1bc0d2d80 con 0x55eb1fc3a4d0 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 10 client.16320 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 20 client.16320 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc5b1b0 0x55eb20086ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 4 client.16320 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.188+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb1ff69050 con 0x55eb1fc5b1b0 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.189+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc5b1b0 0x55eb20086ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.189+0000 7fc1a9ffb640 10 client.16320 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.189+0000 7fc1c259c640 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc5b1b0 0x55eb20086ab0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92bd3ebec1d24246 server_cookie=bb0cb36111794a18 in_seq=0 out_seq=0 2024-05-14T15:28:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.443+0000 7fc1a9ffb640 1 -- 172.21.15.5:0/1690412047 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb1ff69050 con 0x55eb1fc5b1b0 2024-05-14T15:28:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.443+0000 7fc1a9ffb640 10 client.16320 handle_command_reply: tid=0 2024-05-14T15:28:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.443+0000 7fc1cfa03740 1 client.16320 shutdown 2024-05-14T15:28:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 msgr2=0x55eb20082c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20082c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc003fd0 tx=0x7fc1bc0c9bd0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 shutdown_connections 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eb1fc5b1b0 0x55eb20086ab0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb1fc322f0 0x55eb20014130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb1fc3a4d0 0x55eb20082c40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 --2- 172.21.15.5:0/1690412047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb1fc19d80 0x55eb20013bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 >> 172.21.15.5:0/1690412047 conn(0x55eb1fc4f300 msgr2=0x55eb200050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1a9ffb640 0 client.16320 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 shutdown_connections 2024-05-14T15:28:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 1 -- 172.21.15.5:0/1690412047 wait complete. 2024-05-14T15:28:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.444+0000 7fc1cfa03740 20 client.16320 trim_cache size 0 max 16384 2024-05-14T15:28:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:28:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:28:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:28:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:28:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc1ac0b2aa0 msgr2=0x7fc1ac0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc1ac0b2aa0 0x7fc1ac0b4fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4103890 tx=0x7fc1b4009260 comp rx=0 tx=0).stop 2024-05-14T15:28:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 msgr2=0x7fc1c419c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 0x7fc1c419c1e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009b30 tx=0x7fc1bc003c10 comp rx=0 tx=0).stop 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 shutdown_connections 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc1ac0b2aa0 0x7fc1ac0b4fb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1c4108bb0 0x7fc1c419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1c4109050 0x7fc1c419c1e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 --2- 172.21.15.5:0/1742986580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1c4103300 0x7fc1c419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 >> 172.21.15.5:0/1742986580 conn(0x7fc1c4078280 msgr2=0x7fc1c4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 shutdown_connections 2024-05-14T15:28:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:39.445+0000 7fc1c8a6e640 1 -- 172.21.15.5:0/1742986580 wait complete. 2024-05-14T15:28:40.105 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:39 smithi005 ceph-mon[31559]: pgmap v119: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-14T15:28:40.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 277234 11.98 MB/sec execute 1 sec latency 149.524 ms 2024-05-14T15:28:40.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:28:40.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:39 smithi144 ceph-mon[35650]: pgmap v119: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-14T15:28:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:39 smithi193 ceph-mon[36001]: pgmap v119: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-14T15:28:40.400 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:41.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280200 14.68 MB/sec execute 2 sec latency 17.900 ms 2024-05-14T15:28:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:41 smithi005 ceph-mon[31559]: pgmap v120: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-14T15:28:42.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 282132 12.17 MB/sec execute 3 sec latency 14.022 ms 2024-05-14T15:28:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 -- 172.21.15.5:0/1101019183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 msgr2=0x7fc578105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/1101019183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc578105890 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc574009990 tx=0x7fc574009ca0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 -- 172.21.15.5:0/1101019183 shutdown_connections 2024-05-14T15:28:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/1101019183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 0x7fc578110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/1101019183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc578105890 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/1101019183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 0x7fc57810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 -- 172.21.15.5:0/1101019183 >> 172.21.15.5:0/1101019183 conn(0x7fc5780fef30 msgr2=0x7fc578101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 -- 172.21.15.5:0/1101019183 shutdown_connections 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 -- 172.21.15.5:0/1101019183 wait complete. 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.189+0000 7fc57f6cf700 1 Processor -- start 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 -- start start 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc57819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 0x7fc57819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 0x7fc57819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc578114440 con 0x7fc578105480 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781142c0 con 0x7fc578105dd0 2024-05-14T15:28:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57f6cf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781145c0 con 0x7fc57810e8f0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57d46b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc57819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57cc6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 0x7fc57819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57d46b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc57819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41494/0 (socket says 172.21.15.5:41494) 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57d46b700 1 -- 172.21.15.5:0/4252007569 learned_addr learned my addr 172.21.15.5:0/4252007569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57dc6c700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 0x7fc57819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57cc6a700 1 -- 172.21.15.5:0/4252007569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 msgr2=0x7fc57819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 0x7fc57819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57cc6a700 1 -- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 msgr2=0x7fc57819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc57819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57cc6a700 1 -- 172.21.15.5:0/4252007569 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57819fca0 con 0x7fc578105dd0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57d46b700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc57819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.190+0000 7fc57dc6c700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 0x7fc57819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.191+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 0x7fc57819b370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc56800e9f0 tx=0x7fc56800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.191+0000 7fc56e7fc700 1 -- 172.21.15.5:0/4252007569 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56800c9c0 con 0x7fc578105dd0 2024-05-14T15:28:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.191+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57819ff90 con 0x7fc578105dd0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.191+0000 7fc56e7fc700 1 -- 172.21.15.5:0/4252007569 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56800cb60 con 0x7fc578105dd0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.191+0000 7fc56e7fc700 1 -- 172.21.15.5:0/4252007569 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568018680 con 0x7fc578105dd0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.191+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781a78b0 con 0x7fc578105dd0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.195+0000 7fc56e7fc700 1 -- 172.21.15.5:0/4252007569 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc568010430 con 0x7fc578105dd0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc56e7fc700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5640b29f0 0x7fc5640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 Processor -- start 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 -- start start 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc57d46b700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5640b29f0 0x7fc5640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc56e7fc700 1 -- 172.21.15.5:0/4252007569 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc5680e2b30 con 0x7fc578105dd0 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5336840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5331220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561cc5252940 con 0x561cc5336d80 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cc51b7da0 con 0x561cc5330e10 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cc524e900 con 0x561cc51b2e40 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc57cc6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc57cc6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41506/0 (socket says 172.21.15.5:41506) 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc57cc6a700 1 -- 172.21.15.5:0/1593894382 learned_addr learned my addr 172.21.15.5:0/1593894382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc57dc6c700 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5331220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.196+0000 7fc57d46b700 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5336840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57d46b700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5640b29f0 0x7fc5640b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b2e0 tx=0x7fc574004710 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57cc6a700 1 -- 172.21.15.5:0/1593894382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 msgr2=0x561cc5336840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5336840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57cc6a700 1 -- 172.21.15.5:0/1593894382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 msgr2=0x561cc5331220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5331220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57cc6a700 1 -- 172.21.15.5:0/1593894382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc519f8d0 con 0x561cc5336d80 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53308d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc5680a99e0 tx=0x7fc5680a9560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e5138ef93c3795 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc5637fe700 1 -- 172.21.15.5:0/1593894382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5680a5050 con 0x561cc5336d80 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc5637fe700 1 -- 172.21.15.5:0/1593894382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5680b2040 con 0x561cc5336d80 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.197+0000 7fc5637fe700 1 -- 172.21.15.5:0/1593894382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5680b2420 con 0x561cc5336d80 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/1593894382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 msgr2=0x561cc53308d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53308d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc5680a99e0 tx=0x7fc5680a9560 comp rx=0 tx=0).stop 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/1593894382 shutdown_connections 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5331220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5336840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/1593894382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53308d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/1593894382 >> 172.21.15.5:0/1593894382 conn(0x561cc51fa630 msgr2=0x561cc532ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/1593894382 shutdown_connections 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/1593894382 wait complete. 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.198+0000 7fc58e0e3b80 1 -- start start 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 Processor -- start 2024-05-14T15:28:42.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc533a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 0x561cc533aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5336d80 0x561cc533afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561cc533d7c0 con 0x561cc5330e10 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cc520bed0 con 0x561cc5336d80 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cc533d940 con 0x561cc51b2e40 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc58e0e3b80 10 client.0 _ll_register_callbacks cb 0x561cc51ea950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc57cc6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 0x561cc533aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc57cc6a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 0x561cc533aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41510/0 (socket says 172.21.15.5:41510) 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc57cc6a700 1 -- 172.21.15.5:0/3221032521 learned_addr learned my addr 172.21.15.5:0/3221032521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc533a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.199+0000 7fc57dc6c700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5336d80 0x561cc533afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57d46b700 1 -- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5336d80 msgr2=0x561cc533afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5336d80 0x561cc533afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57d46b700 1 -- 172.21.15.5:0/3221032521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 msgr2=0x561cc533aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 0x561cc533aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57d46b700 1 -- 172.21.15.5:0/3221032521 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc5209760 con 0x561cc51b2e40 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 0x561cc533aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc533a520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc574006010 tx=0x7fc574009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d402b4d72b55b51b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc560ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc560ff9700 1 -- 172.21.15.5:0/3221032521 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574032030 con 0x561cc51b2e40 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc560ff9700 1 -- 172.21.15.5:0/3221032521 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc574003b30 con 0x561cc51b2e40 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5680b2040 con 0x561cc51b2e40 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc560ff9700 1 -- 172.21.15.5:0/3221032521 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc57403c050 con 0x561cc51b2e40 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc58e0e3b80 10 client.25945 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561cc5348420 con 0x561cc51b2e40 2024-05-14T15:28:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.200+0000 7fc560ff9700 1 -- 172.21.15.5:0/3221032521 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5680b2040 con 0x561cc51b2e40 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc560ff9700 1 -- 172.21.15.5:0/3221032521 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc574022030 con 0x561cc51b2e40 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc58e0e3b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc58e0e3b80 10 client.25945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc58e0e3b80 20 client.25945 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc534aa60 0x561cc534cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc58e0e3b80 4 client.25945 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cc5260320 con 0x561cc534aa60 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc534aa60 0x561cc534cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc560ff9700 10 client.25945 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.201+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc534aa60 0x561cc534cea0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86c9e5bc472597e server_cookie=40d4dbe62a0f8b95 in_seq=0 out_seq=0 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc560ff9700 1 -- 172.21.15.5:0/3221032521 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561cc5260320 con 0x561cc534aa60 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc560ff9700 10 client.25945 handle_command_reply: tid=0 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 client.25945 shutdown 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 msgr2=0x561cc533a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc533a520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc574006010 tx=0x7fc574009310 comp rx=0 tx=0).stop 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 shutdown_connections 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc534aa60 0x561cc534cea0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5336d80 0x561cc533afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc533a520 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc560ff9700 0 client.25945 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3221032521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5330e10 0x561cc533aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.202+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 >> 172.21.15.5:0/3221032521 conn(0x561cc51fa630 msgr2=0x561cc532f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.203+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 shutdown_connections 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.203+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3221032521 wait complete. 2024-05-14T15:28:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.203+0000 7fc58e0e3b80 20 client.25945 trim_cache size 0 max 16384 2024-05-14T15:28:42.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 Processor -- start 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 -- start start 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc53308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5339840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781b2f10 con 0x561cc5336d80 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781b07a0 con 0x561cc5330e10 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ae5f0 con 0x561cc51b2e40 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc57d46b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc53308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc57dc6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc57d46b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc53308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33378/0 (socket says 172.21.15.5:33378) 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc57cc6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5339840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.219+0000 7fc57d46b700 1 -- 172.21.15.5:0/3835497176 learned_addr learned my addr 172.21.15.5:0/3835497176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:42.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57d46b700 1 -- 172.21.15.5:0/3835497176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 msgr2=0x561cc5339840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5339840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57d46b700 1 -- 172.21.15.5:0/3835497176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 msgr2=0x561cc53381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57d46b700 1 -- 172.21.15.5:0/3835497176 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc532bf40 con 0x561cc51b2e40 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57dc6c700 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc57d46b700 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc53308d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc57402e490 tx=0x7fc57402e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c45d2bf273384e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc562ffd700 1 -- 172.21.15.5:0/3835497176 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574044070 con 0x561cc51b2e40 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc562ffd700 1 -- 172.21.15.5:0/3835497176 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5740039b0 con 0x561cc51b2e40 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.220+0000 7fc562ffd700 1 -- 172.21.15.5:0/3835497176 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574032040 con 0x561cc51b2e40 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3835497176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 msgr2=0x561cc53308d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc53308d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc57402e490 tx=0x7fc57402e890 comp rx=0 tx=0).stop 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3835497176 shutdown_connections 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc53381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc53308d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/3835497176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5339840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3835497176 >> 172.21.15.5:0/3835497176 conn(0x561cc51fa630 msgr2=0x561cc5325bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3835497176 shutdown_connections 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.221+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/3835497176 wait complete. 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 -- start start 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 Processor -- start 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5342bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5343110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc5351650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781aec60 con 0x561cc5336d80 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781b00d0 con 0x561cc5330e10 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781af330 con 0x561cc51b2e40 2024-05-14T15:28:42.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc58e0e3b80 10 client.0 _ll_register_callbacks cb 0x561cc51ea950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc57dc6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc5351650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc57d46b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5342bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc57dc6c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc5351650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41528/0 (socket says 172.21.15.5:41528) 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc57dc6c700 1 -- 172.21.15.5:0/4212063437 learned_addr learned my addr 172.21.15.5:0/4212063437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.222+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5343110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc57d46b700 1 -- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 msgr2=0x561cc5343110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc57d46b700 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5343110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc57d46b700 1 -- 172.21.15.5:0/4212063437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 msgr2=0x561cc5351650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc57d46b700 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc5351650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc57d46b700 1 -- 172.21.15.5:0/4212063437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc5351b90 con 0x561cc51b2e40 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc57d46b700 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5342bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c010 tx=0x7fc574017cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a14bf90e8085862 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc560ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc560ff9700 1 -- 172.21.15.5:0/4212063437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57404d070 con 0x561cc51b2e40 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc560ff9700 1 -- 172.21.15.5:0/4212063437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fc57402ea30 con 0x561cc51b2e40 2024-05-14T15:28:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5740039b0 con 0x561cc51b2e40 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc560ff9700 1 -- 172.21.15.5:0/4212063437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc574058050 con 0x561cc51b2e40 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc58e0e3b80 10 client.25951 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.223+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561cc53b48b0 con 0x561cc51b2e40 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc560ff9700 1 -- 172.21.15.5:0/4212063437 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5740039b0 con 0x561cc51b2e40 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc560ff9700 1 -- 172.21.15.5:0/4212063437 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fc574005140 con 0x561cc51b2e40 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc58e0e3b80 10 client.25951 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc58e0e3b80 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc58e0e3b80 20 client.25951 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc53b4e50 0x561cc53b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc58e0e3b80 4 client.25951 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cc525a3c0 con 0x561cc53b4e50 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.224+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc53b4e50 0x561cc53b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc57cc6a700 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc53b4e50 0x561cc53b7270 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5003df991c8dd67 server_cookie=9c02f46f9bf263f3 in_seq=0 out_seq=0 2024-05-14T15:28:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc560ff9700 10 client.25951 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:42.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc560ff9700 1 -- 172.21.15.5:0/4212063437 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cc525a3c0 con 0x561cc53b4e50 2024-05-14T15:28:42.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc560ff9700 10 client.25951 handle_command_reply: tid=0 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 client.25951 shutdown 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 msgr2=0x561cc5342bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5342bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c010 tx=0x7fc574017cb0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 shutdown_connections 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561cc53b4e50 0x561cc53b7270 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cc5336d80 0x561cc5351650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cc51b2e40 0x561cc5342bd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc560ff9700 0 client.25951 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 --2- 172.21.15.5:0/4212063437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cc5330e10 0x561cc5343110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 >> 172.21.15.5:0/4212063437 conn(0x561cc51fa630 msgr2=0x561cc5334150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 shutdown_connections 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 1 -- 172.21.15.5:0/4212063437 wait complete. 2024-05-14T15:28:42.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.225+0000 7fc58e0e3b80 20 client.25951 trim_cache size 0 max 16384 2024-05-14T15:28:42.233 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:28:42.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5640b29f0 msgr2=0x7fc5640b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5640b29f0 0x7fc5640b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b2e0 tx=0x7fc574004710 comp rx=0 tx=0).stop 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 msgr2=0x7fc57819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 0x7fc57819b370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc56800e9f0 tx=0x7fc56800ed00 comp rx=0 tx=0).stop 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 shutdown_connections 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5640b29f0 0x7fc5640b4f00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc57810e8f0 0x7fc57819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc578105480 0x7fc57819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 --2- 172.21.15.5:0/4252007569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc578105dd0 0x7fc57819b370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 >> 172.21.15.5:0/4252007569 conn(0x7fc5780fef30 msgr2=0x7fc5780ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 shutdown_connections 2024-05-14T15:28:42.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:42.226+0000 7fc57f6cf700 1 -- 172.21.15.5:0/4252007569 wait complete. 2024-05-14T15:28:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:41 smithi144 ceph-mon[35650]: pgmap v120: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-14T15:28:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:41 smithi193 ceph-mon[36001]: pgmap v120: 129 pgs: 129 active+clean; 51 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-14T15:28:43.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284506 12.61 MB/sec execute 4 sec latency 13.418 ms 2024-05-14T15:28:44.045 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:28:44.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:28:44.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287419 13.58 MB/sec execute 5 sec latency 19.929 ms 2024-05-14T15:28:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:43 smithi144 ceph-mon[35650]: pgmap v121: 129 pgs: 129 active+clean; 45 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.2 MiB/s wr, 400 op/s 2024-05-14T15:28:44.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:43 smithi193 ceph-mon[36001]: pgmap v121: 129 pgs: 129 active+clean; 45 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.2 MiB/s wr, 400 op/s 2024-05-14T15:28:44.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:43 smithi005 ceph-mon[31559]: pgmap v121: 129 pgs: 129 active+clean; 45 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.2 MiB/s wr, 400 op/s 2024-05-14T15:28:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:45.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 288382 11.88 MB/sec execute 6 sec latency 72.995 ms 2024-05-14T15:28:46.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:45 smithi005 ceph-mon[31559]: pgmap v122: 129 pgs: 129 active+clean; 45 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2140544374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc24107df0 msgr2=0x7fcc24108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2140544374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc24107df0 0x7fcc24108200 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcc180099c0 tx=0x7fcc18009cd0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2140544374 shutdown_connections 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2140544374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24108740 0x7fcc241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2140544374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc241034c0 0x7fcc241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2140544374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc24107df0 0x7fcc24108200 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2140544374 >> 172.21.15.5:0/2140544374 conn(0x7fcc240fedc0 msgr2=0x7fcc241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2140544374 shutdown_connections 2024-05-14T15:28:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.122+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2140544374 wait complete. 2024-05-14T15:28:46.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.123+0000 7fcc28d1c700 1 Processor -- start 2024-05-14T15:28:46.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.123+0000 7fcc28d1c700 1 -- start start 2024-05-14T15:28:46.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc28d1c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 0x7fcc2419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc28d1c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24107df0 0x7fcc2419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc28d1c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc24108740 0x7fcc2419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc28d1c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24113370 con 0x7fcc241034c0 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc28d1c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241131f0 con 0x7fcc24107df0 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc28d1c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241134f0 con 0x7fcc24108740 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 0x7fcc2419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 0x7fcc2419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32988/0 (socket says 172.21.15.5:32988) 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 -- 172.21.15.5:0/2361799201 learned_addr learned my addr 172.21.15.5:0/2361799201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24107df0 0x7fcc2419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc22d9d700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc24108740 0x7fcc2419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 -- 172.21.15.5:0/2361799201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc24108740 msgr2=0x7fcc2419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc24108740 0x7fcc2419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 -- 172.21.15.5:0/2361799201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24107df0 msgr2=0x7fcc2419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24107df0 0x7fcc2419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 -- 172.21.15.5:0/2361799201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2419fd80 con 0x7fcc241034c0 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc22d9d700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc24108740 0x7fcc2419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24107df0 0x7fcc2419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.124+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 0x7fcc2419aeb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9c0 tx=0x7fcc1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.125+0000 7fcc137fe700 1 -- 172.21.15.5:0/2361799201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400cbd0 con 0x7fcc241034c0 2024-05-14T15:28:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.125+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc241a0070 con 0x7fcc241034c0 2024-05-14T15:28:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.125+0000 7fcc137fe700 1 -- 172.21.15.5:0/2361799201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1400ee60 con 0x7fcc241034c0 2024-05-14T15:28:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.125+0000 7fcc137fe700 1 -- 172.21.15.5:0/2361799201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc140055c0 con 0x7fcc241034c0 2024-05-14T15:28:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.125+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc241a7990 con 0x7fcc241034c0 2024-05-14T15:28:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.128+0000 7fcc137fe700 1 -- 172.21.15.5:0/2361799201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcc14021440 con 0x7fcc241034c0 2024-05-14T15:28:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.128+0000 7fcc137fe700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc0c0b29f0 0x7fcc0c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc137fe700 1 -- 172.21.15.5:0/2361799201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fcc140e2ca0 con 0x7fcc241034c0 2024-05-14T15:28:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc0c0b29f0 0x7fcc0c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 Processor -- start 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 -- start start 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc0c0b29f0 0x7fcc0c0b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc18000c00 tx=0x7fcc1801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 0x555984c61950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c61e90 0x555984c622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555984b839c0 con 0x555984ae3ec0 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555984ae8e20 con 0x555984c61e90 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.129+0000 7fcc37730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555984b7f980 con 0x555984c67e00 2024-05-14T15:28:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 0x555984c61950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 0x555984c61950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45518/0 (socket says 172.21.15.5:45518) 2024-05-14T15:28:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc22d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c61e90 0x555984c622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 -- 172.21.15.5:0/1354989247 learned_addr learned my addr 172.21.15.5:0/1354989247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc2259c700 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291140 12.59 MB/sec execute 7 sec latency 17.587 ms 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 -- 172.21.15.5:0/1354989247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c61e90 msgr2=0x555984c622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c61e90 0x555984c622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 -- 172.21.15.5:0/1354989247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 msgr2=0x555984c678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 -- 172.21.15.5:0/1354989247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555984ad0950 con 0x555984c67e00 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 0x555984c61950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcc180099c0 tx=0x7fcc18022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85d119922e896f02 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.130+0000 7fcc10ff9700 1 -- 172.21.15.5:0/1354989247 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180037c0 con 0x555984c67e00 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc10ff9700 1 -- 172.21.15.5:0/1354989247 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc18003e10 con 0x555984c67e00 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc10ff9700 1 -- 172.21.15.5:0/1354989247 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18026ba0 con 0x555984c67e00 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc37730b80 1 -- 172.21.15.5:0/1354989247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 msgr2=0x555984c61950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc37730b80 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 0x555984c61950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcc180099c0 tx=0x7fcc18022000 comp rx=0 tx=0).stop 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc37730b80 1 -- 172.21.15.5:0/1354989247 shutdown_connections 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc37730b80 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c61e90 0x555984c622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.131+0000 7fcc37730b80 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 --2- 172.21.15.5:0/1354989247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c67e00 0x555984c61950 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- 172.21.15.5:0/1354989247 >> 172.21.15.5:0/1354989247 conn(0x555984b2b6b0 msgr2=0x555984c5ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- 172.21.15.5:0/1354989247 shutdown_connections 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- 172.21.15.5:0/1354989247 wait complete. 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- start start 2024-05-14T15:28:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 Processor -- start 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984ae3ec0 0x555984c6b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 0x555984c6ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555984c6e840 con 0x555984c61e90 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555984b3cf50 con 0x555984c67e00 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.132+0000 7fcc37730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555984c6e9c0 con 0x555984ae3ec0 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc37730b80 10 client.0 _ll_register_callbacks cb 0x555984b1b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 0x555984c6ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 0x555984c6ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33008/0 (socket says 172.21.15.5:33008) 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 -- 172.21.15.5:0/2611685073 learned_addr learned my addr 172.21.15.5:0/2611685073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984ae3ec0 0x555984c6b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc22d9d700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 -- 172.21.15.5:0/2611685073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984ae3ec0 msgr2=0x555984c6b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984ae3ec0 0x555984c6b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 -- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 msgr2=0x555984c6bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c6bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 -- 172.21.15.5:0/2611685073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555984b3a7e0 con 0x555984c61e90 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984ae3ec0 0x555984c6b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 0x555984c6ba60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcc180098e0 tx=0x7fcc18003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf333e827de7009 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2611685073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18038070 con 0x555984c61e90 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2611685073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fcc18003bd0 con 0x555984c61e90 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.133+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc18003e10 con 0x555984c61e90 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2611685073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc18044050 con 0x555984c61e90 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 10 client.16350 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555984c6e840 con 0x555984c61e90 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2611685073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc18003e10 con 0x555984c61e90 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2611685073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcc18030b10 con 0x555984c61e90 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 10 client.16350 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 20 client.16350 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984c7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 4 client.16350 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.134+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555984b913a0 con 0x555984c7b750 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.135+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984c7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.135+0000 7fcc09ffb700 10 client.16350 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.135+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984c7db70 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c155f93ae1665136 server_cookie=ff31661d0b8c74ee in_seq=0 out_seq=0 2024-05-14T15:28:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2611685073 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555984b913a0 con 0x555984c7b750 2024-05-14T15:28:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc09ffb700 10 client.16350 handle_command_reply: tid=0 2024-05-14T15:28:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 client.16350 shutdown 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 msgr2=0x555984c6ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 0x555984c6ba60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcc180098e0 tx=0x7fcc18003680 comp rx=0 tx=0).stop 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 shutdown_connections 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984c7db70 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc09ffb700 0 client.16350 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984ae3ec0 0x555984c6b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2611685073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c61e90 0x555984c6ba60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 >> 172.21.15.5:0/2611685073 conn(0x555984b2b6b0 msgr2=0x555984c60940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 shutdown_connections 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 1 -- 172.21.15.5:0/2611685073 wait complete. 2024-05-14T15:28:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.139+0000 7fcc37730b80 20 client.16350 trim_cache size 0 max 16384 2024-05-14T15:28:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 Processor -- start 2024-05-14T15:28:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 -- start start 2024-05-14T15:28:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984ae3ec0 0x555984c61950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c77c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 0x555984c6bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241b2ce0 con 0x555984c67e00 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241b0570 con 0x555984ae3ec0 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc37730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241ae3c0 con 0x555984c61e90 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc22d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 0x555984c6bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc22d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 0x555984c6bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33022/0 (socket says 172.21.15.5:33022) 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc21d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c77c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc22d9d700 1 -- 172.21.15.5:0/3698061430 learned_addr learned my addr 172.21.15.5:0/3698061430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.163+0000 7fcc2259c700 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984ae3ec0 0x555984c61950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc21d9b700 1 -- 172.21.15.5:0/3698061430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984ae3ec0 msgr2=0x555984c61950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984ae3ec0 0x555984c61950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc21d9b700 1 -- 172.21.15.5:0/3698061430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 msgr2=0x555984c6bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 0x555984c6bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc21d9b700 1 -- 172.21.15.5:0/3698061430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555984c69a90 con 0x555984c61e90 2024-05-14T15:28:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc22d9d700 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 0x555984c6bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c77c90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800a010 tx=0x7fcc180036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5583b7371f000ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc0bfff700 1 -- 172.21.15.5:0/3698061430 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18038030 con 0x555984c61e90 2024-05-14T15:28:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc0bfff700 1 -- 172.21.15.5:0/3698061430 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc180039d0 con 0x555984c61e90 2024-05-14T15:28:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.164+0000 7fcc0bfff700 1 -- 172.21.15.5:0/3698061430 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800fdd0 con 0x555984c61e90 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 -- 172.21.15.5:0/3698061430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 msgr2=0x555984c77c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c77c90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800a010 tx=0x7fcc180036f0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 -- 172.21.15.5:0/3698061430 shutdown_connections 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984c67e00 0x555984c6bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984ae3ec0 0x555984c61950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 --2- 172.21.15.5:0/3698061430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c77c90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 -- 172.21.15.5:0/3698061430 >> 172.21.15.5:0/3698061430 conn(0x555984b2b6b0 msgr2=0x555984c69ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 -- 172.21.15.5:0/3698061430 shutdown_connections 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 -- 172.21.15.5:0/3698061430 wait complete. 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.165+0000 7fcc37730b80 1 -- start start 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 Processor -- start 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c7f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c7fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241aea30 con 0x555984ae3ec0 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241afea0 con 0x555984c67e00 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc241af100 con 0x555984c61e90 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc37730b80 10 client.0 _ll_register_callbacks cb 0x555984b1b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c81140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33026/0 (socket says 172.21.15.5:33026) 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 -- 172.21.15.5:0/2673426984 learned_addr learned my addr 172.21.15.5:0/2673426984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc22d9d700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c7fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c7f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 -- 172.21.15.5:0/2673426984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 msgr2=0x555984c7f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c7f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 -- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 msgr2=0x555984c7fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c7fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc2259c700 1 -- 172.21.15.5:0/2673426984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555984c80260 con 0x555984ae3ec0 2024-05-14T15:28:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.166+0000 7fcc22d9d700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c7fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc2259c700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c81140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc140e0630 tx=0x7fcc1400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de782981cf6042 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2673426984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1406f060 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2673426984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fcc140a9e20 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc180039d0 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2673426984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc140c4040 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 10 client.16356 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555984ce6ca0 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2673426984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc180039d0 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2673426984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcc140b2040 con 0x555984ae3ec0 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 20 client.16356 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984ce9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 4 client.16356 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.167+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555984b8b440 con 0x555984c7b750 2024-05-14T15:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.168+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984ce9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.168+0000 7fcc09ffb700 10 client.16356 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.168+0000 7fcc21d9b700 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984ce9320 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a93601cb8c90cb5 server_cookie=a557557c7c002247 in_seq=0 out_seq=0 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.169+0000 7fcc09ffb700 1 -- 172.21.15.5:0/2673426984 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fcc18041080 con 0x555984c7b750 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.169+0000 7fcc09ffb700 10 client.16356 handle_command_reply: tid=0 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.169+0000 7fcc37730b80 1 client.16356 shutdown 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.169+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 msgr2=0x555984c81140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.169+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c81140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc140e0630 tx=0x7fcc1400e940 comp rx=0 tx=0).stop 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 shutdown_connections 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555984c7b750 0x555984ce9320 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc09ffb700 0 client.16356 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555984c67e00 0x555984c7fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555984ae3ec0 0x555984c81140 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 --2- 172.21.15.5:0/2673426984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555984c61e90 0x555984c7f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 >> 172.21.15.5:0/2673426984 conn(0x555984b2b6b0 msgr2=0x555984c6bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 shutdown_connections 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 1 -- 172.21.15.5:0/2673426984 wait complete. 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.170+0000 7fcc37730b80 20 client.16356 trim_cache size 0 max 16384 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5f9b309b-6daa-44bd-94d8-7f04223ef669", 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:28:46.175 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:28:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc0c0b29f0 msgr2=0x7fcc0c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc0c0b29f0 0x7fcc0c0b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcc18000c00 tx=0x7fcc1801a040 comp rx=0 tx=0).stop 2024-05-14T15:28:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 msgr2=0x7fcc2419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 0x7fcc2419aeb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9c0 tx=0x7fcc1400ecd0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 shutdown_connections 2024-05-14T15:28:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc0c0b29f0 0x7fcc0c0b4f00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc24108740 0x7fcc2419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc241034c0 0x7fcc2419aeb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 --2- 172.21.15.5:0/2361799201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc24107df0 0x7fcc2419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 >> 172.21.15.5:0/2361799201 conn(0x7fcc240fedc0 msgr2=0x7fcc2406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 shutdown_connections 2024-05-14T15:28:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:46.171+0000 7fcc28d1c700 1 -- 172.21.15.5:0/2361799201 wait complete. 2024-05-14T15:28:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:45 smithi144 ceph-mon[35650]: pgmap v122: 129 pgs: 129 active+clean; 45 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-05-14T15:28:46.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:45 smithi193 ceph-mon[36001]: pgmap v122: 129 pgs: 129 active+clean; 45 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 316 op/s 2024-05-14T15:28:46.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:28:47.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 293433 12.05 MB/sec execute 8 sec latency 13.539 ms 2024-05-14T15:28:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:28:47.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:46 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:28:47.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:46 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:28:47.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:46 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:28:47.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:46 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:28:47.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:46 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:28:47.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:46 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:28:47.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:46 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:28:47.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:46 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:28:47.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:46 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:28:47.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:46 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:28:47.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:46 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:28:47.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:46 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:28:48.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 295348 12.04 MB/sec execute 9 sec latency 17.311 ms 2024-05-14T15:28:48.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:47 smithi144 ceph-mon[35650]: pgmap v123: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-05-14T15:28:48.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:47 smithi193 ceph-mon[36001]: pgmap v123: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-05-14T15:28:48.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:47 smithi005 ceph-mon[31559]: pgmap v123: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.1 MiB/s wr, 446 op/s 2024-05-14T15:28:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.021+0000 7f552ee0e700 1 -- 172.21.15.5:0/2224836982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 msgr2=0x7f55281049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.021+0000 7f552ee0e700 1 --2- 172.21.15.5:0/2224836982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f55281049f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f55180099c0 tx=0x7f5518009cd0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 -- 172.21.15.5:0/2224836982 shutdown_connections 2024-05-14T15:28:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 --2- 172.21.15.5:0/2224836982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5528109860 0x7f55281111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 --2- 172.21.15.5:0/2224836982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f55281049f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 --2- 172.21.15.5:0/2224836982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528108f10 0x7f5528109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 -- 172.21.15.5:0/2224836982 >> 172.21.15.5:0/2224836982 conn(0x7f55280ffed0 msgr2=0x7f5528102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 -- 172.21.15.5:0/2224836982 shutdown_connections 2024-05-14T15:28:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.022+0000 7f552ee0e700 1 -- 172.21.15.5:0/2224836982 wait complete. 2024-05-14T15:28:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 Processor -- start 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 -- start start 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f552819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5528108f10 0x7f552819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528109860 0x7f552819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114490 con 0x7f55281045e0 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114310 con 0x7f5528108f10 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552ee0e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114610 con 0x7f5528109860 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f552819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f552819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33054/0 (socket says 172.21.15.5:33054) 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 -- 172.21.15.5:0/79587536 learned_addr learned my addr 172.21.15.5:0/79587536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f551ffff700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5528108f10 0x7f552819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552d3ab700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528109860 0x7f552819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 -- 172.21.15.5:0/79587536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528109860 msgr2=0x7f552819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528109860 0x7f552819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 -- 172.21.15.5:0/79587536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5528108f10 msgr2=0x7f552819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5528108f10 0x7f552819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.023+0000 7f552cbaa700 1 -- 172.21.15.5:0/79587536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552819fcd0 con 0x7f55281045e0 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.024+0000 7f552cbaa700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f552819ae00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5518006010 tx=0x7f55180055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.024+0000 7f551dffb700 1 -- 172.21.15.5:0/79587536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518026070 con 0x7f55281045e0 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.024+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552819ff60 con 0x7f55281045e0 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.024+0000 7f551dffb700 1 -- 172.21.15.5:0/79587536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f551800f460 con 0x7f55281045e0 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.024+0000 7f551dffb700 1 -- 172.21.15.5:0/79587536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518009e60 con 0x7f55281045e0 2024-05-14T15:28:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.024+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55281a03f0 con 0x7f55281045e0 2024-05-14T15:28:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.027+0000 7f551dffb700 1 -- 172.21.15.5:0/79587536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5518029070 con 0x7f55281045e0 2024-05-14T15:28:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.027+0000 7f551dffb700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55100b29f0 0x7f55100b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.028+0000 7f551ffff700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55100b29f0 0x7f55100b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.028+0000 7f551dffb700 1 -- 172.21.15.5:0/79587536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f55180e31a0 con 0x7f55281045e0 2024-05-14T15:28:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.028+0000 7f551ffff700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55100b29f0 0x7f55100b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55281076c0 tx=0x7f552400b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.028+0000 7f553d822b80 1 Processor -- start 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.028+0000 7f553d822b80 1 -- start start 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f553d822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa89ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f553d822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f553d822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa84220 0x561c6fa84630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f553d822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c6f9a1940 con 0x561c6fa8a0e0 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f553d822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c6f906da0 con 0x561c6f901e40 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f553d822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c6f99d900 con 0x561c6fa84220 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa83ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33060/0 (socket says 172.21.15.5:33060) 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 -- 172.21.15.5:0/1979295563 learned_addr learned my addr 172.21.15.5:0/1979295563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f552cbaa700 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa89ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f552d3ab700 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa84220 0x561c6fa84630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 -- 172.21.15.5:0/1979295563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa84220 msgr2=0x561c6fa84630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa84220 0x561c6fa84630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 -- 172.21.15.5:0/1979295563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 msgr2=0x561c6fa89ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa89ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f551ffff700 1 -- 172.21.15.5:0/1979295563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6f8ee8d0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.029+0000 7f552cbaa700 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa89ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.030+0000 7f551ffff700 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa83ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5524010f40 tx=0x7f5524013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d157d17473949066 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.030+0000 7f550effd700 1 -- 172.21.15.5:0/1979295563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400be60 con 0x561c6fa8a0e0 2024-05-14T15:28:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.030+0000 7f550effd700 1 -- 172.21.15.5:0/1979295563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5524011db0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.030+0000 7f550effd700 1 -- 172.21.15.5:0/1979295563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552401c6a0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 -- 172.21.15.5:0/1979295563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 msgr2=0x561c6fa83ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa83ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5524010f40 tx=0x7f5524013000 comp rx=0 tx=0).stop 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 -- 172.21.15.5:0/1979295563 shutdown_connections 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa84220 0x561c6fa84630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa89ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 --2- 172.21.15.5:0/1979295563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa83ce0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 -- 172.21.15.5:0/1979295563 >> 172.21.15.5:0/1979295563 conn(0x561c6f949630 msgr2=0x561c6fa7aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 -- 172.21.15.5:0/1979295563 shutdown_connections 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 -- 172.21.15.5:0/1979295563 wait complete. 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.031+0000 7f553d822b80 1 -- start start 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 Processor -- start 2024-05-14T15:28:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa8d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa84220 0x561c6fa8de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c6fa90be0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c6f95aed0 con 0x561c6fa84220 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c6fa90d60 con 0x561c6f901e40 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f553d822b80 10 client.0 _ll_register_callbacks cb 0x561c6f939950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f552d3ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f552d3ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa8e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33070/0 (socket says 172.21.15.5:33070) 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f552cbaa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa8d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f552d3ab700 1 -- 172.21.15.5:0/2766294259 learned_addr learned my addr 172.21.15.5:0/2766294259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.032+0000 7f551ffff700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa84220 0x561c6fa8de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552d3ab700 1 -- 172.21.15.5:0/2766294259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 msgr2=0x561c6fa8d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552d3ab700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa8d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552d3ab700 1 -- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa84220 msgr2=0x561c6fa8de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552d3ab700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa84220 0x561c6fa8de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552d3ab700 1 -- 172.21.15.5:0/2766294259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6f958760 con 0x561c6fa8a0e0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552cbaa700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa8d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f552d3ab700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa8e3d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5520011f60 tx=0x7f5520014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bbf2f273a80546 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f5507fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f5507fff700 1 -- 172.21.15.5:0/2766294259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520014ba0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5524011db0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f5507fff700 1 -- 172.21.15.5:0/2766294259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f5520014d40 con 0x561c6fa8a0e0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f5507fff700 1 -- 172.21.15.5:0/2766294259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5520027050 con 0x561c6fa8a0e0 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f553d822b80 10 client.16368 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:49.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.033+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561c6fa9b870 con 0x561c6fa8a0e0 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f5507fff700 1 -- 172.21.15.5:0/2766294259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5524011db0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f5507fff700 1 -- 172.21.15.5:0/2766294259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5520004730 con 0x561c6fa8a0e0 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f553d822b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f553d822b80 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f553d822b80 20 client.16368 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f553d822b80 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fa9de00 0x561c6faa0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f553d822b80 4 client.16368 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c6f9af320 con 0x561c6fa9de00 2024-05-14T15:28:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.034+0000 7f552cbaa700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fa9de00 0x561c6faa0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.035+0000 7f5507fff700 10 client.16368 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.035+0000 7f552cbaa700 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fa9de00 0x561c6faa0220 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb68b76b22248989 server_cookie=2044122ea8fbb156 in_seq=0 out_seq=0 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.035+0000 7f5507fff700 1 -- 172.21.15.5:0/2766294259 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c6f9af320 con 0x561c6fa9de00 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.035+0000 7f5507fff700 10 client.16368 handle_command_reply: tid=0 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.035+0000 7f553d822b80 1 client.16368 shutdown 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 msgr2=0x561c6fa8e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa8e3d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5520011f60 tx=0x7f5520014600 comp rx=0 tx=0).stop 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 shutdown_connections 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fa9de00 0x561c6faa0220 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f5507fff700 0 client.16368 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa8a0e0 0x561c6fa8e3d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa8d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 --2- 172.21.15.5:0/2766294259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa84220 0x561c6fa8de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 >> 172.21.15.5:0/2766294259 conn(0x561c6f949630 msgr2=0x561c6fa84630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 shutdown_connections 2024-05-14T15:28:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 1 -- 172.21.15.5:0/2766294259 wait complete. 2024-05-14T15:28:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.036+0000 7f553d822b80 20 client.16368 trim_cache size 0 max 16384 2024-05-14T15:28:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.057+0000 7f553d822b80 1 Processor -- start 2024-05-14T15:28:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.057+0000 7f553d822b80 1 -- start start 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f553d822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f553d822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f553d822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa8a0e0 0x561c6fa99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f553d822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55281b0bd0 con 0x561c6fa84220 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f553d822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55281ae460 con 0x561c6f901e40 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f553d822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55281ac2b0 con 0x561c6fa8a0e0 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa87d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33086/0 (socket says 172.21.15.5:33086) 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 -- 172.21.15.5:0/1221335033 learned_addr learned my addr 172.21.15.5:0/1221335033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f552cbaa700 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa8e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f552d3ab700 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa8a0e0 0x561c6fa99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 -- 172.21.15.5:0/1221335033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa8a0e0 msgr2=0x561c6fa99c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa8a0e0 0x561c6fa99c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 -- 172.21.15.5:0/1221335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 msgr2=0x561c6fa8e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa8e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 -- 172.21.15.5:0/1221335033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6fa7b8b0 con 0x561c6fa84220 2024-05-14T15:28:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.058+0000 7f551ffff700 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa87d40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5524010f10 tx=0x7f552400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efa5c0d917a277a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.059+0000 7f550e7fc700 1 -- 172.21.15.5:0/1221335033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524036070 con 0x561c6fa84220 2024-05-14T15:28:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.059+0000 7f550e7fc700 1 -- 172.21.15.5:0/1221335033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55240054f0 con 0x561c6fa84220 2024-05-14T15:28:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.059+0000 7f550e7fc700 1 -- 172.21.15.5:0/1221335033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552401e420 con 0x561c6fa84220 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 -- 172.21.15.5:0/1221335033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 msgr2=0x561c6fa87d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa87d40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5524010f10 tx=0x7f552400bf40 comp rx=0 tx=0).stop 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 -- 172.21.15.5:0/1221335033 shutdown_connections 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6fa8a0e0 0x561c6fa99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f901e40 0x561c6fa8e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 --2- 172.21.15.5:0/1221335033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa87d40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 -- 172.21.15.5:0/1221335033 >> 172.21.15.5:0/1221335033 conn(0x561c6f949630 msgr2=0x561c6fa87630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 -- 172.21.15.5:0/1221335033 shutdown_connections 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 -- 172.21.15.5:0/1221335033 wait complete. 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 -- start start 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.060+0000 7f553d822b80 1 Processor -- start 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa91a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa91fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa8a0e0 0x561c6fa92510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55281ac920 con 0x561c6fa84220 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55281add90 con 0x561c6fa8a0e0 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55281acff0 con 0x561c6f901e40 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f553d822b80 10 client.0 _ll_register_callbacks cb 0x561c6f939950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa91fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa91fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33094/0 (socket says 172.21.15.5:33094) 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 -- 172.21.15.5:0/1206871463 learned_addr learned my addr 172.21.15.5:0/1206871463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f552d3ab700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa8a0e0 0x561c6fa92510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f552cbaa700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa91a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 -- 172.21.15.5:0/1206871463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 msgr2=0x561c6fa91a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa91a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 -- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa8a0e0 msgr2=0x561c6fa92510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa8a0e0 0x561c6fa92510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 -- 172.21.15.5:0/1206871463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6fb07570 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f552d3ab700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa8a0e0 0x561c6fa92510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f551ffff700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa91fd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55240047f0 tx=0x7f552400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a3fd09f5e6af8c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.061+0000 7f5507fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f5507fff700 1 -- 172.21.15.5:0/1206871463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524036040 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55240054f0 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f5507fff700 1 -- 172.21.15.5:0/1206871463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f5524004d60 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f5507fff700 1 -- 172.21.15.5:0/1206871463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5524004f60 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 10 client.16374 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561c6fb086a0 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f5507fff700 1 -- 172.21.15.5:0/1206871463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55240054f0 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f5507fff700 1 -- 172.21.15.5:0/1206871463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5524039070 con 0x561c6fa84220 2024-05-14T15:28:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 20 client.16374 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fb08aa0 0x561c6fb0aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 4 client.16374 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.062+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c6f9a93c0 con 0x561c6fb08aa0 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f552cbaa700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fb08aa0 0x561c6fb0aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f5507fff700 10 client.16374 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f552cbaa700 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fb08aa0 0x561c6fb0aec0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a433a046e1f31d7f server_cookie=f5d96291ce4b0285 in_seq=0 out_seq=0 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f5507fff700 1 -- 172.21.15.5:0/1206871463 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561c6f9a93c0 con 0x561c6fb08aa0 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f5507fff700 10 client.16374 handle_command_reply: tid=0 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f553d822b80 1 client.16374 shutdown 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 msgr2=0x561c6fa91fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.063+0000 7f553d822b80 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa91fd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55240047f0 tx=0x7f552400be20 comp rx=0 tx=0).stop 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 shutdown_connections 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561c6fb08aa0 0x561c6fb0aec0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6fa8a0e0 0x561c6fa92510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c6f901e40 0x561c6fa91a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 --2- 172.21.15.5:0/1206871463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c6fa84220 0x561c6fa91fd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 >> 172.21.15.5:0/1206871463 conn(0x561c6f949630 msgr2=0x561c6fa7aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f5507fff700 0 client.16374 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 shutdown_connections 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 1 -- 172.21.15.5:0/1206871463 wait complete. 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.064+0000 7f553d822b80 20 client.16374 trim_cache size 0 max 16384 2024-05-14T15:28:49.070 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: "5f9b309b-6daa-44bd-94d8-7f04223ef669": { 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5f9b309b-6daa-44bd-94d8-7f04223ef669", 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:28:49.071 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55100b29f0 msgr2=0x7f55100b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55100b29f0 0x7f55100b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55281076c0 tx=0x7f552400b410 comp rx=0 tx=0).stop 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 msgr2=0x7f552819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f552819ae00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5518006010 tx=0x7f55180055f0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 shutdown_connections 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f55100b29f0 0x7f55100b4f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528109860 0x7f552819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55281045e0 0x7f552819ae00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 --2- 172.21.15.5:0/79587536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5528108f10 0x7f552819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 >> 172.21.15.5:0/79587536 conn(0x7f55280ffed0 msgr2=0x7f55281081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 shutdown_connections 2024-05-14T15:28:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:28:49.065+0000 7f552ee0e700 1 -- 172.21.15.5:0/79587536 wait complete. 2024-05-14T15:28:49.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 298260 12.64 MB/sec execute 10 sec latency 18.158 ms 2024-05-14T15:28:49.785 INFO:tasks.cephfs.filesystem:scrub status for tag:5f9b309b-6daa-44bd-94d8-7f04223ef669 - {'path': '/', 'tag': '5f9b309b-6daa-44bd-94d8-7f04223ef669', 'options': 'recursive,force'} 2024-05-14T15:28:50.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:49 smithi005 ceph-mon[31559]: pgmap v124: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 366 op/s 2024-05-14T15:28:50.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 299890 12.02 MB/sec execute 11 sec latency 72.833 ms 2024-05-14T15:28:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:49 smithi144 ceph-mon[35650]: pgmap v124: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 366 op/s 2024-05-14T15:28:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:49 smithi193 ceph-mon[36001]: pgmap v124: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 366 op/s 2024-05-14T15:28:51.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 302104 12.15 MB/sec execute 12 sec latency 15.899 ms 2024-05-14T15:28:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:50 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:28:51.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:50 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:28:51.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:50 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:28:52.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 304891 12.33 MB/sec execute 13 sec latency 44.942 ms 2024-05-14T15:28:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:51 smithi144 ceph-mon[35650]: pgmap v125: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 366 op/s 2024-05-14T15:28:52.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:51 smithi193 ceph-mon[36001]: pgmap v125: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 366 op/s 2024-05-14T15:28:52.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:51 smithi005 ceph-mon[31559]: pgmap v125: 129 pgs: 129 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 366 op/s 2024-05-14T15:28:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1430411142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01032e0 msgr2=0x7f3ad010f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1430411142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01032e0 0x7f3ad010f6b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009ca0 tx=0x7f3ac802a8c0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1430411142 shutdown_connections 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1430411142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0109030 0x7f3ad0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1430411142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01032e0 0x7f3ad010f6b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1430411142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad0108b90 0x7f3ad0102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1430411142 >> 172.21.15.5:0/1430411142 conn(0x7f3ad00fe390 msgr2=0x7f3ad01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1430411142 shutdown_connections 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.569+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1430411142 wait complete. 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 Processor -- start 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 -- start start 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad01032e0 0x7f3ad019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 0x7f3ad019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0109030 0x7f3ad019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112da0 con 0x7f3ad0108b90 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112c20 con 0x7f3ad01032e0 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad6d76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112f20 con 0x7f3ad0109030 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 0x7f3ad019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad4b12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad01032e0 0x7f3ad019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 0x7f3ad019bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33112/0 (socket says 172.21.15.5:33112) 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 -- 172.21.15.5:0/1048392056 learned_addr learned my addr 172.21.15.5:0/1048392056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad01032e0 0x7f3ad019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44090/0 (socket says 172.21.15.5:44090) 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3ad5313640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0109030 0x7f3ad019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 -- 172.21.15.5:0/1048392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0109030 msgr2=0x7f3ad019c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0109030 0x7f3ad019c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 -- 172.21.15.5:0/1048392056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad01032e0 msgr2=0x7f3ad019b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad01032e0 0x7f3ad019b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.570+0000 7f3acffff640 1 -- 172.21.15.5:0/1048392056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad01a6a30 con 0x7f3ad0108b90 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.571+0000 7f3acffff640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 0x7f3ad019bc60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005d00 tx=0x7f3abc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.571+0000 7f3acdffb640 1 -- 172.21.15.5:0/1048392056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc017070 con 0x7f3ad0108b90 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.571+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a6d20 con 0x7f3ad0108b90 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.571+0000 7f3acdffb640 1 -- 172.21.15.5:0/1048392056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3abc009c30 con 0x7f3ad0108b90 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.571+0000 7f3acdffb640 1 -- 172.21.15.5:0/1048392056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00f620 con 0x7f3ad0108b90 2024-05-14T15:28:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.571+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a7210 con 0x7f3ad0108b90 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 Processor -- start 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 -- start start 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa05a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa05fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa06510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d82f96b5d0 con 0x55d82f6444d0 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d82f924890 con 0x55d82f63c2f0 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3addc2e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d82f967590 con 0x55d82f6651b0 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3ad4b12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa05fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3ad5313640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa05a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3ad5313640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa05a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45626/0 (socket says 172.21.15.5:45626) 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3ad4b12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa05fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44102/0 (socket says 172.21.15.5:44102) 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.574+0000 7f3ad5313640 1 -- 172.21.15.5:0/2099551673 learned_addr learned my addr 172.21.15.5:0/2099551673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3ad5313640 1 -- 172.21.15.5:0/2099551673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 msgr2=0x55d82fa05fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3ad5313640 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa05fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3ad5313640 1 -- 172.21.15.5:0/2099551673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 msgr2=0x55d82fa06510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3ad5313640 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa06510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3ad5313640 1 -- 172.21.15.5:0/2099551673 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82f67d210 con 0x55d82f6651b0 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3acdffb640 1 -- 172.21.15.5:0/1048392056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3abc01a070 con 0x7f3ad0108b90 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3ad5313640 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa05a90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8035000 tx=0x7f3ac802f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40762f3e1c57aaa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3abaffd640 1 -- 172.21.15.5:0/2099551673 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8031cf0 con 0x55d82f6651b0 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3abaffd640 1 -- 172.21.15.5:0/2099551673 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac8036030 con 0x55d82f6651b0 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.575+0000 7f3abaffd640 1 -- 172.21.15.5:0/2099551673 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80042a0 con 0x55d82f6651b0 2024-05-14T15:28:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3acdffb640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3ac00b2950 0x7f3ac00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3ac00b2950 0x7f3ac00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3acdffb640 1 -- 172.21.15.5:0/1048392056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f3abc0e2330 con 0x7f3ad0108b90 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 -- 172.21.15.5:0/2099551673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 msgr2=0x55d82fa05a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa05a90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8035000 tx=0x7f3ac802f4c0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 -- 172.21.15.5:0/2099551673 shutdown_connections 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa06510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa05fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 --2- 172.21.15.5:0/2099551673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa05a90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 -- 172.21.15.5:0/2099551673 >> 172.21.15.5:0/2099551673 conn(0x55d82f659300 msgr2=0x55d82fa02340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 -- 172.21.15.5:0/2099551673 shutdown_connections 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3addc2e740 1 -- 172.21.15.5:0/2099551673 wait complete. 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.576+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3ac00b2950 0x7f3ac00b4e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55d82fa03440 tx=0x7f3ac4005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 -- start start 2024-05-14T15:28:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 Processor -- start 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa14b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa1a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d82fa16f40 con 0x55d82f6444d0 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d82f928370 con 0x55d82f63c2f0 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.577+0000 7f3addc2e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d82fa170c0 con 0x55d82f6651b0 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3addc2e740 10 client.0 _ll_register_callbacks cb 0x55d82f67ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa14b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa14b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33126/0 (socket says 172.21.15.5:33126) 2024-05-14T15:28:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 -- 172.21.15.5:0/3515000198 learned_addr learned my addr 172.21.15.5:0/3515000198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3ad5313640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa1a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 -- 172.21.15.5:0/3515000198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 msgr2=0x55d82fa1a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa1a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 -- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 msgr2=0x55d82fa14630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa14630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 -- 172.21.15.5:0/3515000198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82f749620 con 0x55d82f6444d0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3acffff640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa14b70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0a5380 tx=0x7f3abc0a8d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa1a53b8a47e4ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3aaffff640 1 -- 172.21.15.5:0/3515000198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc021070 con 0x55d82f6444d0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac8036030 con 0x55d82f6444d0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.578+0000 7f3aaffff640 1 -- 172.21.15.5:0/3515000198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f3abc0ad120 con 0x55d82f6444d0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3aaffff640 1 -- 172.21.15.5:0/3515000198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3abc009de0 con 0x55d82f6444d0 2024-05-14T15:28:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 10 client.16383 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d82fa23370 con 0x55d82f6444d0 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3aaffff640 1 -- 172.21.15.5:0/3515000198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac8036030 con 0x55d82f6444d0 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3aaffff640 1 -- 172.21.15.5:0/3515000198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3abc0a8560 con 0x55d82f6444d0 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 10 client.16383 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 20 client.16383 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f623d80 0x55d82fa26d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 4 client.16383 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82f9791e0 con 0x55d82f623d80 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.579+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f623d80 0x55d82fa26d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.580+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f623d80 0x55d82fa26d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97944ecdb5349893 server_cookie=9a1964c789ee325 in_seq=0 out_seq=0 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.580+0000 7f3aaffff640 10 client.16383 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3aaffff640 1 -- 172.21.15.5:0/3515000198 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d82f9791e0 con 0x55d82f623d80 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3aaffff640 10 client.16383 handle_command_reply: tid=0 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 client.16383 shutdown 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 msgr2=0x55d82fa14b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa14b70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0a5380 tx=0x7f3abc0a8d10 comp rx=0 tx=0).stop 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 shutdown_connections 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f623d80 0x55d82fa26d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6651b0 0x55d82fa1a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa14b70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3515000198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f63c2f0 0x55d82fa14630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3aaffff640 0 client.16383 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 >> 172.21.15.5:0/3515000198 conn(0x55d82f659300 msgr2=0x55d82fa07890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 shutdown_connections 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 1 -- 172.21.15.5:0/3515000198 wait complete. 2024-05-14T15:28:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.581+0000 7f3addc2e740 20 client.16383 trim_cache size 0 max 16384 2024-05-14T15:28:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 Processor -- start 2024-05-14T15:28:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 -- start start 2024-05-14T15:28:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f63c2f0 0x55d82fa13400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa1a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01b19a0 con 0x55d82f6444d0 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01af270 con 0x55d82f6651b0 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3addc2e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ad0c0 con 0x55d82f63c2f0 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3acffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa1a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3acffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa1a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33130/0 (socket says 172.21.15.5:33130) 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3acffff640 1 -- 172.21.15.5:0/3991269359 learned_addr learned my addr 172.21.15.5:0/3991269359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.598+0000 7f3ad5313640 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3acffff640 1 -- 172.21.15.5:0/3991269359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f63c2f0 msgr2=0x55d82fa13400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3acffff640 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f63c2f0 0x55d82fa13400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3acffff640 1 -- 172.21.15.5:0/3991269359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 msgr2=0x55d82fa1a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3acffff640 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3acffff640 1 -- 172.21.15.5:0/3991269359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82f9fd9a0 con 0x55d82f6444d0 2024-05-14T15:28:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3ad5313640 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:28:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3acffff640 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa1a360 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0df770 tx=0x7f3abc010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a40760fd30d0fad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3aba7fc640 1 -- 172.21.15.5:0/3991269359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc021070 con 0x55d82f6444d0 2024-05-14T15:28:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3aba7fc640 1 -- 172.21.15.5:0/3991269359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3abc0ad600 con 0x55d82f6444d0 2024-05-14T15:28:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.599+0000 7f3aba7fc640 1 -- 172.21.15.5:0/3991269359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0a8650 con 0x55d82f6444d0 2024-05-14T15:28:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 -- 172.21.15.5:0/3991269359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 msgr2=0x55d82fa1a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa1a360 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0df770 tx=0x7f3abc010b70 comp rx=0 tx=0).stop 2024-05-14T15:28:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 -- 172.21.15.5:0/3991269359 shutdown_connections 2024-05-14T15:28:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f6444d0 0x55d82fa1a360 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3991269359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f63c2f0 0x55d82fa13400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 -- 172.21.15.5:0/3991269359 >> 172.21.15.5:0/3991269359 conn(0x55d82f659300 msgr2=0x55d82fa12cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 -- 172.21.15.5:0/3991269359 shutdown_connections 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 -- 172.21.15.5:0/3991269359 wait complete. 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 -- start start 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.600+0000 7f3addc2e740 1 Processor -- start 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 0x55d82fa20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6444d0 0x55d82fa1c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ad730 con 0x55d82f63c2f0 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01aeba0 con 0x55d82f6651b0 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ade00 con 0x55d82f6444d0 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3addc2e740 10 client.0 _ll_register_callbacks cb 0x7f3abc00df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:28:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3ad4b12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 0x55d82fa20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3ad4b12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 0x55d82fa20cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33144/0 (socket says 172.21.15.5:33144) 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3ad4b12640 1 -- 172.21.15.5:0/3522472136 learned_addr learned my addr 172.21.15.5:0/3522472136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3ad5313640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6444d0 0x55d82fa1c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 -- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 msgr2=0x55d82fa1c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 -- 172.21.15.5:0/3522472136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 msgr2=0x55d82fa20cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 0x55d82fa20cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 -- 172.21.15.5:0/3522472136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82fa1ca90 con 0x55d82f6444d0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 0x55d82fa20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3acffff640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6444d0 0x55d82fa1c010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0a9000 tx=0x7f3abc0a8d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8adc490c5d10dd18 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3aaffff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.601+0000 7f3aaffff640 1 -- 172.21.15.5:0/3522472136 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc021070 con 0x55d82f6444d0 2024-05-14T15:28:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3aaffff640 1 -- 172.21.15.5:0/3522472136 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f3abc0a58f0 con 0x55d82f6444d0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3abc0ad600 con 0x55d82f6444d0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3aaffff640 1 -- 172.21.15.5:0/3522472136 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3abc0a4d80 con 0x55d82f6444d0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 10 client.26002 fetch_fsmap learned FSMap version 30 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d82fa1ca90 con 0x55d82f6444d0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3aaffff640 1 -- 172.21.15.5:0/3522472136 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3abc0ad600 con 0x55d82f6444d0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3aaffff640 1 -- 172.21.15.5:0/3522472136 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3abc0ae790 con 0x55d82f6444d0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 10 client.26002 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 20 client.26002 populate_metadata read hostname 'smithi005' 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f5f91c0 0x55d82fa91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 4 client.26002 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d82f973050 con 0x55d82f5f91c0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.602+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f5f91c0 0x55d82fa91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:28:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.603+0000 7f3aaffff640 10 client.26002 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.603+0000 7f3ad4b12640 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f5f91c0 0x55d82fa91090 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1589f259d15508 server_cookie=9346260077bba041 in_seq=0 out_seq=0 2024-05-14T15:28:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3aaffff640 1 -- 172.21.15.5:0/3522472136 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d82f973050 con 0x55d82f5f91c0 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3aaffff640 10 client.26002 handle_command_reply: tid=0 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 client.26002 shutdown 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6444d0 msgr2=0x55d82fa1c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6444d0 0x55d82fa1c010 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0a9000 tx=0x7f3abc0a8d10 comp rx=0 tx=0).stop 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 shutdown_connections 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d82f5f91c0 0x55d82fa91090 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3aaffff640 0 client.26002 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82f6444d0 0x55d82fa1c010 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d82f6651b0 0x55d82fa1c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 --2- 172.21.15.5:0/3522472136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d82f63c2f0 0x55d82fa20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 >> 172.21.15.5:0/3522472136 conn(0x55d82f659300 msgr2=0x55d82fa16420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.915+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 shutdown_connections 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.916+0000 7f3addc2e740 1 -- 172.21.15.5:0/3522472136 wait complete. 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.916+0000 7f3addc2e740 20 client.26002 trim_cache size 0 max 16384 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:28:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:28:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3ac00b2950 msgr2=0x7f3ac00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3ac00b2950 0x7f3ac00b4e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55d82fa03440 tx=0x7f3ac4005f90 comp rx=0 tx=0).stop 2024-05-14T15:28:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 msgr2=0x7f3ad019bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 0x7f3ad019bc60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005d00 tx=0x7f3abc00dde0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 shutdown_connections 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3ac00b2950 0x7f3ac00b4e60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad0109030 0x7f3ad019c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0108b90 0x7f3ad019bc60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 --2- 172.21.15.5:0/1048392056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ad01032e0 0x7f3ad019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 >> 172.21.15.5:0/1048392056 conn(0x7f3ad00fe390 msgr2=0x7f3ad0105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 shutdown_connections 2024-05-14T15:28:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:28:52.917+0000 7f3ad6d76640 1 -- 172.21.15.5:0/1048392056 wait complete. 2024-05-14T15:28:53.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 306201 11.93 MB/sec execute 14 sec latency 118.593 ms 2024-05-14T15:28:54.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 308980 12.27 MB/sec execute 15 sec latency 23.234 ms 2024-05-14T15:28:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:53 smithi144 ceph-mon[35650]: pgmap v126: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.8 MiB/s wr, 490 op/s 2024-05-14T15:28:54.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:53 smithi193 ceph-mon[36001]: pgmap v126: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.8 MiB/s wr, 490 op/s 2024-05-14T15:28:54.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:53 smithi005 ceph-mon[31559]: pgmap v126: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.8 MiB/s wr, 490 op/s 2024-05-14T15:28:55.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 311021 11.94 MB/sec execute 16 sec latency 112.753 ms 2024-05-14T15:28:56.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312890 11.97 MB/sec execute 17 sec latency 15.790 ms 2024-05-14T15:28:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:55 smithi144 ceph-mon[35650]: pgmap v127: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-05-14T15:28:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:55 smithi193 ceph-mon[36001]: pgmap v127: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-05-14T15:28:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:55 smithi005 ceph-mon[31559]: pgmap v127: 129 pgs: 129 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-05-14T15:28:57.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 315452 12.09 MB/sec execute 18 sec latency 214.053 ms 2024-05-14T15:28:58.131 INFO:tasks.workunit.client.0.smithi005.stdout: 1 317281 11.97 MB/sec execute 19 sec latency 17.539 ms 2024-05-14T15:28:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:57 smithi144 ceph-mon[35650]: pgmap v128: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-05-14T15:28:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:57 smithi193 ceph-mon[36001]: pgmap v128: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-05-14T15:28:58.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:57 smithi005 ceph-mon[31559]: pgmap v128: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-05-14T15:28:59.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 319787 12.07 MB/sec execute 20 sec latency 8.664 ms 2024-05-14T15:29:00.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 322170 12.09 MB/sec execute 21 sec latency 38.598 ms 2024-05-14T15:29:00.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:28:59 smithi144 ceph-mon[35650]: pgmap v129: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-05-14T15:29:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:28:59 smithi193 ceph-mon[36001]: pgmap v129: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-05-14T15:29:00.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:28:59 smithi005 ceph-mon[31559]: pgmap v129: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-05-14T15:29:01.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 322583 11.63 MB/sec execute 22 sec latency 108.896 ms 2024-05-14T15:29:02.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 324057 11.42 MB/sec execute 23 sec latency 65.978 ms 2024-05-14T15:29:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:01 smithi144 ceph-mon[35650]: pgmap v130: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-05-14T15:29:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:01 smithi193 ceph-mon[36001]: pgmap v130: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-05-14T15:29:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:01 smithi005 ceph-mon[31559]: pgmap v130: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-05-14T15:29:03.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 326724 11.65 MB/sec execute 24 sec latency 16.886 ms 2024-05-14T15:29:04.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 329231 11.66 MB/sec execute 25 sec latency 12.757 ms 2024-05-14T15:29:04.135 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:03 smithi005 ceph-mon[31559]: pgmap v131: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-05-14T15:29:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:03 smithi144 ceph-mon[35650]: pgmap v131: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-05-14T15:29:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:03 smithi193 ceph-mon[36001]: pgmap v131: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-05-14T15:29:05.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 330691 11.51 MB/sec execute 26 sec latency 38.465 ms 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 -- 172.21.15.5:0/1235398441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 msgr2=0x7f4a60102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 --2- 172.21.15.5:0/1235398441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 0x7f4a60102ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009860 tx=0x7f4a5002a340 comp rx=0 tx=0).stop 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 -- 172.21.15.5:0/1235398441 shutdown_connections 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 --2- 172.21.15.5:0/1235398441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60103310 0x7f4a6010f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 --2- 172.21.15.5:0/1235398441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 0x7f4a60102ca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 --2- 172.21.15.5:0/1235398441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60108bc0 0x7f4a60102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 -- 172.21.15.5:0/1235398441 >> 172.21.15.5:0/1235398441 conn(0x7f4a600781b0 msgr2=0x7f4a60078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 -- 172.21.15.5:0/1235398441 shutdown_connections 2024-05-14T15:29:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.021+0000 7f4a67db7640 1 -- 172.21.15.5:0/1235398441 wait complete. 2024-05-14T15:29:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 Processor -- start 2024-05-14T15:29:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 -- start start 2024-05-14T15:29:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 0x7f4a600ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60108bc0 0x7f4a600ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 0x7f4a600ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60112db0 con 0x7f4a60103310 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60112c30 con 0x7f4a60109060 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a67db7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a60112f30 con 0x7f4a60108bc0 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a65b53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 0x7f4a600ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a65b53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 0x7f4a600ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35568/0 (socket says 172.21.15.5:35568) 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a65b53640 1 -- 172.21.15.5:0/3258623512 learned_addr learned my addr 172.21.15.5:0/3258623512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a66354640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 0x7f4a600ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a65352640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60108bc0 0x7f4a600ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.022+0000 7f4a65b53640 1 -- 172.21.15.5:0/3258623512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60108bc0 msgr2=0x7f4a600ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a65b53640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60108bc0 0x7f4a600ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a65b53640 1 -- 172.21.15.5:0/3258623512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 msgr2=0x7f4a600ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a65b53640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 0x7f4a600ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a65b53640 1 -- 172.21.15.5:0/3258623512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a601a8c20 con 0x7f4a60103310 2024-05-14T15:29:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a65b53640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 0x7f4a600ff3a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00e9a0 tx=0x7f4a5c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a56ffd640 1 -- 172.21.15.5:0/3258623512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00cd80 con 0x7f4a60103310 2024-05-14T15:29:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a56ffd640 1 -- 172.21.15.5:0/3258623512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a5c0035f0 con 0x7f4a60103310 2024-05-14T15:29:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a601a8f10 con 0x7f4a60103310 2024-05-14T15:29:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a56ffd640 1 -- 172.21.15.5:0/3258623512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c010690 con 0x7f4a60103310 2024-05-14T15:29:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.023+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a601a9370 con 0x7f4a60103310 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 Processor -- start 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 -- start start 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a56ffd640 1 -- 172.21.15.5:0/3258623512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4a5c004000 con 0x7f4a60103310 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x55896395cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x55896395d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896395d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5589638c25d0 con 0x5589635bc1b0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55896387b890 con 0x5589635932f0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589638be590 con 0x55896359b4d0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a66354640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x55896395cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a66354640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x55896395cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a66354640 1 -- 172.21.15.5:0/1798804889 learned_addr learned my addr 172.21.15.5:0/1798804889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a65b53640 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x55896395d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.026+0000 7f4a65352640 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896395d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a66354640 1 -- 172.21.15.5:0/1798804889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 msgr2=0x55896395d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a66354640 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896395d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a66354640 1 -- 172.21.15.5:0/1798804889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 msgr2=0x55896395d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a66354640 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x55896395d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a66354640 1 -- 172.21.15.5:0/1798804889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589635d4210 con 0x5589635bc1b0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a66354640 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x55896395cbe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4a58010370 tx=0x7f4a58010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d1e0f23cf177b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a56ffd640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4a4c0b2950 0x7f4a4c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a4bfff640 1 -- 172.21.15.5:0/1798804889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58016070 con 0x5589635bc1b0 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a4bfff640 1 -- 172.21.15.5:0/1798804889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a58004d20 con 0x5589635bc1b0 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a4bfff640 1 -- 172.21.15.5:0/1798804889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58005350 con 0x5589635bc1b0 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a65b53640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4a4c0b2950 0x7f4a4c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.027+0000 7f4a56ffd640 1 -- 172.21.15.5:0/3258623512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4a5c0e2c10 con 0x7f4a60103310 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1798804889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 msgr2=0x55896395cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a65b53640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4a4c0b2950 0x7f4a4c0b4e60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0e0050 tx=0x7f4a5c0e6760 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x55896395cbe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4a58010370 tx=0x7f4a58010830 comp rx=0 tx=0).stop 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1798804889 shutdown_connections 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896395d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x55896395d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1798804889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x55896395cbe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1798804889 >> 172.21.15.5:0/1798804889 conn(0x5589635b0300 msgr2=0x558963959340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1798804889 shutdown_connections 2024-05-14T15:29:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.028+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1798804889 wait complete. 2024-05-14T15:29:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 -- start start 2024-05-14T15:29:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 Processor -- start 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635932f0 0x558963967200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55896359b4d0 0x558963967740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x5589639731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558963969d10 con 0x5589635bc1b0 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55896387f370 con 0x55896359b4d0 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558963969e90 con 0x5589635932f0 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a6ec6d740 10 client.0 _ll_register_callbacks cb 0x5589635d3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.029+0000 7f4a66354640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x5589639731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a66354640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x5589639731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35600/0 (socket says 172.21.15.5:35600) 2024-05-14T15:29:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a66354640 1 -- 172.21.15.5:0/1767951280 learned_addr learned my addr 172.21.15.5:0/1767951280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55896359b4d0 0x558963967740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a66354640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635932f0 0x558963967200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 -- 172.21.15.5:0/1767951280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635932f0 msgr2=0x558963967200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635932f0 0x558963967200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 -- 172.21.15.5:0/1767951280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 msgr2=0x5589639731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x5589639731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 -- 172.21.15.5:0/1767951280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589636a0620 con 0x55896359b4d0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a65352640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55896359b4d0 0x558963967740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a5002a860 tx=0x7f4a50030fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5655474698210d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a497fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a497fa640 1 -- 172.21.15.5:0/1767951280 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50037040 con 0x55896359b4d0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a58004d20 con 0x55896359b4d0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a497fa640 1 -- 172.21.15.5:0/1767951280 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4a50030450 con 0x55896359b4d0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.030+0000 7f4a497fa640 1 -- 172.21.15.5:0/1767951280 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a50030650 con 0x55896359b4d0 2024-05-14T15:29:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 10 client.26027 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589639760b0 con 0x55896359b4d0 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a497fa640 1 -- 172.21.15.5:0/1767951280 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a58004d20 con 0x55896359b4d0 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a497fa640 1 -- 172.21.15.5:0/1767951280 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4a5003d450 con 0x55896359b4d0 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 10 client.26027 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 20 client.26027 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55896357ad80 0x558963979580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 4 client.26027 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589638d01e0 con 0x55896357ad80 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.031+0000 7f4a66354640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55896357ad80 0x558963979580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.032+0000 7f4a497fa640 10 client.26027 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.032+0000 7f4a66354640 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55896357ad80 0x558963979580 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990d7fde26583a5b server_cookie=9424db0daec19de7 in_seq=0 out_seq=0 2024-05-14T15:29:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.032+0000 7f4a497fa640 1 -- 172.21.15.5:0/1767951280 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589638d01e0 con 0x55896357ad80 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.032+0000 7f4a497fa640 10 client.26027 handle_command_reply: tid=0 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.032+0000 7f4a6ec6d740 1 client.26027 shutdown 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55896359b4d0 msgr2=0x558963967740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55896359b4d0 0x558963967740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a5002a860 tx=0x7f4a50030fa0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 shutdown_connections 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55896357ad80 0x558963979580 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635bc1b0 0x5589639731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55896359b4d0 0x558963967740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a497fa640 0 client.26027 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/1767951280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635932f0 0x558963967200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 >> 172.21.15.5:0/1767951280 conn(0x5589635b0300 msgr2=0x55896395ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 shutdown_connections 2024-05-14T15:29:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/1767951280 wait complete. 2024-05-14T15:29:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.033+0000 7f4a6ec6d740 20 client.26027 trim_cache size 0 max 16384 2024-05-14T15:29:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.050+0000 7f4a6ec6d740 1 Processor -- start 2024-05-14T15:29:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 -- start start 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x5589639618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 0x558963959450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635bc1b0 0x558963972da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b5e50 con 0x55896359b4d0 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b3720 con 0x5589635932f0 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b1570 con 0x5589635bc1b0 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 0x558963959450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 0x558963959450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35610/0 (socket says 172.21.15.5:35610) 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 -- 172.21.15.5:0/4280940336 learned_addr learned my addr 172.21.15.5:0/4280940336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a66354640 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x5589639618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a66354640 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635bc1b0 0x558963972da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 -- 172.21.15.5:0/4280940336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635bc1b0 msgr2=0x558963972da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635bc1b0 0x558963972da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 -- 172.21.15.5:0/4280940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 msgr2=0x5589639618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x5589639618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.051+0000 7f4a65352640 1 -- 172.21.15.5:0/4280940336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558963967de0 con 0x55896359b4d0 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.052+0000 7f4a65352640 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 0x558963959450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009780 tx=0x7f4a5002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdedb4c58a3a633f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.052+0000 7f4a4b7fe640 1 -- 172.21.15.5:0/4280940336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50037050 con 0x55896359b4d0 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.052+0000 7f4a4b7fe640 1 -- 172.21.15.5:0/4280940336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50005d30 con 0x55896359b4d0 2024-05-14T15:29:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.052+0000 7f4a4b7fe640 1 -- 172.21.15.5:0/4280940336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5003d5f0 con 0x55896359b4d0 2024-05-14T15:29:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/4280940336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 msgr2=0x558963959450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 0x558963959450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009780 tx=0x7f4a5002a340 comp rx=0 tx=0).stop 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/4280940336 shutdown_connections 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589635bc1b0 0x558963972da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896359b4d0 0x558963959450 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/4280940336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635932f0 0x5589639618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/4280940336 >> 172.21.15.5:0/4280940336 conn(0x5589635b0300 msgr2=0x558963958d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/4280940336 shutdown_connections 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/4280940336 wait complete. 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 -- start start 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.053+0000 7f4a6ec6d740 1 Processor -- start 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 0x55896396f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896396a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635bc1b0 0x55896396ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b1be0 con 0x5589635932f0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b3050 con 0x5589635bc1b0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601b22b0 con 0x55896359b4d0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a6ec6d740 10 client.0 _ll_register_callbacks cb 0x7f4a500096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 0x55896396f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 0x55896396f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35620/0 (socket says 172.21.15.5:35620) 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 -- 172.21.15.5:0/3321972465 learned_addr learned my addr 172.21.15.5:0/3321972465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635bc1b0 0x55896396ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 -- 172.21.15.5:0/3321972465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 msgr2=0x55896396a7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896396a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 -- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635bc1b0 msgr2=0x55896396ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635bc1b0 0x55896396ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 -- 172.21.15.5:0/3321972465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55896396b220 con 0x5589635932f0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a66354640 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 0x55896396f400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4a58010370 tx=0x7f4a58010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf3d1cb2ed948d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a497fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.054+0000 7f4a497fa640 1 -- 172.21.15.5:0/3321972465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58016040 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a497fa640 1 -- 172.21.15.5:0/3321972465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4a580128b0 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a50005d30 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a497fa640 1 -- 172.21.15.5:0/3321972465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a58023040 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 10 client.16407 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5589639e1530 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a497fa640 1 -- 172.21.15.5:0/3321972465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50005d30 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a497fa640 1 -- 172.21.15.5:0/3321972465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4a58029040 con 0x5589635932f0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 10 client.16407 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 20 client.16407 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5589635501c0 0x5589639e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 4 client.16407 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589638ca050 con 0x5589635501c0 2024-05-14T15:29:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.055+0000 7f4a65352640 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5589635501c0 0x5589639e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.056+0000 7f4a497fa640 10 client.16407 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.056+0000 7f4a65352640 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5589635501c0 0x5589639e3b60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fdec922f753d1ee server_cookie=7f845670947e9ab6 in_seq=0 out_seq=0 2024-05-14T15:29:06.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 333179 11.60 MB/sec execute 27 sec latency 89.203 ms 2024-05-14T15:29:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.291+0000 7f4a497fa640 1 -- 172.21.15.5:0/3321972465 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589638ca050 con 0x5589635501c0 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.291+0000 7f4a497fa640 10 client.16407 handle_command_reply: tid=0 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.291+0000 7f4a6ec6d740 1 client.16407 shutdown 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 msgr2=0x55896396f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 0x55896396f400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4a58010370 tx=0x7f4a58010830 comp rx=0 tx=0).stop 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 shutdown_connections 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5589635501c0 0x5589639e3b60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55896359b4d0 0x55896396a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589635bc1b0 0x55896396ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a497fa640 0 client.16407 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 --2- 172.21.15.5:0/3321972465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589635932f0 0x55896396f400 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 >> 172.21.15.5:0/3321972465 conn(0x5589635b0300 msgr2=0x55896395c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 shutdown_connections 2024-05-14T15:29:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 1 -- 172.21.15.5:0/3321972465 wait complete. 2024-05-14T15:29:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a6ec6d740 20 client.16407 trim_cache size 0 max 16384 2024-05-14T15:29:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:29:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:29:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:29:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4a4c0b2950 msgr2=0x7f4a4c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4a4c0b2950 0x7f4a4c0b4e60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0e0050 tx=0x7f4a5c0e6760 comp rx=0 tx=0).stop 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 msgr2=0x7f4a600ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 0x7f4a600ff3a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00e9a0 tx=0x7f4a5c00ee60 comp rx=0 tx=0).stop 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 shutdown_connections 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4a4c0b2950 0x7f4a4c0b4e60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a60108bc0 0x7f4a600ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.292+0000 7f4a67db7640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a60109060 0x7f4a600ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.293+0000 7f4a67db7640 1 --2- 172.21.15.5:0/3258623512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a60103310 0x7f4a600ff3a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.293+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 >> 172.21.15.5:0/3258623512 conn(0x7f4a600781b0 msgr2=0x7f4a601014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.293+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 shutdown_connections 2024-05-14T15:29:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:06.293+0000 7f4a67db7640 1 -- 172.21.15.5:0/3258623512 wait complete. 2024-05-14T15:29:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:05 smithi144 ceph-mon[35650]: pgmap v132: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 387 op/s 2024-05-14T15:29:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:05 smithi193 ceph-mon[36001]: pgmap v132: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 387 op/s 2024-05-14T15:29:06.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:05 smithi005 ceph-mon[31559]: pgmap v132: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 387 op/s 2024-05-14T15:29:07.132 INFO:tasks.workunit.client.0.smithi005.stdout: 1 334380 11.38 MB/sec execute 28 sec latency 136.726 ms 2024-05-14T15:29:08.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337336 11.62 MB/sec execute 29 sec latency 9.811 ms 2024-05-14T15:29:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:07 smithi144 ceph-mon[35650]: pgmap v133: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 645 op/s 2024-05-14T15:29:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:07 smithi193 ceph-mon[36001]: pgmap v133: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 645 op/s 2024-05-14T15:29:08.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:07 smithi005 ceph-mon[31559]: pgmap v133: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 645 op/s 2024-05-14T15:29:09.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 339872 11.63 MB/sec execute 30 sec latency 13.643 ms 2024-05-14T15:29:10.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 341278 11.50 MB/sec execute 31 sec latency 41.461 ms 2024-05-14T15:29:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:09 smithi144 ceph-mon[35650]: pgmap v134: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 394 op/s 2024-05-14T15:29:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:09 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:29:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:09 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:29:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:09 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:29:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:09 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:09 smithi193 ceph-mon[36001]: pgmap v134: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 394 op/s 2024-05-14T15:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:09 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:09 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:09 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:09 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:29:10.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:09 smithi005 ceph-mon[31559]: pgmap v134: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 394 op/s 2024-05-14T15:29:10.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:09 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:29:10.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:09 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:29:10.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:09 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:29:10.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:09 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:29:11.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 344160 11.68 MB/sec execute 32 sec latency 32.828 ms 2024-05-14T15:29:12.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 346367 11.58 MB/sec execute 33 sec latency 23.016 ms 2024-05-14T15:29:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:11 smithi144 ceph-mon[35650]: pgmap v135: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 394 op/s 2024-05-14T15:29:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:11 smithi193 ceph-mon[36001]: pgmap v135: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 394 op/s 2024-05-14T15:29:12.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:11 smithi005 ceph-mon[31559]: pgmap v135: 129 pgs: 129 active+clean; 41 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 394 op/s 2024-05-14T15:29:13.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348341 11.61 MB/sec execute 34 sec latency 15.251 ms 2024-05-14T15:29:14.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 351276 11.77 MB/sec execute 35 sec latency 12.775 ms 2024-05-14T15:29:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:13 smithi144 ceph-mon[35650]: pgmap v136: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 513 op/s 2024-05-14T15:29:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:13 smithi193 ceph-mon[36001]: pgmap v136: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 513 op/s 2024-05-14T15:29:14.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:13 smithi005 ceph-mon[31559]: pgmap v136: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 513 op/s 2024-05-14T15:29:15.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 352826 11.63 MB/sec execute 36 sec latency 58.673 ms 2024-05-14T15:29:16.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 355134 11.69 MB/sec execute 37 sec latency 21.899 ms 2024-05-14T15:29:16.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:15 smithi144 ceph-mon[35650]: pgmap v137: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-05-14T15:29:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:15 smithi193 ceph-mon[36001]: pgmap v137: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-05-14T15:29:16.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:15 smithi005 ceph-mon[31559]: pgmap v137: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 376 op/s 2024-05-14T15:29:17.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 357673 11.71 MB/sec execute 38 sec latency 16.732 ms 2024-05-14T15:29:18.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359467 11.64 MB/sec execute 39 sec latency 14.730 ms 2024-05-14T15:29:18.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:17 smithi144 ceph-mon[35650]: pgmap v138: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2024-05-14T15:29:18.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:17 smithi193 ceph-mon[36001]: pgmap v138: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2024-05-14T15:29:18.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:17 smithi005 ceph-mon[31559]: pgmap v138: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.8 MiB/s wr, 497 op/s 2024-05-14T15:29:19.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 362211 11.77 MB/sec execute 40 sec latency 16.375 ms 2024-05-14T15:29:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 -- 172.21.15.5:0/912045346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4103310 msgr2=0x7efdf410f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 --2- 172.21.15.5:0/912045346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4103310 0x7efdf410f6e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efdec009ca0 tx=0x7efdec02a8c0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 -- 172.21.15.5:0/912045346 shutdown_connections 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 --2- 172.21.15.5:0/912045346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4109060 0x7efdf4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 --2- 172.21.15.5:0/912045346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4103310 0x7efdf410f6e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 --2- 172.21.15.5:0/912045346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 0x7efdf4102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 -- 172.21.15.5:0/912045346 >> 172.21.15.5:0/912045346 conn(0x7efdf40781b0 msgr2=0x7efdf4078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 -- 172.21.15.5:0/912045346 shutdown_connections 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.403+0000 7efdfa049640 1 -- 172.21.15.5:0/912045346 wait complete. 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.404+0000 7efdfa049640 1 Processor -- start 2024-05-14T15:29:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.404+0000 7efdfa049640 1 -- start start 2024-05-14T15:29:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.404+0000 7efdfa049640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 0x7efdf4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdfa049640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 0x7efdf4199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdfa049640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4109060 0x7efdf419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdfa049640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4112dd0 con 0x7efdf4109060 2024-05-14T15:29:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdfa049640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4112c50 con 0x7efdf4108bc0 2024-05-14T15:29:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdfa049640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4112f50 con 0x7efdf4103310 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4109060 0x7efdf419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 0x7efdf4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 0x7efdf4199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 0x7efdf4199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36788/0 (socket says 172.21.15.5:36788) 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4109060 0x7efdf419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54838/0 (socket says 172.21.15.5:54838) 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 -- 172.21.15.5:0/2548484842 learned_addr learned my addr 172.21.15.5:0/2548484842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 -- 172.21.15.5:0/2548484842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 msgr2=0x7efdf4199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 0x7efdf4199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 -- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4109060 msgr2=0x7efdf419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4109060 0x7efdf419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 -- 172.21.15.5:0/2548484842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf419e5d0 con 0x7efdf4103310 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 0x7efdf4199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf37fe640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 0x7efdf4199770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efde400e9c0 tx=0x7efde400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf0ff9640 1 -- 172.21.15.5:0/2548484842 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde400cd80 con 0x7efdf4103310 2024-05-14T15:29:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.405+0000 7efdf0ff9640 1 -- 172.21.15.5:0/2548484842 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde40054d0 con 0x7efdf4103310 2024-05-14T15:29:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.406+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf419e8c0 con 0x7efdf4103310 2024-05-14T15:29:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.406+0000 7efdf0ff9640 1 -- 172.21.15.5:0/2548484842 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4010430 con 0x7efdf4103310 2024-05-14T15:29:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.406+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41a6240 con 0x7efdf4103310 2024-05-14T15:29:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 Processor -- start 2024-05-14T15:29:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 -- start start 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f3e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0202f0 0x560e4f3e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f3ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e4f34f5d0 con 0x560e4f0284d0 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560e4f308890 con 0x560e4f0491b0 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.408+0000 7efe00fd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560e4f34b590 con 0x560e4f0202f0 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f3e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f3ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f3e9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41472/0 (socket says 172.21.15.5:41472) 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 -- 172.21.15.5:0/3693554163 learned_addr learned my addr 172.21.15.5:0/3693554163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f3ea4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54844/0 (socket says 172.21.15.5:54844) 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf0ff9640 1 -- 172.21.15.5:0/2548484842 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7efde4004fd0 con 0x7efdf4103310 2024-05-14T15:29:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 -- 172.21.15.5:0/3693554163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0202f0 msgr2=0x560e4f3e9f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0202f0 0x560e4f3e9f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 -- 172.21.15.5:0/3693554163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 msgr2=0x560e4f3ea4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f3ea4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 -- 172.21.15.5:0/3693554163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4f061210 con 0x560e4f0491b0 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdf3fff640 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f3e9a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdec02f410 tx=0x7efdec02f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af880bb75b68c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdd9ffb640 1 -- 172.21.15.5:0/3693554163 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec031cf0 con 0x560e4f0491b0 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdd9ffb640 1 -- 172.21.15.5:0/3693554163 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec036030 con 0x560e4f0491b0 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.409+0000 7efdd9ffb640 1 -- 172.21.15.5:0/3693554163 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec004240 con 0x560e4f0491b0 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.410+0000 7efdf0ff9640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efddc0b2950 0x7efddc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.410+0000 7efdf37fe640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efddc0b2950 0x7efddc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.410+0000 7efdf0ff9640 1 -- 172.21.15.5:0/2548484842 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7efde40e2e40 con 0x7efdf4103310 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.410+0000 7efe00fd3740 1 -- 172.21.15.5:0/3693554163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 msgr2=0x560e4f3e9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f3e9a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdec02f410 tx=0x7efdec02f440 comp rx=0 tx=0).stop 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efdf37fe640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efddc0b2950 0x7efddc0b4e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x560e4f3e76d0 tx=0x7efde40049b0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 -- 172.21.15.5:0/3693554163 shutdown_connections 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0202f0 0x560e4f3e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f3ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 --2- 172.21.15.5:0/3693554163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f3e9a50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 -- 172.21.15.5:0/3693554163 >> 172.21.15.5:0/3693554163 conn(0x560e4f03d300 msgr2=0x560e4f3e6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 -- 172.21.15.5:0/3693554163 shutdown_connections 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 -- 172.21.15.5:0/3693554163 wait complete. 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 -- start start 2024-05-14T15:29:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.411+0000 7efe00fd3740 1 Processor -- start 2024-05-14T15:29:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f3f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0491b0 0x560e4f402250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e4f3f8da0 con 0x560e4f0491b0 2024-05-14T15:29:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560e4f30c370 con 0x560e4f0202f0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560e4f3f8f20 con 0x560e4f0284d0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efe00fd3740 10 client.0 _ll_register_callbacks cb 0x560e4f060a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41478/0 (socket says 172.21.15.5:41478) 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 -- 172.21.15.5:0/191521228 learned_addr learned my addr 172.21.15.5:0/191521228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0491b0 0x560e4f402250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f3f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 -- 172.21.15.5:0/191521228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 msgr2=0x560e4f3f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f3f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 -- 172.21.15.5:0/191521228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0491b0 msgr2=0x560e4f402250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.412+0000 7efdf3fff640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0491b0 0x560e4f402250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdf3fff640 1 -- 172.21.15.5:0/191521228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4f12d620 con 0x560e4f0202f0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdf3fff640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3f6340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efdec009620 tx=0x7efdec003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a924f18ece9a08 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdd2ffd640 1 -- 172.21.15.5:0/191521228 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec059060 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdec036030 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdd2ffd640 1 -- 172.21.15.5:0/191521228 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7efdec031450 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdd2ffd640 1 -- 172.21.15.5:0/191521228 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdec031e60 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efe00fd3740 10 client.26045 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e4f3f8da0 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.413+0000 7efdd2ffd640 1 -- 172.21.15.5:0/191521228 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec036030 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efdd2ffd640 1 -- 172.21.15.5:0/191521228 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efdec043070 con 0x560e4f0202f0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efe00fd3740 10 client.26045 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efe00fd3740 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efe00fd3740 20 client.26045 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efe00fd3740 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4f007d80 0x560e4f4081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efe00fd3740 4 client.26045 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e4f35d1e0 con 0x560e4f007d80 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4f007d80 0x560e4f4081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efdd2ffd640 10 client.26045 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.414+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4f007d80 0x560e4f4081d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69da64075a8df3f server_cookie=7e2d9e20d11c6236 in_seq=0 out_seq=0 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efdd2ffd640 1 -- 172.21.15.5:0/191521228 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e4f35d1e0 con 0x560e4f007d80 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efdd2ffd640 10 client.26045 handle_command_reply: tid=0 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 client.26045 shutdown 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 msgr2=0x560e4f3f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3f6340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efdec009620 tx=0x7efdec003d70 comp rx=0 tx=0).stop 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 shutdown_connections 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4f007d80 0x560e4f4081d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f3f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efdd2ffd640 0 client.26045 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0491b0 0x560e4f402250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 --2- 172.21.15.5:0/191521228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3f6340 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.415+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 >> 172.21.15.5:0/191521228 conn(0x560e4f03d300 msgr2=0x560e4f3ea3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.416+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 shutdown_connections 2024-05-14T15:29:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.416+0000 7efe00fd3740 1 -- 172.21.15.5:0/191521228 wait complete. 2024-05-14T15:29:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.416+0000 7efe00fd3740 20 client.26045 trim_cache size 0 max 16384 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 Processor -- start 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 -- start start 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f40d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0491b0 0x560e4f40de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41b1990 con 0x560e4f0284d0 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41af260 con 0x560e4f0202f0 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efe00fd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41ad0b0 con 0x560e4f0491b0 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.433+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f40d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f40d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54866/0 (socket says 172.21.15.5:54866) 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 -- 172.21.15.5:0/1409815474 learned_addr learned my addr 172.21.15.5:0/1409815474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf3fff640 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0491b0 0x560e4f40de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 -- 172.21.15.5:0/1409815474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0491b0 msgr2=0x560e4f40de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0491b0 0x560e4f40de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf3fff640 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 -- 172.21.15.5:0/1409815474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 msgr2=0x560e4f3efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 -- 172.21.15.5:0/1409815474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4f3ea980 con 0x560e4f0284d0 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f40d920 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efde8009fd0 tx=0x7efde8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6533f432d9a09b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdd97fa640 1 -- 172.21.15.5:0/1409815474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8025070 con 0x560e4f0284d0 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.434+0000 7efdd97fa640 1 -- 172.21.15.5:0/1409815474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8003aa0 con 0x560e4f0284d0 2024-05-14T15:29:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efdd97fa640 1 -- 172.21.15.5:0/1409815474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde80206b0 con 0x560e4f0284d0 2024-05-14T15:29:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 -- 172.21.15.5:0/1409815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 msgr2=0x560e4f40d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f40d920 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efde8009fd0 tx=0x7efde8005f90 comp rx=0 tx=0).stop 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 -- 172.21.15.5:0/1409815474 shutdown_connections 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0491b0 0x560e4f40de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0284d0 0x560e4f40d920 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 --2- 172.21.15.5:0/1409815474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0202f0 0x560e4f3efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.435+0000 7efe00fd3740 1 -- 172.21.15.5:0/1409815474 >> 172.21.15.5:0/1409815474 conn(0x560e4f03d300 msgr2=0x560e4f3ec0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.436+0000 7efe00fd3740 1 -- 172.21.15.5:0/1409815474 shutdown_connections 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.436+0000 7efe00fd3740 1 -- 172.21.15.5:0/1409815474 wait complete. 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.436+0000 7efe00fd3740 1 -- start start 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.436+0000 7efe00fd3740 1 Processor -- start 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 0x560e4f401250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f401790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f401cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41ad720 con 0x560e4f0202f0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41aeb90 con 0x560e4f0491b0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41addf0 con 0x560e4f0284d0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 10 client.0 _ll_register_callbacks cb 0x7efde8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 0x560e4f401250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 0x560e4f401250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54868/0 (socket says 172.21.15.5:54868) 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 -- 172.21.15.5:0/4152549525 learned_addr learned my addr 172.21.15.5:0/4152549525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f401790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f401cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 -- 172.21.15.5:0/4152549525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 msgr2=0x560e4f401790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f401790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 -- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 msgr2=0x560e4f401cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f401cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 -- 172.21.15.5:0/4152549525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4f3fd3e0 con 0x560e4f0202f0 2024-05-14T15:29:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdf3fff640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 0x560e4f401250 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efdec002230 tx=0x7efdec003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840e762e12aa064c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdd2ffd640 1 -- 172.21.15.5:0/4152549525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec059040 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efdd2ffd640 1 -- 172.21.15.5:0/4152549525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7efdec004b40 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.437+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde8003aa0 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efdd2ffd640 1 -- 172.21.15.5:0/4152549525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdec06e050 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 10 client.16425 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e4f3fe670 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efdd2ffd640 1 -- 172.21.15.5:0/4152549525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8003aa0 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efdd2ffd640 1 -- 172.21.15.5:0/4152549525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efdec064030 con 0x560e4f0202f0 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 10 client.16425 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 20 client.16425 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4efdd1c0 0x560e4f4724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 4 client.16425 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:29:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.438+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e4f357050 con 0x560e4efdd1c0 2024-05-14T15:29:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.439+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4efdd1c0 0x560e4f4724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.439+0000 7efdd2ffd640 10 client.16425 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.439+0000 7efdf2ffd640 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4efdd1c0 0x560e4f4724e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5143ee9b4264c4 server_cookie=55a2ac4b7f31cec3 in_seq=0 out_seq=0 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efdd2ffd640 1 -- 172.21.15.5:0/4152549525 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e4f357050 con 0x560e4efdd1c0 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efdd2ffd640 10 client.16425 handle_command_reply: tid=0 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efe00fd3740 1 client.16425 shutdown 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 msgr2=0x560e4f401250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efe00fd3740 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 0x560e4f401250 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efdec002230 tx=0x7efdec003d70 comp rx=0 tx=0).stop 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 shutdown_connections 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.696+0000 7efe00fd3740 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560e4efdd1c0 0x560e4f4724e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efdd2ffd640 0 client.16425 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e4f0284d0 0x560e4f401790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e4f0491b0 0x560e4f401cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 1 --2- 172.21.15.5:0/4152549525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e4f0202f0 0x560e4f401250 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 >> 172.21.15.5:0/4152549525 conn(0x560e4f03d300 msgr2=0x560e4f3ea100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 shutdown_connections 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 1 -- 172.21.15.5:0/4152549525 wait complete. 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.697+0000 7efe00fd3740 20 client.16425 trim_cache size 0 max 16384 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:29:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.698+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efddc0b2950 msgr2=0x7efddc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.698+0000 7efdfa049640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efddc0b2950 0x7efddc0b4e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x560e4f3e76d0 tx=0x7efde40049b0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.698+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 msgr2=0x7efdf4199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 0x7efdf4199770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efde400e9c0 tx=0x7efde400ee80 comp rx=0 tx=0).stop 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 shutdown_connections 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efddc0b2950 0x7efddc0b4e60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf4109060 0x7efdf419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdf4108bc0 0x7efdf4199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 --2- 172.21.15.5:0/2548484842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdf4103310 0x7efdf4199770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 >> 172.21.15.5:0/2548484842 conn(0x7efdf40781b0 msgr2=0x7efdf40ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 shutdown_connections 2024-05-14T15:29:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:19.699+0000 7efdfa049640 1 -- 172.21.15.5:0/2548484842 wait complete. 2024-05-14T15:29:19.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:29:20.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:19 smithi005 ceph-mon[31559]: pgmap v139: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-05-14T15:29:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:20.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 363706 11.62 MB/sec execute 41 sec latency 156.974 ms 2024-05-14T15:29:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:19 smithi144 ceph-mon[35650]: pgmap v139: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-05-14T15:29:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:19 smithi193 ceph-mon[36001]: pgmap v139: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-05-14T15:29:21.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 365719 11.67 MB/sec execute 42 sec latency 74.700 ms 2024-05-14T15:29:22.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 368484 11.70 MB/sec execute 43 sec latency 14.285 ms 2024-05-14T15:29:22.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:21 smithi005 ceph-mon[31559]: pgmap v140: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-05-14T15:29:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.175+0000 7f4f15a48700 1 -- 172.21.15.5:0/3966764695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10103550 msgr2=0x7f4f10103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.175+0000 7f4f15a48700 1 --2- 172.21.15.5:0/3966764695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10103550 0x7f4f10103960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4f000099c0 tx=0x7f4f00009cd0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 -- 172.21.15.5:0/3966764695 shutdown_connections 2024-05-14T15:29:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 --2- 172.21.15.5:0/3966764695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10108780 0x7f4f101100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 --2- 172.21.15.5:0/3966764695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10103550 0x7f4f10103960 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 --2- 172.21.15.5:0/3966764695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10107e30 0x7f4f10108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 -- 172.21.15.5:0/3966764695 >> 172.21.15.5:0/3966764695 conn(0x7f4f10078e00 msgr2=0x7f4f10079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 -- 172.21.15.5:0/3966764695 shutdown_connections 2024-05-14T15:29:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.176+0000 7f4f15a48700 1 -- 172.21.15.5:0/3966764695 wait complete. 2024-05-14T15:29:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.177+0000 7f4f15a48700 1 Processor -- start 2024-05-14T15:29:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.177+0000 7f4f15a48700 1 -- start start 2024-05-14T15:29:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.177+0000 7f4f15a48700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 0x7f4f1019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f15a48700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10107e30 0x7f4f1019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f15a48700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10108780 0x7f4f1019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f15a48700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101133b0 con 0x7f4f10108780 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f15a48700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10113230 con 0x7f4f10103550 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f15a48700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10113530 con 0x7f4f10107e30 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 0x7f4f1019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10108780 0x7f4f1019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 0x7f4f1019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41506/0 (socket says 172.21.15.5:41506) 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 -- 172.21.15.5:0/2942254140 learned_addr learned my addr 172.21.15.5:0/2942254140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10107e30 0x7f4f1019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 -- 172.21.15.5:0/2942254140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10107e30 msgr2=0x7f4f1019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10107e30 0x7f4f1019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 -- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10108780 msgr2=0x7f4f1019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10108780 0x7f4f1019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 -- 172.21.15.5:0/2942254140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f101a7040 con 0x7f4f10103550 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10107e30 0x7f4f1019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10108780 0x7f4f1019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.178+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 0x7f4f1019bcd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f00004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.179+0000 7f4f14a46700 1 -- 172.21.15.5:0/2942254140 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00026070 con 0x7f4f10103550 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.179+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f101a72d0 con 0x7f4f10103550 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.179+0000 7f4f14a46700 1 -- 172.21.15.5:0/2942254140 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f00004630 con 0x7f4f10103550 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.179+0000 7f4f14a46700 1 -- 172.21.15.5:0/2942254140 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000fb20 con 0x7f4f10103550 2024-05-14T15:29:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.179+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101a76c0 con 0x7f4f10103550 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.183+0000 7f4f14a46700 1 -- 172.21.15.5:0/2942254140 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4f00029070 con 0x7f4f10103550 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.183+0000 7f4f2445cb80 1 Processor -- start 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.183+0000 7f4f2445cb80 1 -- start start 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 0x559a4b1b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a4b0d7940 con 0x559a4b037e40 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a4b03cda0 con 0x559a4b1b5e10 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a4b0d3900 con 0x559a4b1bbd80 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 0x559a4b1b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 0x559a4b1b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36854/0 (socket says 172.21.15.5:36854) 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0effd700 1 -- 172.21.15.5:0/1410771165 learned_addr learned my addr 172.21.15.5:0/1410771165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f14a46700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef80b2a70 0x7f4ef80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0e7fc700 1 -- 172.21.15.5:0/1410771165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 msgr2=0x559a4b1b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 0x559a4b1b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0e7fc700 1 -- 172.21.15.5:0/1410771165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 msgr2=0x559a4b1bb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0e7fc700 1 -- 172.21.15.5:0/1410771165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4b0248d0 con 0x559a4b1b5e10 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.184+0000 7f4f0effd700 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 0x559a4b1b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.185+0000 7f4f14a46700 1 -- 172.21.15.5:0/2942254140 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4f000e3200 con 0x7f4f10103550 2024-05-14T15:29:22.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.185+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef80b2a70 0x7f4ef80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.185+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b6220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f1006d180 tx=0x7f4f040117a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930d997a777a6f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:22.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.185+0000 7f4ef5ffb700 1 -- 172.21.15.5:0/1410771165 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04004230 con 0x559a4b1b5e10 2024-05-14T15:29:22.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.185+0000 7f4ef5ffb700 1 -- 172.21.15.5:0/1410771165 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f040043d0 con 0x559a4b1b5e10 2024-05-14T15:29:22.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.185+0000 7f4ef5ffb700 1 -- 172.21.15.5:0/1410771165 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04005650 con 0x559a4b1b5e10 2024-05-14T15:29:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 -- 172.21.15.5:0/1410771165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 msgr2=0x559a4b1b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef80b2a70 0x7f4ef80b4f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009fd0 tx=0x7f4f08009040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b6220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4f1006d180 tx=0x7f4f040117a0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 -- 172.21.15.5:0/1410771165 shutdown_connections 2024-05-14T15:29:22.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b6220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/1410771165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1bbd80 0x559a4b1b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 -- 172.21.15.5:0/1410771165 >> 172.21.15.5:0/1410771165 conn(0x559a4b07f630 msgr2=0x559a4b1b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 -- 172.21.15.5:0/1410771165 shutdown_connections 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.186+0000 7f4f2445cb80 1 -- 172.21.15.5:0/1410771165 wait complete. 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 -- start start 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 Processor -- start 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a4b1c2820 con 0x559a4b037e40 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a4b090ed0 con 0x559a4b1bbd80 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a4b1c29a0 con 0x559a4b1b5e10 2024-05-14T15:29:22.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f2445cb80 10 client.0 _ll_register_callbacks cb 0x559a4b06f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.187+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54898/0 (socket says 172.21.15.5:54898) 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0effd700 1 -- 172.21.15.5:0/2399242102 learned_addr learned my addr 172.21.15.5:0/2399242102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2399242102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 msgr2=0x559a4b1bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1bf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2399242102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 msgr2=0x559a4b1bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2399242102 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4b08e760 con 0x559a4b1bbd80 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1bfea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f10076ad0 tx=0x7f4f08004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c6067ff89183a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4eeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2399242102 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800eb80 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f040043d0 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.188+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2399242102 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4f0800ed20 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2399242102 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f0802b820 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 10 client.26060 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a4b1cd470 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2399242102 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f040043d0 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2399242102 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4f08017ab0 con 0x559a4b1bbd80 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 10 client.26060 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 20 client.26060 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b1cfa00 0x559a4b1d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 4 client.26060 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:22.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a4b0e5320 con 0x559a4b1cfa00 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.189+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b1cfa00 0x559a4b1d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.190+0000 7f4eeeffd700 10 client.26060 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.190+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b1cfa00 0x559a4b1d1e40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2718d40d01eb17 server_cookie=9ec3831d709e6d6c in_seq=0 out_seq=0 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.190+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2399242102 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a4b0e5320 con 0x559a4b1cfa00 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.190+0000 7f4eeeffd700 10 client.26060 handle_command_reply: tid=0 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.190+0000 7f4f2445cb80 1 client.26060 shutdown 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 msgr2=0x559a4b1bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1bfea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f10076ad0 tx=0x7f4f08004ad0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 shutdown_connections 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b1cfa00 0x559a4b1d1e40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1bfea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4eeeffd700 0 client.26060 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2399242102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 >> 172.21.15.5:0/2399242102 conn(0x559a4b07f630 msgr2=0x559a4b1b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:22.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 shutdown_connections 2024-05-14T15:29:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2399242102 wait complete. 2024-05-14T15:29:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.191+0000 7f4f2445cb80 20 client.26060 trim_cache size 0 max 16384 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 Processor -- start 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 -- start start 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b037e40 0x559a4b1b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 0x559a4b1cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b2e80 con 0x559a4b1bbd80 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b0710 con 0x559a4b1b5e10 2024-05-14T15:29:22.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101ae560 con 0x559a4b037e40 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 0x559a4b1cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 0x559a4b1cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54912/0 (socket says 172.21.15.5:54912) 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2563967130 learned_addr learned my addr 172.21.15.5:0/2563967130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.212+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b037e40 0x559a4b1b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2563967130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b037e40 msgr2=0x559a4b1b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b037e40 0x559a4b1b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2563967130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 msgr2=0x559a4b1b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4f0f7fe700 1 -- 172.21.15.5:0/2563967130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4b1bb3a0 con 0x559a4b1bbd80 2024-05-14T15:29:22.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 0x559a4b1cb8e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4f0802d040 tx=0x7f4f080043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3346e0f01a5bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4ef57fa700 1 -- 172.21.15.5:0/2563967130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0802b400 con 0x559a4b1bbd80 2024-05-14T15:29:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4ef57fa700 1 -- 172.21.15.5:0/2563967130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f08027070 con 0x559a4b1bbd80 2024-05-14T15:29:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.213+0000 7f4ef57fa700 1 -- 172.21.15.5:0/2563967130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800eb20 con 0x559a4b1bbd80 2024-05-14T15:29:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2563967130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 msgr2=0x559a4b1cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 0x559a4b1cb8e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4f0802d040 tx=0x7f4f080043a0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2563967130 shutdown_connections 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b1bbd80 0x559a4b1cb8e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b037e40 0x559a4b1b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2563967130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1b5e10 0x559a4b1b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2563967130 >> 172.21.15.5:0/2563967130 conn(0x559a4b07f630 msgr2=0x559a4b1b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2563967130 shutdown_connections 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.214+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2563967130 wait complete. 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 -- start start 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 Processor -- start 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101aebd0 con 0x559a4b037e40 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101b0040 con 0x559a4b1bbd80 2024-05-14T15:29:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101af2a0 con 0x559a4b1b5e10 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f2445cb80 10 client.0 _ll_register_callbacks cb 0x559a4b06f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f0effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54918/0 (socket says 172.21.15.5:54918) 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.215+0000 7f4f0effd700 1 -- 172.21.15.5:0/2253711298 learned_addr learned my addr 172.21.15.5:0/2253711298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0f7fe700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0effd700 1 -- 172.21.15.5:0/2253711298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 msgr2=0x559a4b1d69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1d69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0effd700 1 -- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 msgr2=0x559a4b1d6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1d6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0effd700 1 -- 172.21.15.5:0/2253711298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4b1d74f0 con 0x559a4b037e40 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f0effd700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1c8080 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f000a7420 tx=0x7f4f000e0af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc6e8911a831453 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4eeeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2253711298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f000a6070 con 0x559a4b037e40 2024-05-14T15:29:22.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2253711298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f4f000a9410 con 0x559a4b037e40 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f08027070 con 0x559a4b037e40 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2253711298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f000a9920 con 0x559a4b037e40 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f2445cb80 10 client.16443 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.216+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a4b239f90 con 0x559a4b037e40 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2253711298 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f08027070 con 0x559a4b037e40 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2253711298 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4f000e0ce0 con 0x559a4b037e40 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f2445cb80 10 client.16443 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f2445cb80 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f2445cb80 20 client.16443 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b23a6a0 0x559a4b23cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f2445cb80 4 client.16443 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a4b0df3c0 con 0x559a4b23a6a0 2024-05-14T15:29:22.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b23a6a0 0x559a4b23cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.217+0000 7f4eeeffd700 10 client.16443 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f0e7fc700 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b23a6a0 0x559a4b23cac0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63bd331aa611f1b server_cookie=461c73bcd374fd8 in_seq=0 out_seq=0 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4eeeffd700 1 -- 172.21.15.5:0/2253711298 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a4b0df3c0 con 0x559a4b23a6a0 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4eeeffd700 10 client.16443 handle_command_reply: tid=0 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 client.16443 shutdown 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 msgr2=0x559a4b1c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1c8080 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f000a7420 tx=0x7f4f000e0af0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 shutdown_connections 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a4b23a6a0 0x559a4b23cac0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a4b1bbd80 0x559a4b1d6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a4b037e40 0x559a4b1c8080 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4eeeffd700 0 client.16443 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 --2- 172.21.15.5:0/2253711298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a4b1b5e10 0x559a4b1d69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 >> 172.21.15.5:0/2253711298 conn(0x559a4b07f630 msgr2=0x559a4b1b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 shutdown_connections 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 1 -- 172.21.15.5:0/2253711298 wait complete. 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.218+0000 7f4f2445cb80 20 client.16443 trim_cache size 0 max 16384 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:29:22.224 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef80b2a70 msgr2=0x7f4ef80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef80b2a70 0x7f4ef80b4f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009fd0 tx=0x7f4f08009040 comp rx=0 tx=0).stop 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 msgr2=0x7f4f1019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 0x7f4f1019bcd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f00004270 comp rx=0 tx=0).stop 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 shutdown_connections 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef80b2a70 0x7f4ef80b4f80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f10108780 0x7f4f1019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f10103550 0x7f4f1019bcd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 --2- 172.21.15.5:0/2942254140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10107e30 0x7f4f1019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:22.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 >> 172.21.15.5:0/2942254140 conn(0x7f4f10078e00 msgr2=0x7f4f10100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:22.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 shutdown_connections 2024-05-14T15:29:22.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:22.219+0000 7f4f15a48700 1 -- 172.21.15.5:0/2942254140 wait complete. 2024-05-14T15:29:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:21 smithi144 ceph-mon[35650]: pgmap v140: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-05-14T15:29:22.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:21 smithi193 ceph-mon[36001]: pgmap v140: 129 pgs: 129 active+clean; 56 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-05-14T15:29:23.090 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:29:23.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:29:23.133 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370293 11.68 MB/sec execute 44 sec latency 13.784 ms 2024-05-14T15:29:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:24.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:23 smithi005 ceph-mon[31559]: pgmap v141: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 517 op/s 2024-05-14T15:29:24.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 372877 11.74 MB/sec execute 45 sec latency 56.124 ms 2024-05-14T15:29:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:23 smithi144 ceph-mon[35650]: pgmap v141: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 517 op/s 2024-05-14T15:29:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:23 smithi193 ceph-mon[36001]: pgmap v141: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 517 op/s 2024-05-14T15:29:25.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 375454 11.76 MB/sec execute 46 sec latency 13.388 ms 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.272+0000 7fa2bce59700 1 -- 172.21.15.5:0/2774404676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 msgr2=0x7fa2b8108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.272+0000 7fa2bce59700 1 --2- 172.21.15.5:0/2774404676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b8108050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009990 tx=0x7fa2ac009ca0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.272+0000 7fa2bce59700 1 -- 172.21.15.5:0/2774404676 shutdown_connections 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.273+0000 7fa2bce59700 1 --2- 172.21.15.5:0/2774404676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8108590 0x7fa2b810fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.273+0000 7fa2bce59700 1 --2- 172.21.15.5:0/2774404676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2b8103310 0x7fa2b8103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.273+0000 7fa2bce59700 1 --2- 172.21.15.5:0/2774404676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b8108050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.273+0000 7fa2bce59700 1 -- 172.21.15.5:0/2774404676 >> 172.21.15.5:0/2774404676 conn(0x7fa2b8078af0 msgr2=0x7fa2b8078f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.273+0000 7fa2bce59700 1 -- 172.21.15.5:0/2774404676 shutdown_connections 2024-05-14T15:29:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.273+0000 7fa2bce59700 1 -- 172.21.15.5:0/2774404676 wait complete. 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 Processor -- start 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 -- start start 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8103310 0x7fa2b81a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2b8108590 0x7fa2b81a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b8112120 con 0x7fa2b8103310 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b8111fa0 con 0x7fa2b8107c40 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2bce59700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b81122a0 con 0x7fa2b8108590 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2b5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:25.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2b6d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2b8108590 0x7fa2b81a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2b5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b81a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:41564/0 (socket says 172.21.15.5:41564) 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.274+0000 7fa2b5d9b700 1 -- 172.21.15.5:0/1282425965 learned_addr learned my addr 172.21.15.5:0/1282425965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b659c700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8103310 0x7fa2b81a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b5d9b700 1 -- 172.21.15.5:0/1282425965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2b8108590 msgr2=0x7fa2b81a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b5d9b700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2b8108590 0x7fa2b81a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b5d9b700 1 -- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8103310 msgr2=0x7fa2b81a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b5d9b700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8103310 0x7fa2b81a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b5d9b700 1 -- 172.21.15.5:0/1282425965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b81a60d0 con 0x7fa2b8107c40 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b659c700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8103310 0x7fa2b81a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2b5d9b700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b81a17a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0098e0 tx=0x7fa2ac0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac027070 con 0x7fa2b8107c40 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac022470 con 0x7fa2b8107c40 2024-05-14T15:29:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b81a6360 con 0x7fa2b8107c40 2024-05-14T15:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.275+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac017650 con 0x7fa2b8107c40 2024-05-14T15:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.276+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b81a6840 con 0x7fa2b8107c40 2024-05-14T15:29:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.277+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b80512c0 con 0x7fa2b8107c40 2024-05-14T15:29:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.279+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa2ac02a070 con 0x7fa2b8107c40 2024-05-14T15:29:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.279+0000 7fa2a77fe700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2a00b29f0 0x7fa2a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.280+0000 7fa2b659c700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2a00b29f0 0x7fa2a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.280+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fa2ac0e3be0 con 0x7fa2b8107c40 2024-05-14T15:29:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.280+0000 7fa2b659c700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2a00b29f0 0x7fa2a00b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800bcd0 tx=0x7fa2a800d040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.283+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2ac013070 con 0x7fa2b8107c40 2024-05-14T15:29:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.570+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2b802e5f0 con 0x7fa2b8107c40 2024-05-14T15:29:25.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.570+0000 7fa2a77fe700 1 -- 172.21.15.5:0/1282425965 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11864 (secure 0 0 0) 0x7fa2ac0ab290 con 0x7fa2b8107c40 2024-05-14T15:29:25.572 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:29:25.572 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:27:33.754380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25691":{"gid":25691,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2401236249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2401236249},{"type":"v1","addr":"172.21.15.144:6835","nonce":2401236249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2a00b29f0 msgr2=0x7fa2a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2a00b29f0 0x7fa2a00b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800bcd0 tx=0x7fa2a800d040 comp rx=0 tx=0).stop 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 msgr2=0x7fa2b81a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b81a17a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0098e0 tx=0x7fa2ac0047b0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 shutdown_connections 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa2a00b29f0 0x7fa2a00b4f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2b8108590 0x7fa2b81a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b8103310 0x7fa2b81a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 --2- 172.21.15.5:0/1282425965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b8107c40 0x7fa2b81a17a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 >> 172.21.15.5:0/1282425965 conn(0x7fa2b8078af0 msgr2=0x7fa2b8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:25.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 shutdown_connections 2024-05-14T15:29:25.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:25.573+0000 7fa2bce59700 1 -- 172.21.15.5:0/1282425965 wait complete. 2024-05-14T15:29:25.576 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-05-14T15:29:26.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 376777 11.66 MB/sec execute 47 sec latency 91.445 ms 2024-05-14T15:29:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:25 smithi144 ceph-mon[35650]: pgmap v142: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 398 op/s 2024-05-14T15:29:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:25 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1282425965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:29:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:25 smithi193 ceph-mon[36001]: pgmap v142: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 398 op/s 2024-05-14T15:29:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:25 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1282425965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:29:26.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:25 smithi005 ceph-mon[31559]: pgmap v142: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 398 op/s 2024-05-14T15:29:26.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:25 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1282425965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:29:26.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:29:26.902 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:27.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 379697 11.77 MB/sec execute 48 sec latency 19.141 ms 2024-05-14T15:29:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:27 smithi005 ceph-mon[31559]: pgmap v143: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-05-14T15:29:28.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381765 11.68 MB/sec execute 49 sec latency 49.516 ms 2024-05-14T15:29:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:27 smithi144 ceph-mon[35650]: pgmap v143: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-05-14T15:29:28.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:27 smithi193 ceph-mon[36001]: pgmap v143: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-05-14T15:29:29.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 383700 11.71 MB/sec execute 50 sec latency 36.864 ms 2024-05-14T15:29:30.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 386138 11.73 MB/sec execute 51 sec latency 106.433 ms 2024-05-14T15:29:30.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:29 smithi005 ceph-mon[31559]: pgmap v144: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 397 op/s 2024-05-14T15:29:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 -- 172.21.15.5:0/999492074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a881034c0 msgr2=0x7f2a881038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 --2- 172.21.15.5:0/999492074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a881034c0 0x7f2a881038d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009920 tx=0x7f2a84009c30 comp rx=0 tx=0).stop 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 -- 172.21.15.5:0/999492074 shutdown_connections 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 --2- 172.21.15.5:0/999492074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a88108740 0x7f2a881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 --2- 172.21.15.5:0/999492074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a881034c0 0x7f2a881038d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 --2- 172.21.15.5:0/999492074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88107df0 0x7f2a88108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 -- 172.21.15.5:0/999492074 >> 172.21.15.5:0/999492074 conn(0x7f2a880fedc0 msgr2=0x7f2a881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 -- 172.21.15.5:0/999492074 shutdown_connections 2024-05-14T15:29:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.200+0000 7f2a8f569700 1 -- 172.21.15.5:0/999492074 wait complete. 2024-05-14T15:29:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 Processor -- start 2024-05-14T15:29:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 -- start start 2024-05-14T15:29:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 0x7f2a8819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 0x7f2a8819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88108740 0x7f2a8819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88113370 con 0x7f2a88108740 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.201+0000 7f2a8f569700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881131f0 con 0x7f2a881034c0 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8f569700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881134f0 con 0x7f2a88107df0 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 0x7f2a8819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 0x7f2a8819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 0x7f2a8819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47382/0 (socket says 172.21.15.5:47382) 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 0x7f2a8819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33414/0 (socket says 172.21.15.5:33414) 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8d305700 1 -- 172.21.15.5:0/2206920594 learned_addr learned my addr 172.21.15.5:0/2206920594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8db06700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88108740 0x7f2a8819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 -- 172.21.15.5:0/2206920594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 msgr2=0x7f2a8819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 0x7f2a8819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 -- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88108740 msgr2=0x7f2a8819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88108740 0x7f2a8819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 -- 172.21.15.5:0/2206920594 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8819fd80 con 0x7f2a88107df0 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8d305700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 0x7f2a8819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.202+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 0x7f2a8819b450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9c0 tx=0x7f2a7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.203+0000 7f2a7e7fc700 1 -- 172.21.15.5:0/2206920594 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800cb70 con 0x7f2a88107df0 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.203+0000 7f2a7e7fc700 1 -- 172.21.15.5:0/2206920594 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7800ee60 con 0x7f2a88107df0 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.203+0000 7f2a7e7fc700 1 -- 172.21.15.5:0/2206920594 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78003d60 con 0x7f2a88107df0 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.203+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a881a0070 con 0x7f2a88107df0 2024-05-14T15:29:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.203+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881a7990 con 0x7f2a88107df0 2024-05-14T15:29:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.206+0000 7f2a7e7fc700 1 -- 172.21.15.5:0/2206920594 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2a78003f00 con 0x7f2a88107df0 2024-05-14T15:29:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.207+0000 7f2a7e7fc700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2a740b2a70 0x7f2a740b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.207+0000 7f2a8d305700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2a740b2a70 0x7f2a740b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.207+0000 7f2a7e7fc700 1 -- 172.21.15.5:0/2206920594 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f2a780e2a50 con 0x7f2a88107df0 2024-05-14T15:29:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.207+0000 7f2a8d305700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2a740b2a70 0x7f2a740b4f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009680 tx=0x7f2a84019040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 Processor -- start 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 -- start start 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 0x55e4781c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4780e6860 con 0x55e4781c4c60 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e4780857c0 con 0x55e4781c9470 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4780e2820 con 0x55e478046e40 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a8cb04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 0x55e4781c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a8cb04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 0x55e4781c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60798/0 (socket says 172.21.15.5:60798) 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a8cb04700 1 -- 172.21.15.5:0/1843524144 learned_addr learned my addr 172.21.15.5:0/1843524144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a8db06700 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.208+0000 7f2a8d305700 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8cb04700 1 -- 172.21.15.5:0/1843524144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 msgr2=0x55e4781c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8cb04700 1 -- 172.21.15.5:0/1843524144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 msgr2=0x55e4781c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8cb04700 1 -- 172.21.15.5:0/1843524144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4780338d0 con 0x55e4781c4c60 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8db06700 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 0x55e4781c8f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2a780dfe40 tx=0x7f2a780a9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fc7bcfd658ad70 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a737fe700 1 -- 172.21.15.5:0/1843524144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780a9970 con 0x55e4781c4c60 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a737fe700 1 -- 172.21.15.5:0/1843524144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a780add20 con 0x55e4781c4c60 2024-05-14T15:29:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.209+0000 7f2a737fe700 1 -- 172.21.15.5:0/1843524144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a780a9e30 con 0x55e4781c4c60 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 -- 172.21.15.5:0/1843524144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 msgr2=0x55e4781c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 0x55e4781c8f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2a780dfe40 tx=0x7f2a780a9500 comp rx=0 tx=0).stop 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 -- 172.21.15.5:0/1843524144 shutdown_connections 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/1843524144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c4c60 0x55e4781c8f30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 -- 172.21.15.5:0/1843524144 >> 172.21.15.5:0/1843524144 conn(0x55e47808e600 msgr2=0x55e4781bf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 -- 172.21.15.5:0/1843524144 shutdown_connections 2024-05-14T15:29:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.210+0000 7f2a9df7db80 1 -- 172.21.15.5:0/1843524144 wait complete. 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 -- start start 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 Processor -- start 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4781d1420 con 0x55e478046e40 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e47804cc30 con 0x55e4781c9470 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.211+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4781d15a0 con 0x55e4781c4c60 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a9df7db80 10 client.0 _ll_register_callbacks cb 0x55e47807e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60814/0 (socket says 172.21.15.5:60814) 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8d305700 1 -- 172.21.15.5:0/575119531 learned_addr learned my addr 172.21.15.5:0/575119531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 -- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 msgr2=0x55e4781ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 -- 172.21.15.5:0/575119531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 msgr2=0x55e4781ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 -- 172.21.15.5:0/575119531 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e478082690 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a8db06700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781cecf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009cd0 tx=0x7f2a80007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3485160048bb56d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.212+0000 7f2a70ff9700 1 -- 172.21.15.5:0/575119531 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80014d20 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a780add20 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a70ff9700 1 -- 172.21.15.5:0/575119531 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f2a80020400 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a70ff9700 1 -- 172.21.15.5:0/575119531 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a80028050 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 10 client.26084 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4781dc150 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a70ff9700 1 -- 172.21.15.5:0/575119531 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a780add20 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a70ff9700 1 -- 172.21.15.5:0/575119531 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2a800166a0 con 0x55e4781c9470 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 10 client.26084 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 10 client.26084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:29:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 20 client.26084 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4781de6e0 0x55e4781e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 4 client.26084 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.213+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4780f4240 con 0x55e4781de6e0 2024-05-14T15:29:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.214+0000 7f2a8d305700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4781de6e0 0x55e4781e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.214+0000 7f2a8d305700 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4781de6e0 0x55e4781e0b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c56a46cf72f80178 server_cookie=8d6cfda4274e9a04 in_seq=0 out_seq=0 2024-05-14T15:29:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.214+0000 7f2a70ff9700 10 client.26084 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:29:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a70ff9700 1 -- 172.21.15.5:0/575119531 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4780f4240 con 0x55e4781de6e0 2024-05-14T15:29:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a70ff9700 10 client.26084 handle_command_reply: tid=0 2024-05-14T15:29:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 client.26084 shutdown 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 msgr2=0x55e4781cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781cecf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009cd0 tx=0x7f2a80007e40 comp rx=0 tx=0).stop 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 shutdown_connections 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4781de6e0 0x55e4781e0b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781cecf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/575119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 >> 172.21.15.5:0/575119531 conn(0x55e47808e600 msgr2=0x55e4781c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a70ff9700 0 client.26084 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 shutdown_connections 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.215+0000 7f2a9df7db80 1 -- 172.21.15.5:0/575119531 wait complete. 2024-05-14T15:29:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.216+0000 7f2a9df7db80 20 client.26084 trim_cache size 0 max 16384 2024-05-14T15:29:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.232+0000 7f2a9df7db80 1 Processor -- start 2024-05-14T15:29:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 -- start start 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c4c60 0x55e4781b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c9470 0x55e4781c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b3030 con 0x55e4781c9470 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b08c0 con 0x55e4781c4c60 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881ae710 con 0x55e478046e40 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33446/0 (socket says 172.21.15.5:33446) 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 -- 172.21.15.5:0/499663930 learned_addr learned my addr 172.21.15.5:0/499663930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c4c60 0x55e4781b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8db06700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c9470 0x55e4781c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 -- 172.21.15.5:0/499663930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c4c60 msgr2=0x55e4781b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c4c60 0x55e4781b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 -- 172.21.15.5:0/499663930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c9470 msgr2=0x55e4781c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.233+0000 7f2a8d305700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c9470 0x55e4781c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a8d305700 1 -- 172.21.15.5:0/499663930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4781c0bd0 con 0x55e478046e40 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c4c60 0x55e4781b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a8db06700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c9470 0x55e4781c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a8d305700 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781bf610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a8401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8d1ce72096ec83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a72ffd700 1 -- 172.21.15.5:0/499663930 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400f850 con 0x55e478046e40 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a72ffd700 1 -- 172.21.15.5:0/499663930 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a84005420 con 0x55e478046e40 2024-05-14T15:29:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.234+0000 7f2a72ffd700 1 -- 172.21.15.5:0/499663930 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400b560 con 0x55e478046e40 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 -- 172.21.15.5:0/499663930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 msgr2=0x55e4781bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781bf610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a8401f000 comp rx=0 tx=0).stop 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 -- 172.21.15.5:0/499663930 shutdown_connections 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4781c9470 0x55e4781c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e478046e40 0x55e4781bf610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/499663930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c4c60 0x55e4781b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 -- 172.21.15.5:0/499663930 >> 172.21.15.5:0/499663930 conn(0x55e47808e600 msgr2=0x55e4781c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 -- 172.21.15.5:0/499663930 shutdown_connections 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.235+0000 7f2a9df7db80 1 -- 172.21.15.5:0/499663930 wait complete. 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 -- start start 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 Processor -- start 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881aed80 con 0x55e478046e40 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b01f0 con 0x55e4781c9470 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881af450 con 0x55e4781c4c60 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a9df7db80 10 client.0 _ll_register_callbacks cb 0x55e47807e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a8cb04700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a8d305700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60822/0 (socket says 172.21.15.5:60822) 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a8cb04700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33458/0 (socket says 172.21.15.5:33458) 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a8d305700 1 -- 172.21.15.5:0/2182785349 learned_addr learned my addr 172.21.15.5:0/2182785349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.236+0000 7f2a8db06700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8d305700 1 -- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 msgr2=0x55e4781d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8d305700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8d305700 1 -- 172.21.15.5:0/2182785349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 msgr2=0x55e4781d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8d305700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8d305700 1 -- 172.21.15.5:0/2182785349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4782470a0 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8db06700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a8d305700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781d17f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2a840098f0 tx=0x7f2a8400f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35bbb10c9408b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a70ff9700 1 -- 172.21.15.5:0/2182785349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84041060 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a70ff9700 1 -- 172.21.15.5:0/2182785349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f2a84038040 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a84005420 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a70ff9700 1 -- 172.21.15.5:0/2182785349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a84048050 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a9df7db80 10 client.16464 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.237+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e478248300 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a70ff9700 1 -- 172.21.15.5:0/2182785349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a84005420 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a70ff9700 1 -- 172.21.15.5:0/2182785349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2a8400bcc0 con 0x55e478046e40 2024-05-14T15:29:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a9df7db80 10 client.16464 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a9df7db80 10 client.16464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a9df7db80 20 client.16464 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4782488a0 0x55e47824acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a9df7db80 4 client.16464 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4780ee2e0 con 0x55e4782488a0 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.238+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4782488a0 0x55e47824acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a70ff9700 10 client.16464 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a8cb04700 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4782488a0 0x55e47824acc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b424f5977c5e4a44 server_cookie=e968bbff907dd7e9 in_seq=0 out_seq=0 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a70ff9700 1 -- 172.21.15.5:0/2182785349 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4780ee2e0 con 0x55e4782488a0 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a70ff9700 10 client.16464 handle_command_reply: tid=0 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 client.16464 shutdown 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 msgr2=0x55e4781d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781d17f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2a840098f0 tx=0x7f2a8400f960 comp rx=0 tx=0).stop 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 shutdown_connections 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55e4782488a0 0x55e47824acc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e4781c9470 0x55e4781d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e478046e40 0x55e4781d17f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a70ff9700 0 client.16464 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 --2- 172.21.15.5:0/2182785349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4781c4c60 0x55e4781d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 >> 172.21.15.5:0/2182785349 conn(0x55e47808e600 msgr2=0x55e4781bf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 shutdown_connections 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 1 -- 172.21.15.5:0/2182785349 wait complete. 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.239+0000 7f2a9df7db80 20 client.16464 trim_cache size 0 max 16384 2024-05-14T15:29:30.245 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:29:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2a740b2a70 msgr2=0x7f2a740b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2a740b2a70 0x7f2a740b4f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009680 tx=0x7f2a84019040 comp rx=0 tx=0).stop 2024-05-14T15:29:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 msgr2=0x7f2a8819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 0x7f2a8819b450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9c0 tx=0x7f2a7800ecd0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 shutdown_connections 2024-05-14T15:29:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2a740b2a70 0x7f2a740b4f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a88108740 0x7f2a8819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a881034c0 0x7f2a8819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 --2- 172.21.15.5:0/2206920594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a88107df0 0x7f2a8819b450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 >> 172.21.15.5:0/2206920594 conn(0x7f2a880fedc0 msgr2=0x7f2a8806ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 shutdown_connections 2024-05-14T15:29:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:30.240+0000 7f2a8f569700 1 -- 172.21.15.5:0/2206920594 wait complete. 2024-05-14T15:29:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:29 smithi144 ceph-mon[35650]: pgmap v144: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 397 op/s 2024-05-14T15:29:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:29 smithi193 ceph-mon[36001]: pgmap v144: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 397 op/s 2024-05-14T15:29:31.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 387489 11.64 MB/sec execute 52 sec latency 40.357 ms 2024-05-14T15:29:31.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:29:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:32.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 390447 11.77 MB/sec execute 53 sec latency 14.671 ms 2024-05-14T15:29:32.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:31 smithi005 ceph-mon[31559]: pgmap v145: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 397 op/s 2024-05-14T15:29:32.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:31 smithi144 ceph-mon[35650]: pgmap v145: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 397 op/s 2024-05-14T15:29:32.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:31 smithi193 ceph-mon[36001]: pgmap v145: 129 pgs: 129 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 397 op/s 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 -- 172.21.15.5:0/4045203895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8141021e0 msgr2=0x7fe8141025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/4045203895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8141021e0 0x7fe8141025f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe804009900 tx=0x7fe80402a410 comp rx=0 tx=0).stop 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 -- 172.21.15.5:0/4045203895 shutdown_connections 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/4045203895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81410e6e0 0x7fe814110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/4045203895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8141021e0 0x7fe8141025f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/4045203895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe814109d30 0x7fe814101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 -- 172.21.15.5:0/4045203895 >> 172.21.15.5:0/4045203895 conn(0x7fe8140fd9d0 msgr2=0x7fe8140ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 -- 172.21.15.5:0/4045203895 shutdown_connections 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.807+0000 7fe81c2cd640 1 -- 172.21.15.5:0/4045203895 wait complete. 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 Processor -- start 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 -- start start 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8141021e0 0x7fe8141a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 0x7fe8141a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81410e6e0 0x7fe81419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe814113ef0 con 0x7fe814109d30 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe814113d70 con 0x7fe81410e6e0 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81c2cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe814114070 con 0x7fe8141021e0 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 0x7fe8141a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81a069640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8141021e0 0x7fe8141a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 0x7fe8141a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60846/0 (socket says 172.21.15.5:60846) 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 -- 172.21.15.5:0/342013388 learned_addr learned my addr 172.21.15.5:0/342013388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe81a86a640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81410e6e0 0x7fe81419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 -- 172.21.15.5:0/342013388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8141021e0 msgr2=0x7fe8141a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8141021e0 0x7fe8141a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 -- 172.21.15.5:0/342013388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81410e6e0 msgr2=0x7fe81419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81410e6e0 0x7fe81419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.808+0000 7fe819868640 1 -- 172.21.15.5:0/342013388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81419ae20 con 0x7fe814109d30 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe81a069640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8141021e0 0x7fe8141a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe819868640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 0x7fe8141a0ab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe804005ab0 tx=0x7fe804003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe80b7fe640 1 -- 172.21.15.5:0/342013388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80403f070 con 0x7fe814109d30 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe80b7fe640 1 -- 172.21.15.5:0/342013388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe804031b50 con 0x7fe814109d30 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81419b050 con 0x7fe814109d30 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe80b7fe640 1 -- 172.21.15.5:0/342013388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804035030 con 0x7fe814109d30 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.809+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81419b540 con 0x7fe814109d30 2024-05-14T15:29:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.812+0000 7fe823178740 1 Processor -- start 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 -- start start 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe80b7fe640 1 -- 172.21.15.5:0/342013388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe804042070 con 0x7fe814109d30 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad52ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad52f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad52f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad4945d0 con 0x55f1ad16d4d0 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad44d890 con 0x55f1ad18e1b0 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe823178740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad490590 con 0x55f1ad1652f0 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe819868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad52f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe81a069640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad52f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe81a86a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad52ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe819868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad52f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60852/0 (socket says 172.21.15.5:60852) 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.813+0000 7fe819868640 1 -- 172.21.15.5:0/3824282228 learned_addr learned my addr 172.21.15.5:0/3824282228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a069640 1 -- 172.21.15.5:0/3824282228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 msgr2=0x55f1ad52ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a069640 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad52ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a069640 1 -- 172.21.15.5:0/3824282228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 msgr2=0x55f1ad52f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a069640 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad52f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a069640 1 -- 172.21.15.5:0/3824282228 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ad1a6210 con 0x55f1ad1652f0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a069640 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad52f120 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe81406b5f0 tx=0x7fe8100118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b950c6cfe732584 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe80b7fe640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe8000b2950 0x7fe8000b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe808bfe640 1 -- 172.21.15.5:0/3824282228 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81001e070 con 0x55f1ad1652f0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe808bfe640 1 -- 172.21.15.5:0/3824282228 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8100035f0 con 0x55f1ad1652f0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe808bfe640 1 -- 172.21.15.5:0/3824282228 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000f740 con 0x55f1ad1652f0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe81a86a640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe8000b2950 0x7fe8000b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.814+0000 7fe80b7fe640 1 -- 172.21.15.5:0/342013388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fe804102fb0 con 0x7fe814109d30 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 -- 172.21.15.5:0/3824282228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 msgr2=0x55f1ad52f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe81a86a640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe8000b2950 0x7fe8000b4e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55f1ad52bd00 tx=0x7fe80c009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad52f120 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe81406b5f0 tx=0x7fe8100118d0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 -- 172.21.15.5:0/3824282228 shutdown_connections 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad52f120 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad52f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 --2- 172.21.15.5:0/3824282228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad52ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 -- 172.21.15.5:0/3824282228 >> 172.21.15.5:0/3824282228 conn(0x55f1ad182300 msgr2=0x55f1ad52b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 -- 172.21.15.5:0/3824282228 shutdown_connections 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.815+0000 7fe823178740 1 -- 172.21.15.5:0/3824282228 wait complete. 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 -- start start 2024-05-14T15:29:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 Processor -- start 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad539200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad539740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad5451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad53bd10 con 0x55f1ad16d4d0 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad451370 con 0x55f1ad18e1b0 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f1ad53be90 con 0x55f1ad1652f0 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe823178740 10 client.0 _ll_register_callbacks cb 0x55f1ad1a5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe819868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad539740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe81a86a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad5451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe819868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad539740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60864/0 (socket says 172.21.15.5:60864) 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe819868640 1 -- 172.21.15.5:0/2212462719 learned_addr learned my addr 172.21.15.5:0/2212462719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.816+0000 7fe81a069640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad539200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe81a86a640 1 -- 172.21.15.5:0/2212462719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 msgr2=0x55f1ad539200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe81a86a640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad539200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe81a86a640 1 -- 172.21.15.5:0/2212462719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 msgr2=0x55f1ad539740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe81a86a640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad539740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe81a86a640 1 -- 172.21.15.5:0/2212462719 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ad272620 con 0x55f1ad18e1b0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe819868640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad539740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe81a86a640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad5451f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe814102c80 tx=0x7fe80c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a247b73d9b8ba0f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe7fdffb640 1 -- 172.21.15.5:0/2212462719 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00f870 con 0x55f1ad18e1b0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8100035f0 con 0x55f1ad18e1b0 2024-05-14T15:29:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe7fdffb640 1 -- 172.21.15.5:0/2212462719 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fe80c00fa10 con 0x55f1ad18e1b0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe7fdffb640 1 -- 172.21.15.5:0/2212462719 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe80c00fc10 con 0x55f1ad18e1b0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe823178740 10 client.26102 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1ad548160 con 0x55f1ad18e1b0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.817+0000 7fe7fdffb640 1 -- 172.21.15.5:0/2212462719 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8100035f0 con 0x55f1ad18e1b0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe7fdffb640 1 -- 172.21.15.5:0/2212462719 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe80c026070 con 0x55f1ad18e1b0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe823178740 10 client.26102 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe823178740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe823178740 20 client.26102 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe823178740 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad14cd80 0x55f1ad54b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe823178740 4 client.26102 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1ad4a21e0 con 0x55f1ad14cd80 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe81a069640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad14cd80 0x55f1ad54b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe81a069640 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad14cd80 0x55f1ad54b630 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b4e3096478d6a1 server_cookie=57f1d0fba9d8bda7 in_seq=0 out_seq=0 2024-05-14T15:29:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.818+0000 7fe7fdffb640 10 client.26102 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe7fdffb640 1 -- 172.21.15.5:0/2212462719 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1ad4a21e0 con 0x55f1ad14cd80 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe7fdffb640 10 client.26102 handle_command_reply: tid=0 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 client.26102 shutdown 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 msgr2=0x55f1ad5451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad5451f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe814102c80 tx=0x7fe80c004ab0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 shutdown_connections 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad14cd80 0x55f1ad54b630 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad18e1b0 0x55f1ad5451f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe7fdffb640 0 client.26102 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad539740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 --2- 172.21.15.5:0/2212462719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad539200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 >> 172.21.15.5:0/2212462719 conn(0x55f1ad182300 msgr2=0x55f1ad52ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 shutdown_connections 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 1 -- 172.21.15.5:0/2212462719 wait complete. 2024-05-14T15:29:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.819+0000 7fe823178740 20 client.26102 trim_cache size 0 max 16384 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.835+0000 7fe823178740 1 Processor -- start 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.835+0000 7fe823178740 1 -- start start 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe823178740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 0x55f1ad538c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe823178740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad550e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe823178740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad16d4d0 0x55f1ad551350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe823178740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b3bc0 con 0x55f1ad14cd80 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe823178740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b1490 con 0x55f1ad16d4d0 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe823178740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141af2e0 con 0x55f1ad1652f0 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 0x55f1ad538c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 0x55f1ad538c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60868/0 (socket says 172.21.15.5:60868) 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 -- 172.21.15.5:0/120301510 learned_addr learned my addr 172.21.15.5:0/120301510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a86a640 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad16d4d0 0x55f1ad551350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe819868640 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad550e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 -- 172.21.15.5:0/120301510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 msgr2=0x55f1ad550e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad550e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 -- 172.21.15.5:0/120301510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad16d4d0 msgr2=0x55f1ad551350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad16d4d0 0x55f1ad551350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 -- 172.21.15.5:0/120301510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ad52fdc0 con 0x55f1ad14cd80 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe81a069640 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 0x55f1ad538c40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe810009250 tx=0x7fe8100037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc881f5967674f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.836+0000 7fe7fffff640 1 -- 172.21.15.5:0/120301510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81001e040 con 0x55f1ad14cd80 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe7fffff640 1 -- 172.21.15.5:0/120301510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8100050b0 con 0x55f1ad14cd80 2024-05-14T15:29:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe7fffff640 1 -- 172.21.15.5:0/120301510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81002a5b0 con 0x55f1ad14cd80 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 -- 172.21.15.5:0/120301510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 msgr2=0x55f1ad538c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 0x55f1ad538c40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe810009250 tx=0x7fe8100037d0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 -- 172.21.15.5:0/120301510 shutdown_connections 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad1652f0 0x55f1ad550e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad16d4d0 0x55f1ad551350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 --2- 172.21.15.5:0/120301510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad14cd80 0x55f1ad538c40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 -- 172.21.15.5:0/120301510 >> 172.21.15.5:0/120301510 conn(0x55f1ad182300 msgr2=0x55f1ad538530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 -- 172.21.15.5:0/120301510 shutdown_connections 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.837+0000 7fe823178740 1 -- 172.21.15.5:0/120301510 wait complete. 2024-05-14T15:29:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 -- start start 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 Processor -- start 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad14cd80 0x55f1ad53c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad1652f0 0x55f1ad53c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad53cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141af950 con 0x55f1ad16d4d0 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b0dc0 con 0x55f1ad1652f0 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b0020 con 0x55f1ad14cd80 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe823178740 10 client.0 _ll_register_callbacks cb 0x7fe80c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe81a86a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad53cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe81a069640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad14cd80 0x55f1ad53c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe81a86a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad53cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60882/0 (socket says 172.21.15.5:60882) 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe81a86a640 1 -- 172.21.15.5:0/1434617423 learned_addr learned my addr 172.21.15.5:0/1434617423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.838+0000 7fe819868640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad1652f0 0x55f1ad53c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a069640 1 -- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad1652f0 msgr2=0x55f1ad53c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a069640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad1652f0 0x55f1ad53c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a069640 1 -- 172.21.15.5:0/1434617423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 msgr2=0x55f1ad53cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a069640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad53cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a069640 1 -- 172.21.15.5:0/1434617423 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ad5b1b60 con 0x55f1ad14cd80 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a86a640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad53cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe81a069640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad14cd80 0x55f1ad53c110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe810009ef0 tx=0x7fe8100035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78f90fd86b46b41 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe7fdffb640 1 -- 172.21.15.5:0/1434617423 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810038020 con 0x55f1ad14cd80 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe7fdffb640 1 -- 172.21.15.5:0/1434617423 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fe810005320 con 0x55f1ad14cd80 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8100050b0 con 0x55f1ad14cd80 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe7fdffb640 1 -- 172.21.15.5:0/1434617423 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe81001e040 con 0x55f1ad14cd80 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe823178740 10 client.26092 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1ad5b2df0 con 0x55f1ad14cd80 2024-05-14T15:29:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.839+0000 7fe7fdffb640 1 -- 172.21.15.5:0/1434617423 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8100050b0 con 0x55f1ad14cd80 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe7fdffb640 1 -- 172.21.15.5:0/1434617423 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe81002a890 con 0x55f1ad14cd80 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe823178740 10 client.26092 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe823178740 10 client.26092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe823178740 20 client.26092 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe823178740 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad18e1b0 0x55f1ad5b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe823178740 4 client.26092 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1ad49c050 con 0x55f1ad18e1b0 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe819868640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad18e1b0 0x55f1ad5b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe819868640 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad18e1b0 0x55f1ad5b5320 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0067cef73f4b03 server_cookie=a30e61ad7e26b919 in_seq=0 out_seq=0 2024-05-14T15:29:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:32.840+0000 7fe7fdffb640 10 client.26092 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.128+0000 7fe7fdffb640 1 -- 172.21.15.5:0/1434617423 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1ad49c050 con 0x55f1ad18e1b0 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.128+0000 7fe7fdffb640 10 client.26092 handle_command_reply: tid=0 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.128+0000 7fe823178740 1 client.26092 shutdown 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad14cd80 msgr2=0x55f1ad53c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad14cd80 0x55f1ad53c110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe810009ef0 tx=0x7fe8100035f0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 shutdown_connections 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f1ad18e1b0 0x55f1ad5b5320 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1ad16d4d0 0x55f1ad53cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f1ad1652f0 0x55f1ad53c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 --2- 172.21.15.5:0/1434617423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1ad14cd80 0x55f1ad53c110 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe810009ef0 tx=0x7fe8100035f0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 >> 172.21.15.5:0/1434617423 conn(0x55f1ad182300 msgr2=0x55f1ad52e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe7fdffb640 0 client.26092 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 shutdown_connections 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 1 -- 172.21.15.5:0/1434617423 wait complete. 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.129+0000 7fe823178740 20 client.26092 trim_cache size 0 max 16384 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:29:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:29:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe8000b2950 msgr2=0x7fe8000b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe8000b2950 0x7fe8000b4e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55f1ad52bd00 tx=0x7fe80c009500 comp rx=0 tx=0).stop 2024-05-14T15:29:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 msgr2=0x7fe8141a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 0x7fe8141a0ab0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe804005ab0 tx=0x7fe804003b90 comp rx=0 tx=0).stop 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 shutdown_connections 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe8000b2950 0x7fe8000b4e60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81410e6e0 0x7fe81419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe814109d30 0x7fe8141a0ab0 secure :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe804005ab0 tx=0x7fe804003b90 comp rx=0 tx=0).stop 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 --2- 172.21.15.5:0/342013388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8141021e0 0x7fe8141a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 >> 172.21.15.5:0/342013388 conn(0x7fe8140fd9d0 msgr2=0x7fe8140fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.132+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 shutdown_connections 2024-05-14T15:29:33.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 391778 11.65 MB/sec execute 54 sec latency 84.762 ms 2024-05-14T15:29:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:33.133+0000 7fe81c2cd640 1 -- 172.21.15.5:0/342013388 wait complete. 2024-05-14T15:29:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.637+0000 7fb2ca385700 1 -- 172.21.15.5:0/3795762658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41034e0 msgr2=0x7fb2c41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.637+0000 7fb2ca385700 1 --2- 172.21.15.5:0/3795762658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41034e0 0x7fb2c41038f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40099c0 tx=0x7fb2b4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 -- 172.21.15.5:0/3795762658 shutdown_connections 2024-05-14T15:29:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 --2- 172.21.15.5:0/3795762658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c4108760 0x7fb2c41100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 --2- 172.21.15.5:0/3795762658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41034e0 0x7fb2c41038f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 --2- 172.21.15.5:0/3795762658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c4107e10 0x7fb2c4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 -- 172.21.15.5:0/3795762658 >> 172.21.15.5:0/3795762658 conn(0x7fb2c40fef30 msgr2=0x7fb2c4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 -- 172.21.15.5:0/3795762658 shutdown_connections 2024-05-14T15:29:33.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 -- 172.21.15.5:0/3795762658 wait complete. 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.638+0000 7fb2ca385700 1 Processor -- start 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 -- start start 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 0x7fb2c419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4107e10 0x7fb2c419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c4108760 0x7fb2c419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4113390 con 0x7fb2c4107e10 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4113210 con 0x7fb2c41034e0 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2ca385700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4113510 con 0x7fb2c4108760 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4107e10 0x7fb2c419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 0x7fb2c419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4107e10 0x7fb2c419c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60888/0 (socket says 172.21.15.5:60888) 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 0x7fb2c419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47502/0 (socket says 172.21.15.5:47502) 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2bbfff700 1 -- 172.21.15.5:0/2472922834 learned_addr learned my addr 172.21.15.5:0/2472922834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c8922700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c4108760 0x7fb2c419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c3fff700 1 -- 172.21.15.5:0/2472922834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c4108760 msgr2=0x7fb2c419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c4108760 0x7fb2c419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c3fff700 1 -- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4107e10 msgr2=0x7fb2c419c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.639+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4107e10 0x7fb2c419c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2c3fff700 1 -- 172.21.15.5:0/2472922834 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c41a7000 con 0x7fb2c41034e0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 0x7fb2c419bc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40098e0 tx=0x7fb2b4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2c1ffb700 1 -- 172.21.15.5:0/2472922834 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4026070 con 0x7fb2c41034e0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c41a7290 con 0x7fb2c41034e0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2c1ffb700 1 -- 172.21.15.5:0/2472922834 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b4003c70 con 0x7fb2c41034e0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2c1ffb700 1 -- 172.21.15.5:0/2472922834 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400f9c0 con 0x7fb2c41034e0 2024-05-14T15:29:33.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.640+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a76e0 con 0x7fb2c41034e0 2024-05-14T15:29:33.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.644+0000 7fb2c1ffb700 1 -- 172.21.15.5:0/2472922834 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb2b4029070 con 0x7fb2c41034e0 2024-05-14T15:29:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.644+0000 7fb2c1ffb700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb2b00b29f0 0x7fb2b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.644+0000 7fb2d8d99b80 1 Processor -- start 2024-05-14T15:29:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 -- start start 2024-05-14T15:29:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb2b00b29f0 0x7fb2b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c1ffb700 1 -- 172.21.15.5:0/2472922834 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb2b40e3140 con 0x7fb2c41034e0 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d3b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d36220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556496c57940 con 0x556496d35e10 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556496bbcda0 con 0x556496bb7e40 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556496c53900 con 0x556496d3bd80 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d3b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d3b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47516/0 (socket says 172.21.15.5:47516) 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 -- 172.21.15.5:0/628870895 learned_addr learned my addr 172.21.15.5:0/628870895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c8922700 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d36220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb2b00b29f0 0x7fb2b00b4f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009680 tx=0x7fb2ac009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 -- 172.21.15.5:0/628870895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 msgr2=0x556496d358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 -- 172.21.15.5:0/628870895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 msgr2=0x556496d36220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d36220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.645+0000 7fb2c3fff700 1 -- 172.21.15.5:0/628870895 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556496ba48d0 con 0x556496bb7e40 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.646+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d3b840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40a9df0 tx=0x7fb2b4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15fc611600a2aa90 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.646+0000 7fb2ba7fc700 1 -- 172.21.15.5:0/628870895 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4003e20 con 0x556496bb7e40 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.646+0000 7fb2ba7fc700 1 -- 172.21.15.5:0/628870895 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b40aa7f0 con 0x556496bb7e40 2024-05-14T15:29:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.646+0000 7fb2ba7fc700 1 -- 172.21.15.5:0/628870895 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40b2c90 con 0x556496bb7e40 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/628870895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 msgr2=0x556496d3b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d3b840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40a9df0 tx=0x7fb2b4009310 comp rx=0 tx=0).stop 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/628870895 shutdown_connections 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d36220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d3b840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/628870895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/628870895 >> 172.21.15.5:0/628870895 conn(0x556496bff630 msgr2=0x556496d30a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/628870895 shutdown_connections 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/628870895 wait complete. 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.647+0000 7fb2d8d99b80 1 -- start start 2024-05-14T15:29:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 Processor -- start 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d3f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d40000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556496d425e0 con 0x556496bb7e40 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556496c10ed0 con 0x556496d3bd80 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556496d42760 con 0x556496d35e10 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2d8d99b80 10 client.0 _ll_register_callbacks cb 0x556496bef950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d3f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d3f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60914/0 (socket says 172.21.15.5:60914) 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c8922700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 -- 172.21.15.5:0/3130016042 learned_addr learned my addr 172.21.15.5:0/3130016042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 -- 172.21.15.5:0/3130016042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 msgr2=0x556496d3fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 -- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 msgr2=0x556496d40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d40000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c3fff700 1 -- 172.21.15.5:0/3130016042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556496c0e760 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2c8922700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d40000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.648+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d3f580 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40a6170 tx=0x7fb2b40aa6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae45c1c81278fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2a37fe700 1 -- 172.21.15.5:0/3130016042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40c3020 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b40aa7f0 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2a37fe700 1 -- 172.21.15.5:0/3130016042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb2b40b2a30 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2a37fe700 1 -- 172.21.15.5:0/3130016042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2b4003e20 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2d8d99b80 10 client.16494 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556496d425e0 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.649+0000 7fb2a37fe700 1 -- 172.21.15.5:0/3130016042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b40aa7f0 con 0x556496bb7e40 2024-05-14T15:29:33.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2a37fe700 1 -- 172.21.15.5:0/3130016042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb2b400fc80 con 0x556496bb7e40 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2d8d99b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2d8d99b80 10 client.16494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2d8d99b80 20 client.16494 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496d4f4f0 0x556496d51910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2d8d99b80 4 client.16494 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556496c65320 con 0x556496d4f4f0 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496d4f4f0 0x556496d51910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496d4f4f0 0x556496d51910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ca622130b775a98 server_cookie=70be665485688dd1 in_seq=0 out_seq=0 2024-05-14T15:29:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.650+0000 7fb2a37fe700 10 client.16494 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:29:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.651+0000 7fb2a37fe700 1 -- 172.21.15.5:0/3130016042 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556496c65320 con 0x556496d4f4f0 2024-05-14T15:29:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.651+0000 7fb2a37fe700 10 client.16494 handle_command_reply: tid=0 2024-05-14T15:29:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.651+0000 7fb2d8d99b80 1 client.16494 shutdown 2024-05-14T15:29:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 msgr2=0x556496d3f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d3f580 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40a6170 tx=0x7fb2b40aa6a0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 shutdown_connections 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496d4f4f0 0x556496d51910 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d40000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d3f580 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/3130016042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 >> 172.21.15.5:0/3130016042 conn(0x556496bff630 msgr2=0x556496d348c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2a37fe700 0 client.16494 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 shutdown_connections 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/3130016042 wait complete. 2024-05-14T15:29:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.652+0000 7fb2d8d99b80 20 client.16494 trim_cache size 0 max 16384 2024-05-14T15:29:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 Processor -- start 2024-05-14T15:29:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 -- start start 2024-05-14T15:29:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d3fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41b2eb0 con 0x556496bb7e40 2024-05-14T15:29:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41b0740 con 0x556496d3bd80 2024-05-14T15:29:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.673+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41ae590 con 0x556496d35e10 2024-05-14T15:29:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.674+0000 7fb2c8922700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d3fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.674+0000 7fb2c8922700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d3fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47540/0 (socket says 172.21.15.5:47540) 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.674+0000 7fb2c8922700 1 -- 172.21.15.5:0/815661555 learned_addr learned my addr 172.21.15.5:0/815661555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.674+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c8922700 1 -- 172.21.15.5:0/815661555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 msgr2=0x556496d3d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c8922700 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c8922700 1 -- 172.21.15.5:0/815661555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 msgr2=0x556496d358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c8922700 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c8922700 1 -- 172.21.15.5:0/815661555 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556496d3e8a0 con 0x556496d3bd80 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:29:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.675+0000 7fb2c8922700 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d3fe40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc011be0 tx=0x7fb2bc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa709d8a652e258e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.676+0000 7fb2b9ffb700 1 -- 172.21.15.5:0/815661555 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0093a0 con 0x556496d3bd80 2024-05-14T15:29:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.676+0000 7fb2b9ffb700 1 -- 172.21.15.5:0/815661555 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc007bb0 con 0x556496d3bd80 2024-05-14T15:29:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2b9ffb700 1 -- 172.21.15.5:0/815661555 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc023c10 con 0x556496d3bd80 2024-05-14T15:29:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/815661555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 msgr2=0x556496d3fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d3fe40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc011be0 tx=0x7fb2bc009920 comp rx=0 tx=0).stop 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/815661555 shutdown_connections 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496d3bd80 0x556496d3fe40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496bb7e40 0x556496d358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/815661555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d35e10 0x556496d3d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.677+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/815661555 >> 172.21.15.5:0/815661555 conn(0x556496bff630 msgr2=0x556496d30270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.678+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/815661555 shutdown_connections 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.678+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/815661555 wait complete. 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.678+0000 7fb2d8d99b80 1 -- start start 2024-05-14T15:29:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.678+0000 7fb2d8d99b80 1 Processor -- start 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d47bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41aec00 con 0x556496d35e10 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41b0070 con 0x556496bb7e40 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41af2d0 con 0x556496d3bd80 2024-05-14T15:29:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.679+0000 7fb2d8d99b80 10 client.0 _ll_register_callbacks cb 0x556496bef950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.680+0000 7fb2bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.680+0000 7fb2bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d56100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60924/0 (socket says 172.21.15.5:60924) 2024-05-14T15:29:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.680+0000 7fb2bbfff700 1 -- 172.21.15.5:0/2818081583 learned_addr learned my addr 172.21.15.5:0/2818081583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.681+0000 7fb2bbfff700 1 -- 172.21.15.5:0/2818081583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 msgr2=0x556496d56640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:29:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.681+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d56640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.681+0000 7fb2bbfff700 1 -- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 msgr2=0x556496d47bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:29:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.681+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d47bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.681+0000 7fb2bbfff700 1 -- 172.21.15.5:0/2818081583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556496d48440 con 0x556496d35e10 2024-05-14T15:29:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.684+0000 7fb2bbfff700 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d56100 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009530 tx=0x7fb2ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9cf4151646f450 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.684+0000 7fb2a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.684+0000 7fb2a37fe700 1 -- 172.21.15.5:0/2818081583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac02ad80 con 0x556496d35e10 2024-05-14T15:29:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.684+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2bc007bb0 con 0x556496d35e10 2024-05-14T15:29:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.685+0000 7fb2a37fe700 1 -- 172.21.15.5:0/2818081583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb2ac035470 con 0x556496d35e10 2024-05-14T15:29:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.687+0000 7fb2a37fe700 1 -- 172.21.15.5:0/2818081583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2ac035670 con 0x556496d35e10 2024-05-14T15:29:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.687+0000 7fb2d8d99b80 10 client.16497 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.687+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556496dc1e70 con 0x556496d35e10 2024-05-14T15:29:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.687+0000 7fb2a37fe700 1 -- 172.21.15.5:0/2818081583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc007bb0 con 0x556496d35e10 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2a37fe700 1 -- 172.21.15.5:0/2818081583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb2ac02c450 con 0x556496d35e10 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2d8d99b80 10 client.16497 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2d8d99b80 10 client.16497 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2d8d99b80 20 client.16497 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496dc2540 0x556496dc4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2d8d99b80 4 client.16497 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:29:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.688+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556496c5f3c0 con 0x556496dc2540 2024-05-14T15:29:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.689+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496dc2540 0x556496dc4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.691+0000 7fb2c3fff700 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496dc2540 0x556496dc4960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d4a2a3f8b106d34 server_cookie=5b88c8f374601560 in_seq=0 out_seq=0 2024-05-14T15:29:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.691+0000 7fb2a37fe700 10 client.16497 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.692+0000 7fb2a37fe700 1 -- 172.21.15.5:0/2818081583 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556496c5f3c0 con 0x556496dc2540 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.692+0000 7fb2a37fe700 10 client.16497 handle_command_reply: tid=0 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.692+0000 7fb2d8d99b80 1 client.16497 shutdown 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 msgr2=0x556496d56100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d56100 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009530 tx=0x7fb2ac0047c0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 shutdown_connections 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556496dc2540 0x556496dc4960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556496d3bd80 0x556496d56640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556496bb7e40 0x556496d47bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 --2- 172.21.15.5:0/2818081583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556496d35e10 0x556496d56100 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 >> 172.21.15.5:0/2818081583 conn(0x556496bff630 msgr2=0x556496d40b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.693+0000 7fb2a37fe700 0 client.16497 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:29:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.694+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 shutdown_connections 2024-05-14T15:29:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.694+0000 7fb2d8d99b80 1 -- 172.21.15.5:0/2818081583 wait complete. 2024-05-14T15:29:33.695 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:29:33.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.694+0000 7fb2d8d99b80 20 client.16497 trim_cache size 0 max 16384 2024-05-14T15:29:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.696+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb2b00b29f0 msgr2=0x7fb2b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.696+0000 7fb2ca385700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb2b00b29f0 0x7fb2b00b4f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009680 tx=0x7fb2ac009260 comp rx=0 tx=0).stop 2024-05-14T15:29:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.696+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 msgr2=0x7fb2c419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.696+0000 7fb2ca385700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 0x7fb2c419bc90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40098e0 tx=0x7fb2b4004270 comp rx=0 tx=0).stop 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.698+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 shutdown_connections 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.698+0000 7fb2ca385700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb2b00b29f0 0x7fb2b00b4f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.698+0000 7fb2ca385700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c4108760 0x7fb2c419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.698+0000 7fb2ca385700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c41034e0 0x7fb2c419bc90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.698+0000 7fb2ca385700 1 --2- 172.21.15.5:0/2472922834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c4107e10 0x7fb2c419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.698+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 >> 172.21.15.5:0/2472922834 conn(0x7fb2c40fef30 msgr2=0x7fb2c41096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.699+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 shutdown_connections 2024-05-14T15:29:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:33.701+0000 7fb2ca385700 1 -- 172.21.15.5:0/2472922834 wait complete. 2024-05-14T15:29:34.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 394190 11.69 MB/sec execute 55 sec latency 39.587 ms 2024-05-14T15:29:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:33 smithi144 ceph-mon[35650]: pgmap v146: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 509 op/s 2024-05-14T15:29:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:33 smithi193 ceph-mon[36001]: pgmap v146: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 509 op/s 2024-05-14T15:29:34.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:33 smithi005 ceph-mon[31559]: pgmap v146: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 509 op/s 2024-05-14T15:29:34.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:29:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:35.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 396743 11.71 MB/sec execute 56 sec latency 19.091 ms 2024-05-14T15:29:36.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 398181 11.63 MB/sec execute 57 sec latency 46.880 ms 2024-05-14T15:29:36.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:35 smithi005 ceph-mon[31559]: pgmap v147: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-05-14T15:29:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:35 smithi144 ceph-mon[35650]: pgmap v147: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-05-14T15:29:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:35 smithi193 ceph-mon[36001]: pgmap v147: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-05-14T15:29:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 -- 172.21.15.5:0/3316184028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108020 msgr2=0x7f35a0108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/3316184028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108020 0x7f35a0108430 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f358c0099c0 tx=0x7f358c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 -- 172.21.15.5:0/3316184028 shutdown_connections 2024-05-14T15:29:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/3316184028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0108970 0x7f35a01102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/3316184028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 0x7f35a0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/3316184028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108020 0x7f35a0108430 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 -- 172.21.15.5:0/3316184028 >> 172.21.15.5:0/3316184028 conn(0x7f35a00fe520 msgr2=0x7f35a0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 -- 172.21.15.5:0/3316184028 shutdown_connections 2024-05-14T15:29:36.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.949+0000 7f35a6ce5700 1 -- 172.21.15.5:0/3316184028 wait complete. 2024-05-14T15:29:36.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.950+0000 7f35a6ce5700 1 Processor -- start 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.950+0000 7f35a6ce5700 1 -- start start 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.950+0000 7f35a6ce5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 0x7f35a019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.950+0000 7f35a6ce5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0108020 0x7f35a019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a6ce5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 0x7f35a019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a6ce5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0113580 con 0x7f35a0108970 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a6ce5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0113400 con 0x7f35a0108020 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a6ce5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35a0113700 con 0x7f35a0103c10 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 0x7f35a019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 0x7f35a019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 0x7f35a019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59160/0 (socket says 172.21.15.5:59160) 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 -- 172.21.15.5:0/1959668599 learned_addr learned my addr 172.21.15.5:0/1959668599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f359ffff700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0108020 0x7f35a019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 -- 172.21.15.5:0/1959668599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 msgr2=0x7f35a019be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 0x7f35a019be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 -- 172.21.15.5:0/1959668599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0108020 msgr2=0x7f35a019c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0108020 0x7f35a019c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 -- 172.21.15.5:0/1959668599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a0199c70 con 0x7f35a0108970 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a4a81700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 0x7f35a019be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f35a5282700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 0x7f35a019c920 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f359800ef30 tx=0x7f359800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f359dffb700 1 -- 172.21.15.5:0/1959668599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980092e0 con 0x7f35a0108970 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.951+0000 7f359dffb700 1 -- 172.21.15.5:0/1959668599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3598004bb0 con 0x7f35a0108970 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.952+0000 7f359dffb700 1 -- 172.21.15.5:0/1959668599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598004020 con 0x7f35a0108970 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.952+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a0199f00 con 0x7f35a0108970 2024-05-14T15:29:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.952+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a019a410 con 0x7f35a0108970 2024-05-14T15:29:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.955+0000 7f359dffb700 1 -- 172.21.15.5:0/1959668599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f359801b030 con 0x7f35a0108970 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f359dffb700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35900b2a70 0x7f35900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f35a4a81700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35900b2a70 0x7f35900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f359dffb700 1 -- 172.21.15.5:0/1959668599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f35980e2420 con 0x7f35a0108970 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f35b56f9b80 1 Processor -- start 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f35b56f9b80 1 -- start start 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f35a4a81700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35900b2a70 0x7f35900b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3594011490 tx=0x7f3594009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d728ff9e40 0x55d72917d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.956+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 0x55d7291778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d729178220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d729099940 con 0x55d72917dd80 2024-05-14T15:29:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d728ffeda0 con 0x55d728ff9e40 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d729095900 con 0x55d729177e10 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f359ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 0x55d7291778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f359ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 0x55d7291778d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59170/0 (socket says 172.21.15.5:59170) 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f359ffff700 1 -- 172.21.15.5:0/3265045951 learned_addr learned my addr 172.21.15.5:0/3265045951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a4a81700 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d728ff9e40 0x55d72917d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d729178220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 -- 172.21.15.5:0/3265045951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d728ff9e40 msgr2=0x55d72917d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d728ff9e40 0x55d72917d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 -- 172.21.15.5:0/3265045951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 msgr2=0x55d7291778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 0x55d7291778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 -- 172.21.15.5:0/3265045951 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d728fe68d0 con 0x55d729177e10 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f359ffff700 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 0x55d7291778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f35a5282700 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d729178220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35980df750 tx=0x7f359800ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d502d2380466d72c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.957+0000 7f358affd700 1 -- 172.21.15.5:0/3265045951 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980a9370 con 0x55d729177e10 2024-05-14T15:29:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f358affd700 1 -- 172.21.15.5:0/3265045951 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35980ad720 con 0x55d729177e10 2024-05-14T15:29:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f358affd700 1 -- 172.21.15.5:0/3265045951 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980b1eb0 con 0x55d729177e10 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3265045951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 msgr2=0x55d729178220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d729178220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35980df750 tx=0x7f359800ca60 comp rx=0 tx=0).stop 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3265045951 shutdown_connections 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d729178220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d728ff9e40 0x55d72917d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3265045951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d72917dd80 0x55d7291778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.958+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3265045951 >> 172.21.15.5:0/3265045951 conn(0x55d729041630 msgr2=0x55d729172a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.959+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3265045951 shutdown_connections 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.959+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3265045951 wait complete. 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.959+0000 7f35b56f9b80 1 -- start start 2024-05-14T15:29:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.959+0000 7f35b56f9b80 1 Processor -- start 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.959+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d729181f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d729184820 con 0x55d728ff9e40 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d729052ed0 con 0x55d729177e10 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7291849a0 con 0x55d72917dd80 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291814c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59186/0 (socket says 172.21.15.5:59186) 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 -- 172.21.15.5:0/3264449401 learned_addr learned my addr 172.21.15.5:0/3264449401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35b56f9b80 10 client.0 _ll_register_callbacks cb 0x55d729031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f359ffff700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a5282700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d729181f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 -- 172.21.15.5:0/3264449401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 msgr2=0x55d729181f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d729181f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 -- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 msgr2=0x55d729181a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 -- 172.21.15.5:0/3264449401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d729050760 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f359ffff700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f35a4a81700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291814c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3594003ad0 tx=0x7f35940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649745567c36fdee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.960+0000 7f357ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f357ffff700 1 -- 172.21.15.5:0/3264449401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359401fa90 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35980ad720 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f357ffff700 1 -- 172.21.15.5:0/3264449401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f359401fc30 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f357ffff700 1 -- 172.21.15.5:0/3264449401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f359401fe30 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f35b56f9b80 10 client.16509 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d72918f4d0 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f357ffff700 1 -- 172.21.15.5:0/3264449401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35980ad720 con 0x55d728ff9e40 2024-05-14T15:29:36.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f357ffff700 1 -- 172.21.15.5:0/3264449401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3594026070 con 0x55d728ff9e40 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.961+0000 7f35b56f9b80 10 client.16509 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f35b56f9b80 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f35b56f9b80 20 client.16509 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d729191a60 0x55d729193ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f35b56f9b80 4 client.16509 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7290a7320 con 0x55d729191a60 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f359ffff700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d729191a60 0x55d729193ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f359ffff700 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d729191a60 0x55d729193ef0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d0204ddab4bab7 server_cookie=81eb55767ac4a890 in_seq=0 out_seq=0 2024-05-14T15:29:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.962+0000 7f357ffff700 10 client.16509 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f357ffff700 1 -- 172.21.15.5:0/3264449401 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7290a7320 con 0x55d729191a60 2024-05-14T15:29:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f357ffff700 10 client.16509 handle_command_reply: tid=0 2024-05-14T15:29:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 client.16509 shutdown 2024-05-14T15:29:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 msgr2=0x55d7291814c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291814c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3594003ad0 tx=0x7f35940047c0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 shutdown_connections 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d729191a60 0x55d729193ef0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d729181f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291814c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/3264449401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 >> 172.21.15.5:0/3264449401 conn(0x55d729041630 msgr2=0x55d7291768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f357ffff700 0 client.16509 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 shutdown_connections 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 1 -- 172.21.15.5:0/3264449401 wait complete. 2024-05-14T15:29:36.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.964+0000 7f35b56f9b80 20 client.16509 trim_cache size 0 max 16384 2024-05-14T15:29:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 Processor -- start 2024-05-14T15:29:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 -- start start 2024-05-14T15:29:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d7291804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01b5140 con 0x55d728ff9e40 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01b29d0 con 0x55d729177e10 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01b0820 con 0x55d72917dd80 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291778d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59194/0 (socket says 172.21.15.5:59194) 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35a4a81700 1 -- 172.21.15.5:0/210256681 learned_addr learned my addr 172.21.15.5:0/210256681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f35a5282700 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d7291804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.980+0000 7f359ffff700 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f35a5282700 1 -- 172.21.15.5:0/210256681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 msgr2=0x55d729181d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f35a5282700 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f35a5282700 1 -- 172.21.15.5:0/210256681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 msgr2=0x55d7291778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f35a5282700 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f35a5282700 1 -- 172.21.15.5:0/210256681 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d729174020 con 0x55d72917dd80 2024-05-14T15:29:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f35a5282700 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d7291804e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35980df780 tx=0x7f35980a52b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62efd037d0e04445 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f358a7fc700 1 -- 172.21.15.5:0/210256681 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980b2140 con 0x55d72917dd80 2024-05-14T15:29:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f358a7fc700 1 -- 172.21.15.5:0/210256681 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35980ad720 con 0x55d72917dd80 2024-05-14T15:29:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.981+0000 7f358a7fc700 1 -- 172.21.15.5:0/210256681 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35980b1ef0 con 0x55d72917dd80 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 -- 172.21.15.5:0/210256681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 msgr2=0x55d7291804e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d7291804e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f35980df780 tx=0x7f35980a52b0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 -- 172.21.15.5:0/210256681 shutdown_connections 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d72917dd80 0x55d7291804e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d7291778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/210256681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d729177e10 0x55d729181d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 -- 172.21.15.5:0/210256681 >> 172.21.15.5:0/210256681 conn(0x55d729041630 msgr2=0x55d72917fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 -- 172.21.15.5:0/210256681 shutdown_connections 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 -- 172.21.15.5:0/210256681 wait complete. 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 -- start start 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.982+0000 7f35b56f9b80 1 Processor -- start 2024-05-14T15:29:36.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d729184f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d7291854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72917dd80 0x55d7291fa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01b0e90 con 0x55d728ff9e40 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01b2300 con 0x55d72917dd80 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01b1560 con 0x55d729177e10 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35b56f9b80 10 client.0 _ll_register_callbacks cb 0x55d729031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d729184f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35a4a81700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d729184f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59200/0 (socket says 172.21.15.5:59200) 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f359ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d7291854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35a4a81700 1 -- 172.21.15.5:0/789831899 learned_addr learned my addr 172.21.15.5:0/789831899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35a5282700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72917dd80 0x55d7291fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f359ffff700 1 -- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72917dd80 msgr2=0x55d7291fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f359ffff700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72917dd80 0x55d7291fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f359ffff700 1 -- 172.21.15.5:0/789831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 msgr2=0x55d729184f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f359ffff700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d729184f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f359ffff700 1 -- 172.21.15.5:0/789831899 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7291fa790 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.983+0000 7f35a4a81700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d729184f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f359ffff700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d7291854b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55d72917d300 tx=0x7f358c005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f159acc6a773fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f357ffff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f357ffff700 1 -- 172.21.15.5:0/789831899 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0039d0 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f357ffff700 1 -- 172.21.15.5:0/789831899 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f358c003b70 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35980ad720 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f357ffff700 1 -- 172.21.15.5:0/789831899 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f358c038050 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 10 client.26125 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d7291fb9d0 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f357ffff700 1 -- 172.21.15.5:0/789831899 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35980ad720 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f357ffff700 1 -- 172.21.15.5:0/789831899 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f358c0293d0 con 0x55d729177e10 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 10 client.26125 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 20 client.26125 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d7291fc110 0x55d7291fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 4 client.26125 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.984+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7290a13c0 con 0x55d7291fc110 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.985+0000 7f35a4a81700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d7291fc110 0x55d7291fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.985+0000 7f357ffff700 10 client.26125 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:36.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.985+0000 7f35a4a81700 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d7291fc110 0x55d7291fe530 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57223c3234b49e13 server_cookie=3a4a5a6965c2de16 in_seq=0 out_seq=0 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f357ffff700 1 -- 172.21.15.5:0/789831899 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7290a13c0 con 0x55d7291fc110 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f357ffff700 10 client.26125 handle_command_reply: tid=0 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 client.26125 shutdown 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 msgr2=0x55d7291854b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d7291854b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55d72917d300 tx=0x7f358c005680 comp rx=0 tx=0).stop 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 shutdown_connections 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d7291fc110 0x55d7291fe530 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d72917dd80 0x55d7291fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d728ff9e40 0x55d729184f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 --2- 172.21.15.5:0/789831899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d729177e10 0x55d7291854b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f357ffff700 0 client.26125 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.992+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 >> 172.21.15.5:0/789831899 conn(0x55d729041630 msgr2=0x55d7291828a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 shutdown_connections 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35b56f9b80 1 -- 172.21.15.5:0/789831899 wait complete. 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35b56f9b80 20 client.26125 trim_cache size 0 max 16384 2024-05-14T15:29:36.995 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35900b2a70 msgr2=0x7f35900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35900b2a70 0x7f35900b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3594011490 tx=0x7f3594009260 comp rx=0 tx=0).stop 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 msgr2=0x7f35a019c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 0x7f35a019c920 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f359800ef30 tx=0x7f359800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 shutdown_connections 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f35900b2a70 0x7f35900b4f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a0108970 0x7f35a019c920 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a0103c10 0x7f35a019be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 --2- 172.21.15.5:0/1959668599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a0108020 0x7f35a019c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.993+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 >> 172.21.15.5:0/1959668599 conn(0x7f35a00fe520 msgr2=0x7f35a01098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.994+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 shutdown_connections 2024-05-14T15:29:36.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:36.994+0000 7f35a6ce5700 1 -- 172.21.15.5:0/1959668599 wait complete. 2024-05-14T15:29:37.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 401127 11.74 MB/sec execute 58 sec latency 19.454 ms 2024-05-14T15:29:38.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 403462 11.68 MB/sec execute 59 sec latency 18.554 ms 2024-05-14T15:29:38.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:37 smithi144 ceph-mon[35650]: pgmap v148: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 484 op/s 2024-05-14T15:29:38.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:37 smithi193 ceph-mon[36001]: pgmap v148: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 484 op/s 2024-05-14T15:29:38.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:37 smithi005 ceph-mon[31559]: pgmap v148: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.1 MiB/s wr, 484 op/s 2024-05-14T15:29:38.947 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:29:38.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:29:39.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 405313 11.68 MB/sec execute 60 sec latency 13.189 ms 2024-05-14T15:29:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:40.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 408149 11.79 MB/sec execute 61 sec latency 20.098 ms 2024-05-14T15:29:40.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:39 smithi144 ceph-mon[35650]: pgmap v149: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-14T15:29:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:39 smithi193 ceph-mon[36001]: pgmap v149: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-14T15:29:40.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:39 smithi005 ceph-mon[31559]: pgmap v149: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-14T15:29:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.108+0000 7f49c401c700 1 -- 172.21.15.5:0/944152184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc109870 msgr2=0x7f49bc1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.108+0000 7f49c401c700 1 --2- 172.21.15.5:0/944152184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc109870 0x7f49bc1111d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b0d0 tx=0x7f49b400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 -- 172.21.15.5:0/944152184 shutdown_connections 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 --2- 172.21.15.5:0/944152184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc109870 0x7f49bc1111d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 --2- 172.21.15.5:0/944152184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49bc1045f0 0x7f49bc104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 --2- 172.21.15.5:0/944152184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49bc108f20 0x7f49bc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 -- 172.21.15.5:0/944152184 >> 172.21.15.5:0/944152184 conn(0x7f49bc100040 msgr2=0x7f49bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 -- 172.21.15.5:0/944152184 shutdown_connections 2024-05-14T15:29:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.109+0000 7f49c401c700 1 -- 172.21.15.5:0/944152184 wait complete. 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 Processor -- start 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 -- start start 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 0x7f49bc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49bc108f20 0x7f49bc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49bc109870 0x7f49bc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1144a0 con 0x7f49bc1045f0 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc114320 con 0x7f49bc109870 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c401c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc114620 con 0x7f49bc108f20 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 0x7f49bc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 0x7f49bc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59216/0 (socket says 172.21.15.5:59216) 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c1db8700 1 -- 172.21.15.5:0/1318533028 learned_addr learned my addr 172.21.15.5:0/1318533028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.110+0000 7f49c25b9700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49bc109870 0x7f49bc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c15b7700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49bc108f20 0x7f49bc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c1db8700 1 -- 172.21.15.5:0/1318533028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49bc108f20 msgr2=0x7f49bc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c1db8700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49bc108f20 0x7f49bc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c1db8700 1 -- 172.21.15.5:0/1318533028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49bc109870 msgr2=0x7f49bc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c1db8700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49bc109870 0x7f49bc19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c1db8700 1 -- 172.21.15.5:0/1318533028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc19fc80 con 0x7f49bc1045f0 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c1db8700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 0x7f49bc19adb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e9a0 tx=0x7f49b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49b2ffd700 1 -- 172.21.15.5:0/1318533028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b800ca10 con 0x7f49bc1045f0 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49b2ffd700 1 -- 172.21.15.5:0/1318533028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b800cbb0 con 0x7f49bc1045f0 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc19ff70 con 0x7f49bc1045f0 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49b2ffd700 1 -- 172.21.15.5:0/1318533028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8018680 con 0x7f49bc1045f0 2024-05-14T15:29:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.111+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc1a7890 con 0x7f49bc1045f0 2024-05-14T15:29:41.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.114+0000 7f49b2ffd700 1 -- 172.21.15.5:0/1318533028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f49b8010430 con 0x7f49bc1045f0 2024-05-14T15:29:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.115+0000 7f49b2ffd700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f49a80b29f0 0x7f49a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.115+0000 7f49c15b7700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f49a80b29f0 0x7f49a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.115+0000 7f49d2a30b80 1 Processor -- start 2024-05-14T15:29:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.115+0000 7f49b2ffd700 1 -- 172.21.15.5:0/1318533028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f49b80e2b90 con 0x7f49bc1045f0 2024-05-14T15:29:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 -- start start 2024-05-14T15:29:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db42bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4db47d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd4da608e0 con 0x55bd4db47910 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd4d9ff840 con 0x55bd4db430f0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd4da5c8a0 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49c15b7700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f49a80b29f0 0x7f49a80b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49bc103a40 tx=0x7f49ac009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db42bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49c25b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4db47d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db42bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:58940/0 (socket says 172.21.15.5:58940) 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49c1db8700 1 -- 172.21.15.5:0/3404413626 learned_addr learned my addr 172.21.15.5:0/3404413626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.116+0000 7f49c15b7700 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49c1db8700 1 -- 172.21.15.5:0/3404413626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 msgr2=0x55bd4db473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49c1db8700 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49c1db8700 1 -- 172.21.15.5:0/3404413626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 msgr2=0x55bd4db47d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49c1db8700 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4db47d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49c1db8700 1 -- 172.21.15.5:0/3404413626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4d9ad950 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49c1db8700 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db42bb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f49b80a9750 tx=0x7f49b80b2310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85bd51738969fa9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49a7fff700 1 -- 172.21.15.5:0/3404413626 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a5050 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49a7fff700 1 -- 172.21.15.5:0/3404413626 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b80b2040 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.117+0000 7f49a7fff700 1 -- 172.21.15.5:0/3404413626 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80b2420 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3404413626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 msgr2=0x55bd4db42bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db42bb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f49b80a9750 tx=0x7f49b80b2310 comp rx=0 tx=0).stop 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3404413626 shutdown_connections 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4db47d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db42bb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3404413626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3404413626 >> 172.21.15.5:0/3404413626 conn(0x55bd4da08680 msgr2=0x55bd4db39530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3404413626 shutdown_connections 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.118+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3404413626 wait complete. 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 -- start start 2024-05-14T15:29:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 Processor -- start 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db4c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db4cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd4db4f8c0 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd4d9c6cb0 con 0x55bd4db430f0 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd4db4fa40 con 0x55bd4db47910 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49d2a30b80 10 client.0 _ll_register_callbacks cb 0x55bd4d9f89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db4c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db4c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59242/0 (socket says 172.21.15.5:59242) 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49c1db8700 1 -- 172.21.15.5:0/3156058146 learned_addr learned my addr 172.21.15.5:0/3156058146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49c15b7700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db4cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.119+0000 7f49c25b9700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c15b7700 1 -- 172.21.15.5:0/3156058146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 msgr2=0x55bd4db4d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c15b7700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db4d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c15b7700 1 -- 172.21.15.5:0/3156058146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 msgr2=0x55bd4db4c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c15b7700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db4c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c15b7700 1 -- 172.21.15.5:0/3156058146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4d9fc710 con 0x55bd4db430f0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c1db8700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db4c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49c15b7700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db4cb90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00a010 tx=0x7f49ac0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8fc8e1d55e2377 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49a57fa700 1 -- 172.21.15.5:0/3156058146 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac004260 con 0x55bd4db430f0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b80b2040 con 0x55bd4db430f0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49a57fa700 1 -- 172.21.15.5:0/3156058146 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f49ac004400 con 0x55bd4db430f0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49a57fa700 1 -- 172.21.15.5:0/3156058146 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49ac031050 con 0x55bd4db430f0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49d2a30b80 10 client.26153 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.120+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd4db5a550 con 0x55bd4db430f0 2024-05-14T15:29:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49a57fa700 1 -- 172.21.15.5:0/3156058146 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b80b2040 con 0x55bd4db430f0 2024-05-14T15:29:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49a57fa700 1 -- 172.21.15.5:0/3156058146 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f49ac026020 con 0x55bd4db430f0 2024-05-14T15:29:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49d2a30b80 10 client.26153 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49d2a30b80 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49d2a30b80 20 client.26153 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4db5cb90 0x55bd4db5efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49d2a30b80 4 client.26153 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd4da6e2c0 con 0x55bd4db5cb90 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.121+0000 7f49c1db8700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4db5cb90 0x55bd4db5efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.122+0000 7f49a57fa700 10 client.26153 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.122+0000 7f49c1db8700 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4db5cb90 0x55bd4db5efb0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df184f29ed8d58d server_cookie=f2b5e105babce8d5 in_seq=0 out_seq=0 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.122+0000 7f49a57fa700 1 -- 172.21.15.5:0/3156058146 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd4da6e2c0 con 0x55bd4db5cb90 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.122+0000 7f49a57fa700 10 client.26153 handle_command_reply: tid=0 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.122+0000 7f49d2a30b80 1 client.26153 shutdown 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 msgr2=0x55bd4db4cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db4cb90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00a010 tx=0x7f49ac0047c0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 shutdown_connections 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4db5cb90 0x55bd4db5efb0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db4d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db4c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/3156058146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4db430f0 0x55bd4db4cb90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 >> 172.21.15.5:0/3156058146 conn(0x55bd4da08680 msgr2=0x55bd4db3d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49a57fa700 0 client.26153 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 shutdown_connections 2024-05-14T15:29:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 1 -- 172.21.15.5:0/3156058146 wait complete. 2024-05-14T15:29:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.123+0000 7f49d2a30b80 20 client.26153 trim_cache size 0 max 16384 2024-05-14T15:29:41.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409458 11.69 MB/sec execute 62 sec latency 62.196 ms 2024-05-14T15:29:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 Processor -- start 2024-05-14T15:29:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 -- start start 2024-05-14T15:29:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db3df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 0x55bd4db4cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db48d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1b0e50 con 0x55bd4db430f0 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ae6e0 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ac530 con 0x55bd4db47910 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49c15b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 0x55bd4db4cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49c15b7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 0x55bd4db4cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59250/0 (socket says 172.21.15.5:59250) 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49c15b7700 1 -- 172.21.15.5:0/1213508802 learned_addr learned my addr 172.21.15.5:0/1213508802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49c1db8700 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db3df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.147+0000 7f49c25b9700 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db48d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49c15b7700 1 -- 172.21.15.5:0/1213508802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 msgr2=0x55bd4db48d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49c15b7700 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db48d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49c15b7700 1 -- 172.21.15.5:0/1213508802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 msgr2=0x55bd4db3df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49c15b7700 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db3df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49c15b7700 1 -- 172.21.15.5:0/1213508802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4db4d450 con 0x55bd4db430f0 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49c15b7700 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 0x55bd4db4cf10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005950 tx=0x7f49ac009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311389412eece76 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49a77fe700 1 -- 172.21.15.5:0/1213508802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac03b020 con 0x55bd4db430f0 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49a77fe700 1 -- 172.21.15.5:0/1213508802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ac028540 con 0x55bd4db430f0 2024-05-14T15:29:41.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.148+0000 7f49a77fe700 1 -- 172.21.15.5:0/1213508802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac003cb0 con 0x55bd4db430f0 2024-05-14T15:29:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 -- 172.21.15.5:0/1213508802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 msgr2=0x55bd4db4cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 0x55bd4db4cf10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005950 tx=0x7f49ac009330 comp rx=0 tx=0).stop 2024-05-14T15:29:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 -- 172.21.15.5:0/1213508802 shutdown_connections 2024-05-14T15:29:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db47910 0x55bd4db48d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db3df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/1213508802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db430f0 0x55bd4db4cf10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 -- 172.21.15.5:0/1213508802 >> 172.21.15.5:0/1213508802 conn(0x55bd4da08680 msgr2=0x55bd4db392d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 -- 172.21.15.5:0/1213508802 shutdown_connections 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.149+0000 7f49d2a30b80 1 -- 172.21.15.5:0/1213508802 wait complete. 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 -- start start 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 Processor -- start 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db64210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db430f0 0x55bd4db64750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4dbc62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1acba0 con 0x55bd4db47910 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ae010 con 0x55bd4d9c0ec0 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ad270 con 0x55bd4db430f0 2024-05-14T15:29:41.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49d2a30b80 10 client.0 _ll_register_callbacks cb 0x55bd4d9f89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49c25b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4dbc62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49c1db8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db64210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49c25b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4dbc62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59264/0 (socket says 172.21.15.5:59264) 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49c25b9700 1 -- 172.21.15.5:0/476711127 learned_addr learned my addr 172.21.15.5:0/476711127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.150+0000 7f49c15b7700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db430f0 0x55bd4db64750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c25b9700 1 -- 172.21.15.5:0/476711127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db430f0 msgr2=0x55bd4db64750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c25b9700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db430f0 0x55bd4db64750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c25b9700 1 -- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 msgr2=0x55bd4db64210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c25b9700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db64210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c25b9700 1 -- 172.21.15.5:0/476711127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4db64f70 con 0x55bd4db47910 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c1db8700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db64210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49c25b9700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4dbc62a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f49b400bf50 tx=0x7f49b400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a64bb3c2d0c30f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49a57fa700 1 -- 172.21.15.5:0/476711127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b40075c0 con 0x55bd4db47910 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49a57fa700 1 -- 172.21.15.5:0/476711127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f49b4007760 con 0x55bd4db47910 2024-05-14T15:29:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ac028540 con 0x55bd4db47910 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49a57fa700 1 -- 172.21.15.5:0/476711127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49b4007960 con 0x55bd4db47910 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49d2a30b80 10 client.16533 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.151+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd4dbc77d0 con 0x55bd4db47910 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49a57fa700 1 -- 172.21.15.5:0/476711127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ac028540 con 0x55bd4db47910 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49a57fa700 1 -- 172.21.15.5:0/476711127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f49b4011070 con 0x55bd4db47910 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49d2a30b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49d2a30b80 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49d2a30b80 20 client.16533 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4dbc7ee0 0x55bd4dbca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49d2a30b80 4 client.16533 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd4da68360 con 0x55bd4dbc7ee0 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.152+0000 7f49c1db8700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4dbc7ee0 0x55bd4dbca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.153+0000 7f49a57fa700 10 client.16533 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.153+0000 7f49c1db8700 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4dbc7ee0 0x55bd4dbca300 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc02b8ddbd9b2f2 server_cookie=d5fcad4aa7baa305 in_seq=0 out_seq=0 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.156+0000 7f49a57fa700 1 -- 172.21.15.5:0/476711127 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd4da68360 con 0x55bd4dbc7ee0 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.156+0000 7f49a57fa700 10 client.16533 handle_command_reply: tid=0 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.156+0000 7f49d2a30b80 1 client.16533 shutdown 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.156+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 msgr2=0x55bd4dbc62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.156+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4dbc62a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f49b400bf50 tx=0x7f49b400bf80 comp rx=0 tx=0).stop 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 shutdown_connections 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bd4dbc7ee0 0x55bd4dbca300 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd4db47910 0x55bd4dbc62a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd4d9c0ec0 0x55bd4db64210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 --2- 172.21.15.5:0/476711127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd4db430f0 0x55bd4db64750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49a57fa700 0 client.16533 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 >> 172.21.15.5:0/476711127 conn(0x55bd4da08680 msgr2=0x55bd4db495a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 shutdown_connections 2024-05-14T15:29:41.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 1 -- 172.21.15.5:0/476711127 wait complete. 2024-05-14T15:29:41.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.157+0000 7f49d2a30b80 20 client.16533 trim_cache size 0 max 16384 2024-05-14T15:29:41.160 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:29:41.160 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:29:41.160 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3af91ae8-b1fc-4ddd-85b0-35be8b18c219", 2024-05-14T15:29:41.160 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:29:41.160 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f49a80b29f0 msgr2=0x7f49a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f49a80b29f0 0x7f49a80b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49bc103a40 tx=0x7f49ac009260 comp rx=0 tx=0).stop 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 msgr2=0x7f49bc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 0x7f49bc19adb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f49b800e9a0 tx=0x7f49b800ecb0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 shutdown_connections 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f49a80b29f0 0x7f49a80b4f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49bc109870 0x7f49bc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49bc1045f0 0x7f49bc19adb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 --2- 172.21.15.5:0/1318533028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49bc108f20 0x7f49bc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 >> 172.21.15.5:0/1318533028 conn(0x7f49bc100040 msgr2=0x7f49bc108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 shutdown_connections 2024-05-14T15:29:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:41.158+0000 7f49c401c700 1 -- 172.21.15.5:0/1318533028 wait complete. 2024-05-14T15:29:41.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:29:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:41 smithi005 ceph-mon[31559]: pgmap v150: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-14T15:29:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:41 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:29:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:41 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:29:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:41 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:29:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:41 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:29:42.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:29:42.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 411886 11.72 MB/sec execute 63 sec latency 27.716 ms 2024-05-14T15:29:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:41 smithi144 ceph-mon[35650]: pgmap v150: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-14T15:29:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:41 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:29:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:41 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:29:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:41 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:29:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:41 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:29:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:41 smithi193 ceph-mon[36001]: pgmap v150: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-05-14T15:29:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:41 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:29:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:41 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:29:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:41 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:29:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:41 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:29:43.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 414483 11.74 MB/sec execute 64 sec latency 13.338 ms 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 -- 172.21.15.5:0/821711145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8103550 msgr2=0x7f41b8103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 --2- 172.21.15.5:0/821711145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8103550 0x7f41b8103960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f41a80099c0 tx=0x7f41a8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 -- 172.21.15.5:0/821711145 shutdown_connections 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 --2- 172.21.15.5:0/821711145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41b8108780 0x7f41b81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 --2- 172.21.15.5:0/821711145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8103550 0x7f41b8103960 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 --2- 172.21.15.5:0/821711145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b8107e30 0x7f41b8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 -- 172.21.15.5:0/821711145 >> 172.21.15.5:0/821711145 conn(0x7f41b8078e00 msgr2=0x7f41b8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 -- 172.21.15.5:0/821711145 shutdown_connections 2024-05-14T15:29:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.949+0000 7f41bd466700 1 -- 172.21.15.5:0/821711145 wait complete. 2024-05-14T15:29:43.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:43 smithi005 ceph-mon[31559]: pgmap v151: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-05-14T15:29:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 Processor -- start 2024-05-14T15:29:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 -- start start 2024-05-14T15:29:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41b8103550 0x7f41b819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 0x7f41b819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b8108780 0x7f41b81a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81133b0 con 0x7f41b8107e30 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113230 con 0x7f41b8103550 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41bd466700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8113530 con 0x7f41b8108780 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 0x7f41b819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 0x7f41b819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59290/0 (socket says 172.21.15.5:59290) 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41b67fc700 1 -- 172.21.15.5:0/2056782582 learned_addr learned my addr 172.21.15.5:0/2056782582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41b8103550 0x7f41b819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.950+0000 7f41b77fe700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b8108780 0x7f41b81a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41b67fc700 1 -- 172.21.15.5:0/2056782582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b8108780 msgr2=0x7f41b81a6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b8108780 0x7f41b81a6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41b67fc700 1 -- 172.21.15.5:0/2056782582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41b8103550 msgr2=0x7f41b819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41b8103550 0x7f41b819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41b67fc700 1 -- 172.21.15.5:0/2056782582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b819ca00 con 0x7f41b8107e30 2024-05-14T15:29:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 0x7f41b819c150 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e970 tx=0x7f41a000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41affff700 1 -- 172.21.15.5:0/2056782582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0004a30 con 0x7f41b8107e30 2024-05-14T15:29:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41affff700 1 -- 172.21.15.5:0/2056782582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a000cc90 con 0x7f41b8107e30 2024-05-14T15:29:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b81a72f0 con 0x7f41b8107e30 2024-05-14T15:29:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41affff700 1 -- 172.21.15.5:0/2056782582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0005610 con 0x7f41b8107e30 2024-05-14T15:29:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.951+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a77b0 con 0x7f41b8107e30 2024-05-14T15:29:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.954+0000 7f41affff700 1 -- 172.21.15.5:0/2056782582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f41a000c7e0 con 0x7f41b8107e30 2024-05-14T15:29:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.955+0000 7f41affff700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f41a40b29f0 0x7f41a40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.955+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f41a40b29f0 0x7f41a40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.955+0000 7f41affff700 1 -- 172.21.15.5:0/2056782582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f41a00e2b30 con 0x7f41b8107e30 2024-05-14T15:29:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.955+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f41a40b29f0 0x7f41a40b4f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41a8003760 tx=0x7f41a801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 Processor -- start 2024-05-14T15:29:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 -- start start 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f272840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f26c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f26d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f6f18e940 con 0x559f6f26ce10 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f6f0f3da0 con 0x559f6f272d80 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f6f18a900 con 0x559f6f0eee40 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f26c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41b77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f26d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f26c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:54348/0 (socket says 172.21.15.5:54348) 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41b67fc700 1 -- 172.21.15.5:0/3302704918 learned_addr learned my addr 172.21.15.5:0/3302704918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.956+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f272840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41b67fc700 1 -- 172.21.15.5:0/3302704918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 msgr2=0x559f6f272840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41b67fc700 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f272840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41b67fc700 1 -- 172.21.15.5:0/3302704918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 msgr2=0x559f6f26d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41b67fc700 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f26d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41b67fc700 1 -- 172.21.15.5:0/3302704918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6f0db8d0 con 0x559f6f272d80 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41b67fc700 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f26c8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f41a00a99e0 tx=0x7f41a00a98f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc60eb3e2c681c0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41ad7fa700 1 -- 172.21.15.5:0/3302704918 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a00a5040 con 0x559f6f272d80 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41ad7fa700 1 -- 172.21.15.5:0/3302704918 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a00b2050 con 0x559f6f272d80 2024-05-14T15:29:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.957+0000 7f41ad7fa700 1 -- 172.21.15.5:0/3302704918 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a00aa150 con 0x559f6f272d80 2024-05-14T15:29:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/3302704918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 msgr2=0x559f6f26c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f26c8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f41a00a99e0 tx=0x7f41a00a98f0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/3302704918 shutdown_connections 2024-05-14T15:29:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f26d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f272840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/3302704918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f26c8d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/3302704918 >> 172.21.15.5:0/3302704918 conn(0x559f6f136630 msgr2=0x559f6f267a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/3302704918 shutdown_connections 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.958+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/3302704918 wait complete. 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 -- start start 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 Processor -- start 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 0x559f6f276510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f276a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f272d80 0x559f6f276f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f6f2797c0 con 0x559f6f26ce10 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f6f147ed0 con 0x559f6f0eee40 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f6f279940 con 0x559f6f272d80 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41cbe7ab80 10 client.0 _ll_register_callbacks cb 0x559f6f126950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f276a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 0x559f6f276510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f276a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59304/0 (socket says 172.21.15.5:59304) 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41b67fc700 1 -- 172.21.15.5:0/2939587829 learned_addr learned my addr 172.21.15.5:0/2939587829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 0x559f6f276510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:54352/0 (socket says 172.21.15.5:54352) 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.959+0000 7f41b77fe700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f272d80 0x559f6f276f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b67fc700 1 -- 172.21.15.5:0/2939587829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f272d80 msgr2=0x559f6f276f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f272d80 0x559f6f276f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b67fc700 1 -- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 msgr2=0x559f6f276510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 0x559f6f276510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b67fc700 1 -- 172.21.15.5:0/2939587829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6f145760 con 0x559f6f26ce10 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 0x559f6f276510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f276a50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f41a000efd0 tx=0x7f41a0016560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7860ffaaeee02c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f419e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:43.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f419e7fc700 1 -- 172.21.15.5:0/2939587829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a006e060 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f419e7fc700 1 -- 172.21.15.5:0/2939587829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f41a00a9ec0 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a00b2050 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f419e7fc700 1 -- 172.21.15.5:0/2939587829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41a00a5040 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41cbe7ab80 10 client.16545 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f6f284410 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.960+0000 7f419e7fc700 1 -- 172.21.15.5:0/2939587829 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a00b2050 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f419e7fc700 1 -- 172.21.15.5:0/2939587829 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f41a006f250 con 0x559f6f26ce10 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41cbe7ab80 10 client.16545 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41cbe7ab80 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41cbe7ab80 20 client.16545 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f288d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41cbe7ab80 4 client.16545 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f6f19c320 con 0x559f6f2868e0 2024-05-14T15:29:43.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.961+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f288d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f419e7fc700 10 client.16545 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f288d00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd777052a30464a7 server_cookie=fd25648ecf481423 in_seq=0 out_seq=0 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f419e7fc700 1 -- 172.21.15.5:0/2939587829 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f6f19c320 con 0x559f6f2868e0 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f419e7fc700 10 client.16545 handle_command_reply: tid=0 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f41cbe7ab80 1 client.16545 shutdown 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 msgr2=0x559f6f276a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f276a50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f41a000efd0 tx=0x7f41a0016560 comp rx=0 tx=0).stop 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 shutdown_connections 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.962+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f288d00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f419e7fc700 0 client.16545 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f272d80 0x559f6f276f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f0eee40 0x559f6f276510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2939587829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f276a50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 >> 172.21.15.5:0/2939587829 conn(0x559f6f136630 msgr2=0x559f6f26b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:43.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 shutdown_connections 2024-05-14T15:29:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2939587829 wait complete. 2024-05-14T15:29:43.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.963+0000 7f41cbe7ab80 20 client.16545 trim_cache size 0 max 16384 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.983+0000 7f41cbe7ab80 1 Processor -- start 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.983+0000 7f41cbe7ab80 1 -- start start 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 0x559f6f2741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f26ce10 0x559f6f26c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f275850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b0d40 con 0x559f6f0eee40 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81ae5d0 con 0x559f6f272d80 2024-05-14T15:29:43.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81ac420 con 0x559f6f26ce10 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 0x559f6f2741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 0x559f6f2741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59312/0 (socket says 172.21.15.5:59312) 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 -- 172.21.15.5:0/2401984745 learned_addr learned my addr 172.21.15.5:0/2401984745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f26ce10 0x559f6f26c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b77fe700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f275850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 -- 172.21.15.5:0/2401984745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f26ce10 msgr2=0x559f6f26c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f26ce10 0x559f6f26c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 -- 172.21.15.5:0/2401984745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 msgr2=0x559f6f275850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f275850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 -- 172.21.15.5:0/2401984745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6f2693b0 con 0x559f6f0eee40 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f26ce10 0x559f6f26c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b77fe700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f275850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.984+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 0x559f6f2741a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f41a800b580 tx=0x7f41a8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed88b16f1c52dcdf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.985+0000 7f41acff9700 1 -- 172.21.15.5:0/2401984745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a803d070 con 0x559f6f0eee40 2024-05-14T15:29:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.985+0000 7f41acff9700 1 -- 172.21.15.5:0/2401984745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a8038e50 con 0x559f6f0eee40 2024-05-14T15:29:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.985+0000 7f41acff9700 1 -- 172.21.15.5:0/2401984745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8037470 con 0x559f6f0eee40 2024-05-14T15:29:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.985+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2401984745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 msgr2=0x559f6f2741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.985+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 0x559f6f2741a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f41a800b580 tx=0x7f41a8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2401984745 shutdown_connections 2024-05-14T15:29:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f275850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f0eee40 0x559f6f2741a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2401984745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f26ce10 0x559f6f26c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2401984745 >> 172.21.15.5:0/2401984745 conn(0x559f6f136630 msgr2=0x559f6f2764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2401984745 shutdown_connections 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2401984745 wait complete. 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- start start 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 Processor -- start 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f28c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f289780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f289e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81aca90 con 0x559f6f26ce10 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81adf00 con 0x559f6f272d80 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81ad160 con 0x559f6f0eee40 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41cbe7ab80 10 client.0 _ll_register_callbacks cb 0x559f6f126950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.986+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f289780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f289780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59326/0 (socket says 172.21.15.5:59326) 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 -- 172.21.15.5:0/2513528870 learned_addr learned my addr 172.21.15.5:0/2513528870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f28c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b77fe700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f289e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 -- 172.21.15.5:0/2513528870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 msgr2=0x559f6f28c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f28c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 -- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 msgr2=0x559f6f289e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f289e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 -- 172.21.15.5:0/2513528870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6f267f40 con 0x559f6f26ce10 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41b67fc700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f289780 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f41a00a9af0 tx=0x7f41a00a98f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b560b31ee9c3d55 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f419e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f419e7fc700 1 -- 172.21.15.5:0/2513528870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a006e070 con 0x559f6f26ce10 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f419e7fc700 1 -- 172.21.15.5:0/2513528870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f41a00b2630 con 0x559f6f26ce10 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a8038e50 con 0x559f6f26ce10 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f419e7fc700 1 -- 172.21.15.5:0/2513528870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41a00a93a0 con 0x559f6f26ce10 2024-05-14T15:29:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41cbe7ab80 10 client.16551 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.987+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f6f2f0c50 con 0x559f6f26ce10 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f419e7fc700 1 -- 172.21.15.5:0/2513528870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a8038e50 con 0x559f6f26ce10 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f419e7fc700 1 -- 172.21.15.5:0/2513528870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f41a00cc0b0 con 0x559f6f26ce10 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41cbe7ab80 10 client.16551 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41cbe7ab80 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41cbe7ab80 20 client.16551 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f2f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41cbe7ab80 4 client.16551 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f6f1963c0 con 0x559f6f2868e0 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f2f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f41b6ffd700 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f2f3120 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c9418c76ba4895 server_cookie=9b4d49e57129b47 in_seq=0 out_seq=0 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.988+0000 7f419e7fc700 10 client.16551 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f419e7fc700 1 -- 172.21.15.5:0/2513528870 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559f6f1963c0 con 0x559f6f2868e0 2024-05-14T15:29:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f419e7fc700 10 client.16551 handle_command_reply: tid=0 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 client.16551 shutdown 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 msgr2=0x559f6f289780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f289780 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f41a00a9af0 tx=0x7f41a00a98f0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 shutdown_connections 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f6f2868e0 0x559f6f2f3120 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f6f272d80 0x559f6f289e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f6f0eee40 0x559f6f28c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f419e7fc700 0 client.16551 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 --2- 172.21.15.5:0/2513528870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f6f26ce10 0x559f6f289780 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 >> 172.21.15.5:0/2513528870 conn(0x559f6f136630 msgr2=0x559f6f2755b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 shutdown_connections 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 1 -- 172.21.15.5:0/2513528870 wait complete. 2024-05-14T15:29:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.989+0000 7f41cbe7ab80 20 client.16551 trim_cache size 0 max 16384 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "3af91ae8-b1fc-4ddd-85b0-35be8b18c219": { 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "3af91ae8-b1fc-4ddd-85b0-35be8b18c219", 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:29:43.996 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f41a40b29f0 msgr2=0x7f41a40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f41a40b29f0 0x7f41a40b4f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41a8003760 tx=0x7f41a801a040 comp rx=0 tx=0).stop 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 msgr2=0x7f41b819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 0x7f41b819c150 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e970 tx=0x7f41a000ec80 comp rx=0 tx=0).stop 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 shutdown_connections 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f41a40b29f0 0x7f41a40b4f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b8108780 0x7f41b81a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41b8103550 0x7f41b819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 --2- 172.21.15.5:0/2056782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8107e30 0x7f41b819c150 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 >> 172.21.15.5:0/2056782582 conn(0x7f41b8078e00 msgr2=0x7f41b8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 shutdown_connections 2024-05-14T15:29:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:29:43.990+0000 7f41bd466700 1 -- 172.21.15.5:0/2056782582 wait complete. 2024-05-14T15:29:44.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 415964 11.67 MB/sec execute 65 sec latency 59.959 ms 2024-05-14T15:29:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:43 smithi144 ceph-mon[35650]: pgmap v151: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-05-14T15:29:44.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:43 smithi193 ceph-mon[36001]: pgmap v151: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-05-14T15:29:44.736 INFO:tasks.cephfs.filesystem:scrub status for tag:3af91ae8-b1fc-4ddd-85b0-35be8b18c219 - {'path': '/', 'tag': '3af91ae8-b1fc-4ddd-85b0-35be8b18c219', 'options': 'recursive,force'} 2024-05-14T15:29:45.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 418828 11.77 MB/sec execute 66 sec latency 46.787 ms 2024-05-14T15:29:46.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 420205 11.67 MB/sec execute 67 sec latency 162.200 ms 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 -- 172.21.15.5:0/2374803675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 msgr2=0x7fbddc1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 --2- 172.21.15.5:0/2374803675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1051b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009900 tx=0x7fbdcc02a410 comp rx=0 tx=0).stop 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 -- 172.21.15.5:0/2374803675 shutdown_connections 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 --2- 172.21.15.5:0/2374803675 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbddc10b800 0x7fbddc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 --2- 172.21.15.5:0/2374803675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbddc10e1f0 0x7fbddc1106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 --2- 172.21.15.5:0/2374803675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1051b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 -- 172.21.15.5:0/2374803675 >> 172.21.15.5:0/2374803675 conn(0x7fbddc0782e0 msgr2=0x7fbddc078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 -- 172.21.15.5:0/2374803675 shutdown_connections 2024-05-14T15:29:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.241+0000 7fbde1be3640 1 -- 172.21.15.5:0/2374803675 wait complete. 2024-05-14T15:29:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 Processor -- start 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 -- start start 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbddc10b800 0x7fbddc1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbddc10e1f0 0x7fbddc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc113e50 con 0x7fbddc104da0 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc113cd0 con 0x7fbddc10b800 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbde1be3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc113fd0 con 0x7fbddc10e1f0 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.242+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56358/0 (socket says 172.21.15.5:56358) 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 -- 172.21.15.5:0/1474602790 learned_addr learned my addr 172.21.15.5:0/1474602790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddbfff640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbddc10e1f0 0x7fbddc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddaffd640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbddc10b800 0x7fbddc1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 -- 172.21.15.5:0/1474602790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbddc10e1f0 msgr2=0x7fbddc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbddc10e1f0 0x7fbddc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 -- 172.21.15.5:0/1474602790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbddc10b800 msgr2=0x7fbddc1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbddc10b800 0x7fbddc1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 -- 172.21.15.5:0/1474602790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbddc19ad30 con 0x7fbddc104da0 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1a0570 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005940 tx=0x7fbdcc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbdd8ff9640 1 -- 172.21.15.5:0/1474602790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc03f030 con 0x7fbddc104da0 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbdd8ff9640 1 -- 172.21.15.5:0/1474602790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdcc003aa0 con 0x7fbddc104da0 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbddc19afc0 con 0x7fbddc104da0 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbdd8ff9640 1 -- 172.21.15.5:0/1474602790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc035070 con 0x7fbddc104da0 2024-05-14T15:29:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.243+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbddc19b4b0 con 0x7fbddc104da0 2024-05-14T15:29:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.246+0000 7fbdd8ff9640 1 -- 172.21.15.5:0/1474602790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbdcc042030 con 0x7fbddc104da0 2024-05-14T15:29:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.246+0000 7fbde8b6d740 1 Processor -- start 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 -- start start 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb5777be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5778660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdb56dd5d0 con 0x55fdb53ae2f0 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fdb5696890 con 0x55fdb53b64d0 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fdb56d9590 con 0x55fdb53d71b0 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbdd8ff9640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbdc40b2950 0x7fbdc40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56370/0 (socket says 172.21.15.5:56370) 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddb7fe640 1 -- 172.21.15.5:0/2135650084 learned_addr learned my addr 172.21.15.5:0/2135650084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddbfff640 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb5777be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddaffd640 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5778660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddbfff640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbdc40b2950 0x7fbdc40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbdd8ff9640 1 -- 172.21.15.5:0/1474602790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fbdcc102eb0 con 0x7fbddc104da0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddaffd640 1 -- 172.21.15.5:0/2135650084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 msgr2=0x55fdb5777be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddaffd640 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb5777be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddaffd640 1 -- 172.21.15.5:0/2135650084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 msgr2=0x55fdb5778120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddaffd640 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.247+0000 7fbddaffd640 1 -- 172.21.15.5:0/2135650084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb53ef210 con 0x55fdb53b64d0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.248+0000 7fbddbfff640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbdc40b2950 0x7fbdc40b4e60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005cc0 tx=0x7fbdd4005c50 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.248+0000 7fbddaffd640 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5778660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbddc19bf20 tx=0x7fbdd0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111b8324060fa79b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.248+0000 7fbdc1ffb640 1 -- 172.21.15.5:0/2135650084 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0004230 con 0x55fdb53b64d0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.248+0000 7fbdc1ffb640 1 -- 172.21.15.5:0/2135650084 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd0009e10 con 0x55fdb53b64d0 2024-05-14T15:29:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.248+0000 7fbdc1ffb640 1 -- 172.21.15.5:0/2135650084 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0004c60 con 0x55fdb53b64d0 2024-05-14T15:29:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 -- 172.21.15.5:0/2135650084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 msgr2=0x55fdb5778660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5778660 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbddc19bf20 tx=0x7fbdd0011810 comp rx=0 tx=0).stop 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 -- 172.21.15.5:0/2135650084 shutdown_connections 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5778660 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/2135650084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb5777be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 -- 172.21.15.5:0/2135650084 >> 172.21.15.5:0/2135650084 conn(0x55fdb53cb300 msgr2=0x55fdb5774340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 -- 172.21.15.5:0/2135650084 shutdown_connections 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 -- 172.21.15.5:0/2135650084 wait complete. 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.249+0000 7fbde8b6d740 1 -- start start 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 Processor -- start 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5782150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5782690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb578e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdb5784d10 con 0x55fdb53ae2f0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fdb569a370 con 0x55fdb53b64d0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fdb5784e90 con 0x55fdb53d71b0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbde8b6d740 10 client.0 _ll_register_callbacks cb 0x55fdb53eea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5782150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5782150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56380/0 (socket says 172.21.15.5:56380) 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 -- 172.21.15.5:0/655968503 learned_addr learned my addr 172.21.15.5:0/655968503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddaffd640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5782690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddbfff640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb578e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 -- 172.21.15.5:0/655968503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 msgr2=0x55fdb578e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb578e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 -- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 msgr2=0x55fdb5782690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5782690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddb7fe640 1 -- 172.21.15.5:0/655968503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb54bb620 con 0x55fdb53ae2f0 2024-05-14T15:29:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.250+0000 7fbddaffd640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5782690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5782150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc103150 tx=0x7fbdcc0c62e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dce45f6eb6f67b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbdbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbdbaffd640 1 -- 172.21.15.5:0/655968503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc049070 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbdbaffd640 1 -- 172.21.15.5:0/655968503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fbdcc0e7120 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd0009e10 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbdbaffd640 1 -- 172.21.15.5:0/655968503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbdcc0c9590 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 10 client.16563 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdb5790fc0 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbdbaffd640 1 -- 172.21.15.5:0/655968503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd0009e10 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbdbaffd640 1 -- 172.21.15.5:0/655968503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbdcc0ca080 con 0x55fdb53ae2f0 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 10 client.16563 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 20 client.16563 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb5395d80 0x55fdb5794550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 4 client.16563 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.251+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb56eb1e0 con 0x55fdb5395d80 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.252+0000 7fbddaffd640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb5395d80 0x55fdb5794550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.252+0000 7fbdbaffd640 10 client.16563 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.252+0000 7fbddaffd640 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb5395d80 0x55fdb5794550 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc578b4bbdcf958 server_cookie=2dc5c5736f35e023 in_seq=0 out_seq=0 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.253+0000 7fbdbaffd640 1 -- 172.21.15.5:0/655968503 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fdb56eb1e0 con 0x55fdb5395d80 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbdbaffd640 10 client.16563 handle_command_reply: tid=0 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 client.16563 shutdown 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 msgr2=0x55fdb5782150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5782150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc103150 tx=0x7fbdcc0c62e0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 shutdown_connections 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb5395d80 0x55fdb5794550 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53d71b0 0x55fdb578e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5782690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbdbaffd640 0 client.16563 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/655968503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53ae2f0 0x55fdb5782150 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 >> 172.21.15.5:0/655968503 conn(0x55fdb53cb300 msgr2=0x55fdb5779900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 shutdown_connections 2024-05-14T15:29:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 1 -- 172.21.15.5:0/655968503 wait complete. 2024-05-14T15:29:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.254+0000 7fbde8b6d740 20 client.16563 trim_cache size 0 max 16384 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 Processor -- start 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 -- start start 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53b64d0 0x55fdb5783d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb5799d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1b3b10 con 0x55fdb53d71b0 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1b13e0 con 0x55fdb53ae2f0 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1af230 con 0x55fdb53b64d0 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbddbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb5799d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbddbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb5799d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56386/0 (socket says 172.21.15.5:56386) 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbddbfff640 1 -- 172.21.15.5:0/507320325 learned_addr learned my addr 172.21.15.5:0/507320325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.271+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddaffd640 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53b64d0 0x55fdb5783d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddbfff640 1 -- 172.21.15.5:0/507320325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53b64d0 msgr2=0x55fdb5783d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddbfff640 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53b64d0 0x55fdb5783d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddbfff640 1 -- 172.21.15.5:0/507320325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53ae2f0 msgr2=0x55fdb5778aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddbfff640 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddbfff640 1 -- 172.21.15.5:0/507320325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb5778fe0 con 0x55fdb53d71b0 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbddbfff640 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb5799d50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4014430 tx=0x7fbdd4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3496967c42c29b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbdc17fa640 1 -- 172.21.15.5:0/507320325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4004b20 con 0x55fdb53d71b0 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbdc17fa640 1 -- 172.21.15.5:0/507320325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd4003e20 con 0x55fdb53d71b0 2024-05-14T15:29:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.272+0000 7fbdc17fa640 1 -- 172.21.15.5:0/507320325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40044b0 con 0x55fdb53d71b0 2024-05-14T15:29:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 -- 172.21.15.5:0/507320325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 msgr2=0x55fdb5799d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb5799d50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4014430 tx=0x7fbdd4004ab0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 -- 172.21.15.5:0/507320325 shutdown_connections 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53b64d0 0x55fdb5783d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb5799d50 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/507320325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53ae2f0 0x55fdb5778aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 -- 172.21.15.5:0/507320325 >> 172.21.15.5:0/507320325 conn(0x55fdb53cb300 msgr2=0x55fdb5783660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 -- 172.21.15.5:0/507320325 shutdown_connections 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 -- 172.21.15.5:0/507320325 wait complete. 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.273+0000 7fbde8b6d740 1 -- start start 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 Processor -- start 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53ae2f0 0x55fdb5786e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5787370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb57878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1af8a0 con 0x55fdb53d71b0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1b0d10 con 0x55fdb53b64d0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1aff70 con 0x55fdb53ae2f0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbde8b6d740 10 client.0 _ll_register_callbacks cb 0x7fbdcc03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbddbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb57878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53ae2f0 0x55fdb5786e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbddbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb57878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56402/0 (socket says 172.21.15.5:56402) 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbddaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5787370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.274+0000 7fbddbfff640 1 -- 172.21.15.5:0/3924024016 learned_addr learned my addr 172.21.15.5:0/3924024016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddbfff640 1 -- 172.21.15.5:0/3924024016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53ae2f0 msgr2=0x55fdb5786e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddbfff640 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53ae2f0 0x55fdb5786e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddbfff640 1 -- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 msgr2=0x55fdb5787370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddbfff640 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5787370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddbfff640 1 -- 172.21.15.5:0/3924024016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb57fa9b0 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddaffd640 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5787370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbddbfff640 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb57878b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007560 tx=0x7fbdd4004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b736a241eadca46 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbdbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbdbaffd640 1 -- 172.21.15.5:0/3924024016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4004b20 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbdbaffd640 1 -- 172.21.15.5:0/3924024016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fbdd4004cc0 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd4003e20 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbdbaffd640 1 -- 172.21.15.5:0/3924024016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbdd4005050 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbde8b6d740 10 client.16569 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdb57fbc20 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.275+0000 7fbdbaffd640 1 -- 172.21.15.5:0/3924024016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd4003e20 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbdbaffd640 1 -- 172.21.15.5:0/3924024016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fbdd4030070 con 0x55fdb53d71b0 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbde8b6d740 10 client.16569 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbde8b6d740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbde8b6d740 20 client.16569 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb536b1c0 0x55fdb57fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbde8b6d740 4 client.16569 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdb56e5050 con 0x55fdb536b1c0 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb536b1c0 0x55fdb57fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbdbaffd640 10 client.16569 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.276+0000 7fbddb7fe640 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb536b1c0 0x55fdb57fe350 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1f39da2fa2efce server_cookie=262c2866a170cb95 in_seq=0 out_seq=0 2024-05-14T15:29:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:45 smithi144 ceph-mon[35650]: pgmap v152: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-14T15:29:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:45 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:29:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:45 smithi193 ceph-mon[36001]: pgmap v152: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-14T15:29:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:45 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:29:46.329 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:45 smithi005 ceph-mon[31559]: pgmap v152: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-05-14T15:29:46.329 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:45 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:29:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbdbaffd640 1 -- 172.21.15.5:0/3924024016 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fdb56e5050 con 0x55fdb536b1c0 2024-05-14T15:29:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbdbaffd640 10 client.16569 handle_command_reply: tid=0 2024-05-14T15:29:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 client.16569 shutdown 2024-05-14T15:29:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 msgr2=0x55fdb57878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb57878b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007560 tx=0x7fbdd4004440 comp rx=0 tx=0).stop 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 shutdown_connections 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fdb536b1c0 0x55fdb57fe350 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdb53d71b0 0x55fdb57878b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fdb53b64d0 0x55fdb5787370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 --2- 172.21.15.5:0/3924024016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdb53ae2f0 0x55fdb5786e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 >> 172.21.15.5:0/3924024016 conn(0x55fdb53cb300 msgr2=0x55fdb5783f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbdbaffd640 0 client.16569 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 shutdown_connections 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 1 -- 172.21.15.5:0/3924024016 wait complete. 2024-05-14T15:29:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.510+0000 7fbde8b6d740 20 client.16569 trim_cache size 0 max 16384 2024-05-14T15:29:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:29:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:29:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:29:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbdc40b2950 msgr2=0x7fbdc40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbdc40b2950 0x7fbdc40b4e60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4005cc0 tx=0x7fbdd4005c50 comp rx=0 tx=0).stop 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 msgr2=0x7fbddc1a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1a0570 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005940 tx=0x7fbdcc004270 comp rx=0 tx=0).stop 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 shutdown_connections 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbdc40b2950 0x7fbdc40b4e60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbddc10e1f0 0x7fbddc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbddc10b800 0x7fbddc1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 --2- 172.21.15.5:0/1474602790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbddc104da0 0x7fbddc1a0570 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.511+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 >> 172.21.15.5:0/1474602790 conn(0x7fbddc0782e0 msgr2=0x7fbddc1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.512+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 shutdown_connections 2024-05-14T15:29:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:46.512+0000 7fbde1be3640 1 -- 172.21.15.5:0/1474602790 wait complete. 2024-05-14T15:29:47.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 422356 11.70 MB/sec execute 68 sec latency 156.434 ms 2024-05-14T15:29:48.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 424588 11.64 MB/sec execute 69 sec latency 22.973 ms 2024-05-14T15:29:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:47 smithi144 ceph-mon[35650]: pgmap v153: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 597 op/s 2024-05-14T15:29:48.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:47 smithi193 ceph-mon[36001]: pgmap v153: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 597 op/s 2024-05-14T15:29:48.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:47 smithi005 ceph-mon[31559]: pgmap v153: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 597 op/s 2024-05-14T15:29:49.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 426465 11.65 MB/sec execute 70 sec latency 16.052 ms 2024-05-14T15:29:50.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 429460 11.74 MB/sec execute 71 sec latency 8.609 ms 2024-05-14T15:29:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:49 smithi144 ceph-mon[35650]: pgmap v154: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-14T15:29:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:49 smithi193 ceph-mon[36001]: pgmap v154: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-14T15:29:50.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:49 smithi005 ceph-mon[31559]: pgmap v154: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-14T15:29:51.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 430927 11.66 MB/sec execute 72 sec latency 85.802 ms 2024-05-14T15:29:52.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 433214 11.69 MB/sec execute 73 sec latency 21.852 ms 2024-05-14T15:29:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:51 smithi144 ceph-mon[35650]: pgmap v155: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-14T15:29:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:51 smithi193 ceph-mon[36001]: pgmap v155: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-14T15:29:52.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:51 smithi005 ceph-mon[31559]: pgmap v155: 129 pgs: 129 active+clean; 57 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-14T15:29:53.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 435746 11.70 MB/sec execute 74 sec latency 17.192 ms 2024-05-14T15:29:54.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 437421 11.66 MB/sec execute 75 sec latency 44.426 ms 2024-05-14T15:29:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:53 smithi144 ceph-mon[35650]: pgmap v156: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 465 op/s 2024-05-14T15:29:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:53 smithi193 ceph-mon[36001]: pgmap v156: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 465 op/s 2024-05-14T15:29:54.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:53 smithi005 ceph-mon[31559]: pgmap v156: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.6 MiB/s wr, 465 op/s 2024-05-14T15:29:55.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 440258 11.73 MB/sec execute 76 sec latency 8.951 ms 2024-05-14T15:29:56.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 442409 11.68 MB/sec execute 77 sec latency 70.330 ms 2024-05-14T15:29:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:55 smithi144 ceph-mon[35650]: pgmap v157: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-05-14T15:29:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:55 smithi193 ceph-mon[36001]: pgmap v157: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-05-14T15:29:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:55 smithi005 ceph-mon[31559]: pgmap v157: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 365 op/s 2024-05-14T15:29:57.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 444212 11.68 MB/sec execute 78 sec latency 16.392 ms 2024-05-14T15:29:58.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 447166 11.77 MB/sec execute 79 sec latency 9.328 ms 2024-05-14T15:29:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:57 smithi144 ceph-mon[35650]: pgmap v158: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-14T15:29:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:57 smithi193 ceph-mon[36001]: pgmap v158: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-14T15:29:58.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:57 smithi005 ceph-mon[31559]: pgmap v158: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-14T15:29:59.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 449315 11.71 MB/sec execute 80 sec latency 18.134 ms 2024-05-14T15:29:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 -- 172.21.15.5:0/3395120894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 msgr2=0x7fccb4102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 --2- 172.21.15.5:0/3395120894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb4102c80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009900 tx=0x7fcca802a410 comp rx=0 tx=0).stop 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 -- 172.21.15.5:0/3395120894 shutdown_connections 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 --2- 172.21.15.5:0/3395120894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb41032f0 0x7fccb410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 --2- 172.21.15.5:0/3395120894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb4102c80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 --2- 172.21.15.5:0/3395120894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb4108ba0 0x7fccb4102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 -- 172.21.15.5:0/3395120894 >> 172.21.15.5:0/3395120894 conn(0x7fccb40782e0 msgr2=0x7fccb4078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 -- 172.21.15.5:0/3395120894 shutdown_connections 2024-05-14T15:29:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.614+0000 7fccb9d38640 1 -- 172.21.15.5:0/3395120894 wait complete. 2024-05-14T15:29:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 Processor -- start 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 -- start start 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb41032f0 0x7fccb41a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb4108ba0 0x7fccb41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4112d90 con 0x7fccb4109040 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4112c10 con 0x7fccb41032f0 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb9d38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4112f10 con 0x7fccb4108ba0 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb41032f0 0x7fccb41a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb41032f0 0x7fccb41a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51598/0 (socket says 172.21.15.5:51598) 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32812/0 (socket says 172.21.15.5:32812) 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.615+0000 7fccb37fe640 1 -- 172.21.15.5:0/1530558845 learned_addr learned my addr 172.21.15.5:0/1530558845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb4108ba0 0x7fccb41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb3fff640 1 -- 172.21.15.5:0/1530558845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb4108ba0 msgr2=0x7fccb41a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb3fff640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb4108ba0 0x7fccb41a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb3fff640 1 -- 172.21.15.5:0/1530558845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb41032f0 msgr2=0x7fccb41a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb3fff640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb41032f0 0x7fccb41a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb3fff640 1 -- 172.21.15.5:0/1530558845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb419ae50 con 0x7fccb4109040 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb4108ba0 0x7fccb41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb3fff640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb419a720 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fccac009fd0 tx=0x7fccac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb0ff9640 1 -- 172.21.15.5:0/1530558845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac00ce20 con 0x7fccb4109040 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb419b0e0 con 0x7fccb4109040 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb0ff9640 1 -- 172.21.15.5:0/1530558845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccac005020 con 0x7fccb4109040 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb0ff9640 1 -- 172.21.15.5:0/1530558845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac010690 con 0x7fccb4109040 2024-05-14T15:29:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.616+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb41a7340 con 0x7fccb4109040 2024-05-14T15:29:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 Processor -- start 2024-05-14T15:29:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccb0ff9640 1 -- 172.21.15.5:0/1530558845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fccac010830 con 0x7fccb4109040 2024-05-14T15:29:59.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 -- start start 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39811b0 0x559fb3d21be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39582f0 0x559fb3d22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d22660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fb3c875d0 con 0x559fb39604d0 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559fb3c40890 con 0x559fb39811b0 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.619+0000 7fccc0bee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fb3c83590 con 0x559fb39582f0 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d22660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39582f0 0x559fb3d22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d22660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32814/0 (socket says 172.21.15.5:32814) 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb2ffd640 1 -- 172.21.15.5:0/2918123233 learned_addr learned my addr 172.21.15.5:0/2918123233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39811b0 0x559fb3d21be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb0ff9640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc9c0b2950 0x7fcc9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 -- 172.21.15.5:0/2918123233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39811b0 msgr2=0x559fb3d21be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39811b0 0x559fb3d21be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 -- 172.21.15.5:0/2918123233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 msgr2=0x559fb3d22660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d22660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb3fff640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc9c0b2950 0x7fcc9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 -- 172.21.15.5:0/2918123233 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb3999210 con 0x559fb39582f0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d22660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb0ff9640 1 -- 172.21.15.5:0/1530558845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fccac0e3110 con 0x7fccb4109040 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb37fe640 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39582f0 0x559fb3d22120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcca40058c0 tx=0x7fcca4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5874354008d885d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fcc99ffb640 1 -- 172.21.15.5:0/2918123233 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4009da0 con 0x559fb39582f0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.620+0000 7fccb3fff640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc9c0b2950 0x7fcc9c0b4e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fccac0a9d30 tx=0x7fccac0a9e80 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fcc99ffb640 1 -- 172.21.15.5:0/2918123233 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca40054d0 con 0x559fb39582f0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fcc99ffb640 1 -- 172.21.15.5:0/2918123233 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca4019410 con 0x559fb39582f0 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 -- 172.21.15.5:0/2918123233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39582f0 msgr2=0x559fb3d22120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39582f0 0x559fb3d22120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcca40058c0 tx=0x7fcca4011810 comp rx=0 tx=0).stop 2024-05-14T15:29:59.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 -- 172.21.15.5:0/2918123233 shutdown_connections 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39582f0 0x559fb3d22120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d22660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 --2- 172.21.15.5:0/2918123233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39811b0 0x559fb3d21be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 -- 172.21.15.5:0/2918123233 >> 172.21.15.5:0/2918123233 conn(0x559fb3975300 msgr2=0x559fb3d1e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 -- 172.21.15.5:0/2918123233 shutdown_connections 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.621+0000 7fccc0bee740 1 -- 172.21.15.5:0/2918123233 wait complete. 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 -- start start 2024-05-14T15:29:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 Processor -- start 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39604d0 0x559fb3d2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39811b0 0x559fb3d32190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fb3d2ed10 con 0x559fb39811b0 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559fb3c44370 con 0x559fb39582f0 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fb3d2ee90 con 0x559fb39604d0 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.622+0000 7fccc0bee740 10 client.0 _ll_register_callbacks cb 0x559fb3998a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51618/0 (socket says 172.21.15.5:51618) 2024-05-14T15:29:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 -- 172.21.15.5:0/781938476 learned_addr learned my addr 172.21.15.5:0/781938476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39811b0 0x559fb3d32190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb37fe640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39604d0 0x559fb3d2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 -- 172.21.15.5:0/781938476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39604d0 msgr2=0x559fb3d2c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39604d0 0x559fb3d2c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 -- 172.21.15.5:0/781938476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39811b0 msgr2=0x559fb3d32190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39811b0 0x559fb3d32190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 -- 172.21.15.5:0/781938476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb3a65620 con 0x559fb39582f0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2c230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009230 tx=0x7fcca802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a65ddf662083d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fcc92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fcc92ffd640 1 -- 172.21.15.5:0/781938476 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8041020 con 0x559fb39582f0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca40054d0 con 0x559fb39582f0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.623+0000 7fcc92ffd640 1 -- 172.21.15.5:0/781938476 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fcca8004260 con 0x559fb39582f0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fcc92ffd640 1 -- 172.21.15.5:0/781938476 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcca8052050 con 0x559fb39582f0 2024-05-14T15:29:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 10 client.26201 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fb3d3b110 con 0x559fb39582f0 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fcc92ffd640 1 -- 172.21.15.5:0/781938476 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca40054d0 con 0x559fb39582f0 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fcc92ffd640 1 -- 172.21.15.5:0/781938476 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcca803a040 con 0x559fb39582f0 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 10 client.26201 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 10 client.26201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 20 client.26201 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb393fd80 0x559fb3d3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 4 client.26201 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fb3c951e0 con 0x559fb393fd80 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.624+0000 7fccb37fe640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb393fd80 0x559fb3d3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.625+0000 7fccb37fe640 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb393fd80 0x559fb3d3e590 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0df7d477ddcb84 server_cookie=c1744a216d8fa776 in_seq=0 out_seq=0 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.625+0000 7fcc92ffd640 10 client.26201 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.626+0000 7fcc92ffd640 1 -- 172.21.15.5:0/781938476 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559fb3c951e0 con 0x559fb393fd80 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.626+0000 7fcc92ffd640 10 client.26201 handle_command_reply: tid=0 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.626+0000 7fccc0bee740 1 client.26201 shutdown 2024-05-14T15:29:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 msgr2=0x559fb3d2c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2c230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009230 tx=0x7fcca802a410 comp rx=0 tx=0).stop 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 shutdown_connections 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb393fd80 0x559fb3d3e590 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb39604d0 0x559fb3d2c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39811b0 0x559fb3d32190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fcc92ffd640 0 client.26201 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 --2- 172.21.15.5:0/781938476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2c230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 >> 172.21.15.5:0/781938476 conn(0x559fb3975300 msgr2=0x559fb3d21a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 shutdown_connections 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 1 -- 172.21.15.5:0/781938476 wait complete. 2024-05-14T15:29:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.627+0000 7fccc0bee740 20 client.26201 trim_cache size 0 max 16384 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 Processor -- start 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 -- start start 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41b3c90 con 0x559fb39604d0 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41b1560 con 0x559fb39582f0 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccc0bee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41af3b0 con 0x559fb3975300 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d2e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d2e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32832/0 (socket says 172.21.15.5:32832) 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccb2ffd640 1 -- 172.21.15.5:0/631844152 learned_addr learned my addr 172.21.15.5:0/631844152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.642+0000 7fccb37fe640 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 -- 172.21.15.5:0/631844152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 msgr2=0x559fb3d223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d223a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 -- 172.21.15.5:0/631844152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 msgr2=0x559fb3d2de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 -- 172.21.15.5:0/631844152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb3d2a570 con 0x559fb39604d0 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb37fe640 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d2e390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcca803d740 tx=0x7fcca8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb9e9e0246bedfa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fcc997fa640 1 -- 172.21.15.5:0/631844152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca804f020 con 0x559fb39604d0 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fcc997fa640 1 -- 172.21.15.5:0/631844152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca8042040 con 0x559fb39604d0 2024-05-14T15:29:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.643+0000 7fcc997fa640 1 -- 172.21.15.5:0/631844152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8041040 con 0x559fb39604d0 2024-05-14T15:29:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 -- 172.21.15.5:0/631844152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 msgr2=0x559fb3d2e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d2e390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcca803d740 tx=0x7fcca8031400 comp rx=0 tx=0).stop 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 -- 172.21.15.5:0/631844152 shutdown_connections 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d2e390 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 --2- 172.21.15.5:0/631844152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 -- 172.21.15.5:0/631844152 >> 172.21.15.5:0/631844152 conn(0x559fb3d260a0 msgr2=0x559fb3d2d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 -- 172.21.15.5:0/631844152 shutdown_connections 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 -- 172.21.15.5:0/631844152 wait complete. 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.644+0000 7fccc0bee740 1 -- start start 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 Processor -- start 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d38950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d33ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41afa20 con 0x559fb39604d0 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41b0e90 con 0x559fb39582f0 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41b00f0 con 0x559fb3975300 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccc0bee740 10 client.0 _ll_register_callbacks cb 0x7fcca8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d38950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d33ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d38950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51622/0 (socket says 172.21.15.5:51622) 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d33ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32842/0 (socket says 172.21.15.5:32842) 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb2ffd640 1 -- 172.21.15.5:0/1058611368 learned_addr learned my addr 172.21.15.5:0/1058611368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 -- 172.21.15.5:0/1058611368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 msgr2=0x559fb3d341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 -- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 msgr2=0x559fb3d38950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d38950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.645+0000 7fccb37fe640 1 -- 172.21.15.5:0/1058611368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb3d347a0 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccb37fe640 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d33ca0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x559fb3d1f320 tx=0x7fcca4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20847dd1b2af37b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fcc92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fcc92ffd640 1 -- 172.21.15.5:0/1058611368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca40052c0 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fcc92ffd640 1 -- 172.21.15.5:0/1058611368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fcca4016040 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca8042040 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fcc92ffd640 1 -- 172.21.15.5:0/1058611368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcca4005460 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 10 client.16587 fetch_fsmap learned FSMap version 30 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fb3da6630 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fcc92ffd640 1 -- 172.21.15.5:0/1058611368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca8042040 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fcc92ffd640 1 -- 172.21.15.5:0/1058611368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcca4003890 con 0x559fb39604d0 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 10 client.16587 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:29:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 20 client.16587 populate_metadata read hostname 'smithi005' 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb39811b0 0x559fb3da8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 4 client.16587 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.646+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fb3c8f050 con 0x559fb39811b0 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.647+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb39811b0 0x559fb3da8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.647+0000 7fccb2ffd640 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb39811b0 0x559fb3da8c80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d81e0c7fd91b1bb server_cookie=f0bb334a615fbc1a in_seq=0 out_seq=0 2024-05-14T15:29:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.647+0000 7fcc92ffd640 10 client.16587 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fcc92ffd640 1 -- 172.21.15.5:0/1058611368 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fb3c8f050 con 0x559fb39811b0 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fcc92ffd640 10 client.16587 handle_command_reply: tid=0 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fccc0bee740 1 client.16587 shutdown 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 msgr2=0x559fb3d33ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fccc0bee740 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d33ca0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x559fb3d1f320 tx=0x7fcca4009860 comp rx=0 tx=0).stop 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 shutdown_connections 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.932+0000 7fccc0bee740 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559fb39811b0 0x559fb3da8c80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fb3975300 0x559fb3d341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fb39604d0 0x559fb3d33ca0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 1 --2- 172.21.15.5:0/1058611368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fb39582f0 0x559fb3d38950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 >> 172.21.15.5:0/1058611368 conn(0x559fb3d260a0 msgr2=0x559fb3d22100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fcc92ffd640 0 client.16587 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:29:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 shutdown_connections 2024-05-14T15:29:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 1 -- 172.21.15.5:0/1058611368 wait complete. 2024-05-14T15:29:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.933+0000 7fccc0bee740 20 client.16587 trim_cache size 0 max 16384 2024-05-14T15:29:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:29:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:29:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:29:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc9c0b2950 msgr2=0x7fcc9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc9c0b2950 0x7fcc9c0b4e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fccac0a9d30 tx=0x7fccac0a9e80 comp rx=0 tx=0).stop 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 msgr2=0x7fccb419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb419a720 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fccac009fd0 tx=0x7fccac00ef10 comp rx=0 tx=0).stop 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 shutdown_connections 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcc9c0b2950 0x7fcc9c0b4e60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccb4108ba0 0x7fccb41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb4109040 0x7fccb419a720 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 --2- 172.21.15.5:0/1530558845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccb41032f0 0x7fccb41a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:29:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 >> 172.21.15.5:0/1530558845 conn(0x7fccb40782e0 msgr2=0x7fccb4106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:29:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 shutdown_connections 2024-05-14T15:29:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:29:59.934+0000 7fccb9d38640 1 -- 172.21.15.5:0/1530558845 wait complete. 2024-05-14T15:30:00.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 450917 11.71 MB/sec execute 81 sec latency 114.608 ms 2024-05-14T15:30:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:29:59 smithi144 ceph-mon[35650]: pgmap v159: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-05-14T15:30:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:29:59 smithi193 ceph-mon[36001]: pgmap v159: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-05-14T15:30:00.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:29:59 smithi005 ceph-mon[31559]: pgmap v159: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-05-14T15:30:01.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 451526 11.59 MB/sec execute 82 sec latency 138.105 ms 2024-05-14T15:30:01.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:00 smithi144 ceph-mon[35650]: overall HEALTH_OK 2024-05-14T15:30:01.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:00 smithi193 ceph-mon[36001]: overall HEALTH_OK 2024-05-14T15:30:01.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:00 smithi005 ceph-mon[31559]: overall HEALTH_OK 2024-05-14T15:30:02.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 453373 11.57 MB/sec execute 83 sec latency 69.835 ms 2024-05-14T15:30:02.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:01 smithi144 ceph-mon[35650]: pgmap v160: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-05-14T15:30:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:01 smithi193 ceph-mon[36001]: pgmap v160: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-05-14T15:30:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:01 smithi005 ceph-mon[31559]: pgmap v160: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-05-14T15:30:03.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 455173 11.55 MB/sec execute 84 sec latency 16.924 ms 2024-05-14T15:30:04.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 457953 11.61 MB/sec execute 85 sec latency 12.742 ms 2024-05-14T15:30:04.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:03 smithi005 ceph-mon[31559]: pgmap v161: 129 pgs: 129 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 529 op/s 2024-05-14T15:30:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:03 smithi144 ceph-mon[35650]: pgmap v161: 129 pgs: 129 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 529 op/s 2024-05-14T15:30:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:03 smithi193 ceph-mon[36001]: pgmap v161: 129 pgs: 129 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 529 op/s 2024-05-14T15:30:05.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 459500 11.75 MB/sec execute 86 sec latency 34.037 ms 2024-05-14T15:30:06.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 462354 11.81 MB/sec execute 87 sec latency 11.485 ms 2024-05-14T15:30:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:05 smithi144 ceph-mon[35650]: pgmap v162: 129 pgs: 129 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 409 op/s 2024-05-14T15:30:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:05 smithi193 ceph-mon[36001]: pgmap v162: 129 pgs: 129 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 409 op/s 2024-05-14T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:05 smithi005 ceph-mon[31559]: pgmap v162: 129 pgs: 129 active+clean; 55 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 409 op/s 2024-05-14T15:30:07.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 464238 11.75 MB/sec execute 88 sec latency 63.197 ms 2024-05-14T15:30:08.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466039 11.76 MB/sec execute 89 sec latency 189.075 ms 2024-05-14T15:30:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:07 smithi144 ceph-mon[35650]: pgmap v163: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 522 op/s 2024-05-14T15:30:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:07 smithi193 ceph-mon[36001]: pgmap v163: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 522 op/s 2024-05-14T15:30:08.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:07 smithi005 ceph-mon[31559]: pgmap v163: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 522 op/s 2024-05-14T15:30:09.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466039 11.63 MB/sec execute 90 sec latency 1189.106 ms 2024-05-14T15:30:10.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466039 11.50 MB/sec execute 91 sec latency 2189.159 ms 2024-05-14T15:30:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:09 smithi144 ceph-mon[35650]: pgmap v164: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:30:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:09 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:30:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:09 smithi193 ceph-mon[36001]: pgmap v164: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:30:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:09 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:30:10.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:09 smithi005 ceph-mon[31559]: pgmap v164: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:30:10.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:09 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:30:11.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466039 11.38 MB/sec execute 92 sec latency 3189.189 ms 2024-05-14T15:30:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:10 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:30:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:10 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:30:11.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:10 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:30:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:10 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:30:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:10 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:30:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:10 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:30:11.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:10 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:30:11.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:10 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:30:11.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:10 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:30:12.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466039 11.25 MB/sec execute 93 sec latency 4189.264 ms 2024-05-14T15:30:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:11 smithi144 ceph-mon[35650]: pgmap v165: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:30:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:11 smithi193 ceph-mon[36001]: pgmap v165: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:30:12.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:11 smithi005 ceph-mon[31559]: pgmap v165: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:30:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 -- 172.21.15.5:0/1842891946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b01084e0 msgr2=0x7fb1b01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 --2- 172.21.15.5:0/1842891946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b01084e0 0x7fb1b01088f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009900 tx=0x7fb1a402a410 comp rx=0 tx=0).stop 2024-05-14T15:30:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 -- 172.21.15.5:0/1842891946 shutdown_connections 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 --2- 172.21.15.5:0/1842891946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b0108f80 0x7fb1b0110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 --2- 172.21.15.5:0/1842891946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b01084e0 0x7fb1b01088f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 --2- 172.21.15.5:0/1842891946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b0102460 0x7fb1b0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 -- 172.21.15.5:0/1842891946 >> 172.21.15.5:0/1842891946 conn(0x7fb1b00fd9f0 msgr2=0x7fb1b00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 -- 172.21.15.5:0/1842891946 shutdown_connections 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.054+0000 7fb1b6776640 1 -- 172.21.15.5:0/1842891946 wait complete. 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 Processor -- start 2024-05-14T15:30:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 -- start start 2024-05-14T15:30:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 0x7fb1b01a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b01084e0 0x7fb1b01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b0108f80 0x7fb1b019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0113e90 con 0x7fb1b0102460 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0113d10 con 0x7fb1b0108f80 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1b6776640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0114010 con 0x7fb1b01084e0 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.055+0000 7fb1affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 0x7fb1b01a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 0x7fb1b01a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44652/0 (socket says 172.21.15.5:44652) 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 -- 172.21.15.5:0/775554733 learned_addr learned my addr 172.21.15.5:0/775554733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b0108f80 0x7fb1b019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b01084e0 0x7fb1b01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 -- 172.21.15.5:0/775554733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b01084e0 msgr2=0x7fb1b01a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b01084e0 0x7fb1b01a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 -- 172.21.15.5:0/775554733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b0108f80 msgr2=0x7fb1b019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b0108f80 0x7fb1b019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 -- 172.21.15.5:0/775554733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b019ae80 con 0x7fb1b0102460 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1affff640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 0x7fb1b01a05d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d770 tx=0x7fb1a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1ad7fa640 1 -- 172.21.15.5:0/775554733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0017070 con 0x7fb1b0102460 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1ad7fa640 1 -- 172.21.15.5:0/775554733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a0009c30 con 0x7fb1b0102460 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b019b110 con 0x7fb1b0102460 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1ad7fa640 1 -- 172.21.15.5:0/775554733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0003cc0 con 0x7fb1b0102460 2024-05-14T15:30:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.056+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b01a7370 con 0x7fb1b0102460 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.059+0000 7fb1bd62f740 1 Processor -- start 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.059+0000 7fb1bd62f740 1 -- start start 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.059+0000 7fb1ad7fa640 1 -- 172.21.15.5:0/775554733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb1a0003e60 con 0x7fb1b0102460 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77f31b0 0x5611a7b93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b94120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7b94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611a7af95d0 con 0x5611a77d24d0 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5611a7ab2890 con 0x5611a77f31b0 2024-05-14T15:30:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5611a7af5590 con 0x5611a77ca2f0 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7b94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7b94660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44666/0 (socket says 172.21.15.5:44666) 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 -- 172.21.15.5:0/1424298358 learned_addr learned my addr 172.21.15.5:0/1424298358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1affff640 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b94120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77f31b0 0x5611a7b93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 -- 172.21.15.5:0/1424298358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 msgr2=0x5611a7b94120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b94120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 -- 172.21.15.5:0/1424298358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77f31b0 msgr2=0x5611a7b93be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77f31b0 0x5611a7b93be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 -- 172.21.15.5:0/1424298358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a780b210 con 0x5611a77d24d0 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77f31b0 0x5611a7b93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1ad7fa640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1980b2950 0x7fb1980b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.060+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7b94660 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4000c00 tx=0x7fb1a4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ac85ee986b7e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.061+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1980b2950 0x7fb1980b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.061+0000 7fb1ad7fa640 1 -- 172.21.15.5:0/775554733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb1a00e2480 con 0x7fb1b0102460 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.061+0000 7fb1967fc640 1 -- 172.21.15.5:0/1424298358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a403f070 con 0x5611a77d24d0 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.061+0000 7fb1967fc640 1 -- 172.21.15.5:0/1424298358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a4032e40 con 0x5611a77d24d0 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.061+0000 7fb1967fc640 1 -- 172.21.15.5:0/1424298358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4035070 con 0x5611a77d24d0 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 -- 172.21.15.5:0/1424298358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 msgr2=0x5611a7b94660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1980b2950 0x7fb1980b4e60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80092f0 tx=0x7fb1a8009280 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7b94660 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4000c00 tx=0x7fb1a4032c50 comp rx=0 tx=0).stop 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 -- 172.21.15.5:0/1424298358 shutdown_connections 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b94120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7b94660 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/1424298358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77f31b0 0x5611a7b93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 -- 172.21.15.5:0/1424298358 >> 172.21.15.5:0/1424298358 conn(0x5611a77e7300 msgr2=0x5611a7b90340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 -- 172.21.15.5:0/1424298358 shutdown_connections 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 -- 172.21.15.5:0/1424298358 wait complete. 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.062+0000 7fb1bd62f740 1 -- start start 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 Processor -- start 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7b9e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7baa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611a7ba0cb0 con 0x5611a77f31b0 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5611a7ab6370 con 0x5611a77d24d0 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5611a7ba0e30 con 0x5611a77ca2f0 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1bd62f740 10 client.0 _ll_register_callbacks cb 0x5611a780aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7baa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7baa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44668/0 (socket says 172.21.15.5:44668) 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 -- 172.21.15.5:0/257638502 learned_addr learned my addr 172.21.15.5:0/257638502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7b9e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1affff640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 -- 172.21.15.5:0/257638502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 msgr2=0x5611a7b9e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 -- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 msgr2=0x5611a7b9e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7b9e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1b4d13640 1 -- 172.21.15.5:0/257638502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a78d7620 con 0x5611a77f31b0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.063+0000 7fb1affff640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7baa160 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8012870 tx=0x7fb1a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2bc0c1d44c5e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb18f7fe640 1 -- 172.21.15.5:0/257638502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800ead0 con 0x5611a77f31b0 2024-05-14T15:30:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb18f7fe640 1 -- 172.21.15.5:0/257638502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb1a800ec70 con 0x5611a77f31b0 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611a7b071e0 con 0x5611a77f31b0 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb18f7fe640 1 -- 172.21.15.5:0/257638502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1a801f420 con 0x5611a77f31b0 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1bd62f740 10 client.16599 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611a7bad040 con 0x5611a77f31b0 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb18f7fe640 1 -- 172.21.15.5:0/257638502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5611a7b071e0 con 0x5611a77f31b0 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb18f7fe640 1 -- 172.21.15.5:0/257638502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb1a800f040 con 0x5611a77f31b0 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1bd62f740 10 client.16599 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1bd62f740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.064+0000 7fb1bd62f740 20 client.16599 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.065+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77b1d80 0x5611a7bb0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.065+0000 7fb1bd62f740 4 client.16599 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.065+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611a7b01050 con 0x5611a77b1d80 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.065+0000 7fb1affff640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77b1d80 0x5611a7bb0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.065+0000 7fb1affff640 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77b1d80 0x5611a7bb0550 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dabf306111300a8 server_cookie=793e1600d136a3e6 in_seq=0 out_seq=0 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.065+0000 7fb18f7fe640 10 client.16599 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.066+0000 7fb18f7fe640 1 -- 172.21.15.5:0/257638502 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611a7b01050 con 0x5611a77b1d80 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.066+0000 7fb18f7fe640 10 client.16599 handle_command_reply: tid=0 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.066+0000 7fb1bd62f740 1 client.16599 shutdown 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.066+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 msgr2=0x5611a7baa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.066+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7baa160 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8012870 tx=0x7fb1a8004a40 comp rx=0 tx=0).stop 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 shutdown_connections 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77b1d80 0x5611a7bb0550 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7baa160 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7b9e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/257638502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb18f7fe640 0 client.16599 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 >> 172.21.15.5:0/257638502 conn(0x5611a77e7300 msgr2=0x5611a7b95900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 shutdown_connections 2024-05-14T15:30:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 1 -- 172.21.15.5:0/257638502 wait complete. 2024-05-14T15:30:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.067+0000 7fb1bd62f740 20 client.16599 trim_cache size 0 max 16384 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 Processor -- start 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 -- start start 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7bb5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7bb6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01b3be0 con 0x5611a77f31b0 2024-05-14T15:30:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01b14b0 con 0x5611a77d24d0 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01af300 con 0x5611a77ca2f0 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1b4d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7bb6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1b4d13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7bb6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.082+0000 7fb1b4d13640 1 -- 172.21.15.5:0/3201224948 learned_addr learned my addr 172.21.15.5:0/3201224948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1affff640 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7bb5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1b4d13640 1 -- 172.21.15.5:0/3201224948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 msgr2=0x5611a7b9cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1b4d13640 1 -- 172.21.15.5:0/3201224948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 msgr2=0x5611a7bb5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7bb5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1b4d13640 1 -- 172.21.15.5:0/3201224948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a7b8b9a0 con 0x5611a77f31b0 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1affff640 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7bb6290 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8011cb0 tx=0x7fb1a8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6726ba9ca5b494 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb195ffb640 1 -- 172.21.15.5:0/3201224948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8027b50 con 0x5611a77f31b0 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb195ffb640 1 -- 172.21.15.5:0/3201224948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a8027cf0 con 0x5611a77f31b0 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.083+0000 7fb195ffb640 1 -- 172.21.15.5:0/3201224948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a801fba0 con 0x5611a77f31b0 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 -- 172.21.15.5:0/3201224948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 msgr2=0x5611a7bb6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7bb6290 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8011cb0 tx=0x7fb1a8011de0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 -- 172.21.15.5:0/3201224948 shutdown_connections 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77f31b0 0x5611a7bb6290 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77d24d0 0x5611a7bb5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/3201224948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77ca2f0 0x5611a7b9cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 -- 172.21.15.5:0/3201224948 >> 172.21.15.5:0/3201224948 conn(0x5611a77e7300 msgr2=0x5611a7b9c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 -- 172.21.15.5:0/3201224948 shutdown_connections 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 -- 172.21.15.5:0/3201224948 wait complete. 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.084+0000 7fb1bd62f740 1 -- start start 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 Processor -- start 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77ca2f0 0x5611a7ba6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7ba1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77f31b0 0x5611a7ba1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01af970 con 0x5611a77d24d0 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01b0de0 con 0x5611a77ca2f0 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01b0040 con 0x5611a77f31b0 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1bd62f740 10 client.0 _ll_register_callbacks cb 0x7fb1a8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7ba1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7ba1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44692/0 (socket says 172.21.15.5:44692) 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 -- 172.21.15.5:0/835863048 learned_addr learned my addr 172.21.15.5:0/835863048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1affff640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77ca2f0 0x5611a7ba6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77f31b0 0x5611a7ba1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 -- 172.21.15.5:0/835863048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77f31b0 msgr2=0x5611a7ba1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77f31b0 0x5611a7ba1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 -- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77ca2f0 msgr2=0x5611a7ba6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77ca2f0 0x5611a7ba6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1af7fe640 1 -- 172.21.15.5:0/835863048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a7ba2410 con 0x5611a77d24d0 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.085+0000 7fb1b4d13640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77f31b0 0x5611a7ba1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1af7fe640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7ba1890 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4033000 tx=0x7fb1a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6c5d251eff3954 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb18f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb18f7fe640 1 -- 172.21.15.5:0/835863048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a403f070 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb18f7fe640 1 -- 172.21.15.5:0/835863048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb1a4005230 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a8027cf0 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb18f7fe640 1 -- 172.21.15.5:0/835863048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1a4054050 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 10 client.16605 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611a7c1ca80 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb18f7fe640 1 -- 172.21.15.5:0/835863048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a8027cf0 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb18f7fe640 1 -- 172.21.15.5:0/835863048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb1a4004c00 con 0x5611a77d24d0 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 10 client.16605 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 20 client.16605 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77871c0 0x5611a7c1edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 4 client.16605 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:30:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.086+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611a7a7c5f0 con 0x5611a77871c0 2024-05-14T15:30:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.087+0000 7fb1affff640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77871c0 0x5611a7c1edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.087+0000 7fb1affff640 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77871c0 0x5611a7c1edc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb22271ff1e87293 server_cookie=a4880df22e7993e9 in_seq=0 out_seq=0 2024-05-14T15:30:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.087+0000 7fb18f7fe640 10 client.16605 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:13.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466039 11.13 MB/sec execute 94 sec latency 5189.312 ms 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.365+0000 7fb18f7fe640 1 -- 172.21.15.5:0/835863048 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611a7a7c5f0 con 0x5611a77871c0 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.365+0000 7fb18f7fe640 10 client.16605 handle_command_reply: tid=0 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.365+0000 7fb1bd62f740 1 client.16605 shutdown 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 msgr2=0x5611a7ba1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7ba1890 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4033000 tx=0x7fb1a4004270 comp rx=0 tx=0).stop 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 shutdown_connections 2024-05-14T15:30:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5611a77871c0 0x5611a7c1edc0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611a77f31b0 0x5611a7ba1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611a77d24d0 0x5611a7ba1890 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb18f7fe640 0 client.16605 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 --2- 172.21.15.5:0/835863048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611a77ca2f0 0x5611a7ba6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 >> 172.21.15.5:0/835863048 conn(0x5611a77e7300 msgr2=0x5611a7b9f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 shutdown_connections 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 1 -- 172.21.15.5:0/835863048 wait complete. 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.366+0000 7fb1bd62f740 20 client.16605 trim_cache size 0 max 16384 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:30:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:30:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:30:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1980b2950 msgr2=0x7fb1980b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1980b2950 0x7fb1980b4e60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80092f0 tx=0x7fb1a8009280 comp rx=0 tx=0).stop 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 msgr2=0x7fb1b01a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 0x7fb1b01a05d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d770 tx=0x7fb1a000dc30 comp rx=0 tx=0).stop 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 shutdown_connections 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1980b2950 0x7fb1980b4e60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1b01084e0 0x7fb1b01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1b0108f80 0x7fb1b019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 --2- 172.21.15.5:0/775554733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1b0102460 0x7fb1b01a05d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 >> 172.21.15.5:0/775554733 conn(0x7fb1b00fd9f0 msgr2=0x7fb1b010f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 shutdown_connections 2024-05-14T15:30:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:13.367+0000 7fb1b6776640 1 -- 172.21.15.5:0/775554733 wait complete. 2024-05-14T15:30:14.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 467315 11.07 MB/sec execute 95 sec latency 5323.377 ms 2024-05-14T15:30:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:13 smithi144 ceph-mon[35650]: pgmap v166: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 475 op/s 2024-05-14T15:30:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:13 smithi193 ceph-mon[36001]: pgmap v166: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 475 op/s 2024-05-14T15:30:14.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:13 smithi005 ceph-mon[31559]: pgmap v166: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 475 op/s 2024-05-14T15:30:14.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:30:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:15.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 469659 11.10 MB/sec execute 96 sec latency 19.121 ms 2024-05-14T15:30:16.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 471987 11.12 MB/sec execute 97 sec latency 16.221 ms 2024-05-14T15:30:16.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:15 smithi005 ceph-mon[31559]: pgmap v167: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-05-14T15:30:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:15 smithi144 ceph-mon[35650]: pgmap v167: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-05-14T15:30:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:15 smithi193 ceph-mon[36001]: pgmap v167: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.837+0000 7f8e44544700 1 -- 172.21.15.5:0/4083180665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c107e30 msgr2=0x7f8e3c108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.837+0000 7f8e44544700 1 --2- 172.21.15.5:0/4083180665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c107e30 0x7f8e3c108240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e300099c0 tx=0x7f8e30009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 -- 172.21.15.5:0/4083180665 shutdown_connections 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 --2- 172.21.15.5:0/4083180665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c108780 0x7f8e3c1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 --2- 172.21.15.5:0/4083180665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c103550 0x7f8e3c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 --2- 172.21.15.5:0/4083180665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c107e30 0x7f8e3c108240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 -- 172.21.15.5:0/4083180665 >> 172.21.15.5:0/4083180665 conn(0x7f8e3c078e00 msgr2=0x7f8e3c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 -- 172.21.15.5:0/4083180665 shutdown_connections 2024-05-14T15:30:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.838+0000 7f8e44544700 1 -- 172.21.15.5:0/4083180665 wait complete. 2024-05-14T15:30:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.839+0000 7f8e44544700 1 Processor -- start 2024-05-14T15:30:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.839+0000 7f8e44544700 1 -- start start 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.839+0000 7f8e44544700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c103550 0x7f8e3c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e44544700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c107e30 0x7f8e3c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e44544700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 0x7f8e3c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e44544700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c114550 con 0x7f8e3c107e30 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e44544700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1143d0 con 0x7f8e3c103550 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e44544700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1146d0 con 0x7f8e3c108780 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 0x7f8e3c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 0x7f8e3c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52562/0 (socket says 172.21.15.5:52562) 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 -- 172.21.15.5:0/1387696401 learned_addr learned my addr 172.21.15.5:0/1387696401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e422e0700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c103550 0x7f8e3c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 -- 172.21.15.5:0/1387696401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c103550 msgr2=0x7f8e3c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c103550 0x7f8e3c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 -- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c107e30 msgr2=0x7f8e3c19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c107e30 0x7f8e3c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.840+0000 7f8e42ae1700 1 -- 172.21.15.5:0/1387696401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c19fdc0 con 0x7f8e3c108780 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.841+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 0x7f8e3c19f720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400b970 tx=0x7f8e3400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.841+0000 7f8e2f7fe700 1 -- 172.21.15.5:0/1387696401 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34019070 con 0x7f8e3c108780 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.841+0000 7f8e2f7fe700 1 -- 172.21.15.5:0/1387696401 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e34014440 con 0x7f8e3c108780 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.841+0000 7f8e2f7fe700 1 -- 172.21.15.5:0/1387696401 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400ce60 con 0x7f8e3c108780 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.842+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c1a00b0 con 0x7f8e3c108780 2024-05-14T15:30:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.842+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c1a0540 con 0x7f8e3c108780 2024-05-14T15:30:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.848+0000 7f8e2f7fe700 1 -- 172.21.15.5:0/1387696401 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8e3401c070 con 0x7f8e3c108780 2024-05-14T15:30:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.850+0000 7f8e2f7fe700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8e280b2bc0 0x7f8e280b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.850+0000 7f8e2f7fe700 1 -- 172.21.15.5:0/1387696401 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f8e340e1d10 con 0x7f8e3c108780 2024-05-14T15:30:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.850+0000 7f8e422e0700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8e280b2bc0 0x7f8e280b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.850+0000 7f8e52f58b80 1 Processor -- start 2024-05-14T15:30:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 -- start start 2024-05-14T15:30:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307395e40 0x55e307521bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307515470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e422e0700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8e280b2bc0 0x7f8e280b50d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009bf0 tx=0x7f8e38009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e307517dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e307435860 con 0x55e3075159b0 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e3073d47c0 con 0x55e307395e40 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.851+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e307431820 con 0x55e307522100 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.852+0000 7f8e41adf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307515470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.852+0000 7f8e41adf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307515470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52574/0 (socket says 172.21.15.5:52574) 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.852+0000 7f8e41adf700 1 -- 172.21.15.5:0/2325720846 learned_addr learned my addr 172.21.15.5:0/2325720846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.852+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e307517dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.852+0000 7f8e422e0700 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307395e40 0x55e307521bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.853+0000 7f8e42ae1700 1 -- 172.21.15.5:0/2325720846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 msgr2=0x55e307515470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.853+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307515470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.853+0000 7f8e42ae1700 1 -- 172.21.15.5:0/2325720846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307395e40 msgr2=0x55e307521bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.853+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307395e40 0x55e307521bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.853+0000 7f8e42ae1700 1 -- 172.21.15.5:0/2325720846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3073828d0 con 0x55e3075159b0 2024-05-14T15:30:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.853+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e307517dd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8e340df150 tx=0x7f8e34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44699b50d102638a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.854+0000 7f8e2cff9700 1 -- 172.21.15.5:0/2325720846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340ad560 con 0x55e3075159b0 2024-05-14T15:30:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.854+0000 7f8e2cff9700 1 -- 172.21.15.5:0/2325720846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e340b2010 con 0x55e3075159b0 2024-05-14T15:30:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.854+0000 7f8e2cff9700 1 -- 172.21.15.5:0/2325720846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340a9670 con 0x55e3075159b0 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.857+0000 7f8e52f58b80 1 -- 172.21.15.5:0/2325720846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 msgr2=0x55e307517dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.857+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e307517dd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8e340df150 tx=0x7f8e34004ad0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.857+0000 7f8e52f58b80 1 -- 172.21.15.5:0/2325720846 shutdown_connections 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.857+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e307517dd0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.858+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307395e40 0x55e307521bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.858+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/2325720846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307515470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.858+0000 7f8e52f58b80 1 -- 172.21.15.5:0/2325720846 >> 172.21.15.5:0/2325720846 conn(0x55e3073dd600 msgr2=0x55e30750e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.858+0000 7f8e52f58b80 1 -- 172.21.15.5:0/2325720846 shutdown_connections 2024-05-14T15:30:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.859+0000 7f8e52f58b80 1 -- 172.21.15.5:0/2325720846 wait complete. 2024-05-14T15:30:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.859+0000 7f8e52f58b80 1 -- start start 2024-05-14T15:30:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 Processor -- start 2024-05-14T15:30:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e307510a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307510f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307522100 0x55e3075114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e307518690 con 0x55e307522100 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e30739bc30 con 0x55e3075159b0 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.860+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e307518810 con 0x55e307395e40 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e52f58b80 10 client.0 _ll_register_callbacks cb 0x55e3073cd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e422e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e307510a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307510f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307510f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57458/0 (socket says 172.21.15.5:57458) 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 -- 172.21.15.5:0/3958790375 learned_addr learned my addr 172.21.15.5:0/3958790375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 -- 172.21.15.5:0/3958790375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 msgr2=0x55e307510a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e307510a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 -- 172.21.15.5:0/3958790375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307522100 msgr2=0x55e3075114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.861+0000 7f8e41adf700 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307522100 0x55e3075114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.862+0000 7f8e41adf700 1 -- 172.21.15.5:0/3958790375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3073d1690 con 0x55e3075159b0 2024-05-14T15:30:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.862+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e307510a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.863+0000 7f8e41adf700 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307510f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e30004f10 tx=0x7f8e30009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f151c27498279cd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.863+0000 7f8e25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.863+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3958790375 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30027060 con 0x55e3075159b0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.863+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e340b2010 con 0x55e3075159b0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.863+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3958790375 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f8e30005250 con 0x55e3075159b0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.864+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3958790375 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8e30009590 con 0x55e3075159b0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.864+0000 7f8e52f58b80 10 client.26231 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.864+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e30752b0f0 con 0x55e3075159b0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.864+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3958790375 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e340b2010 con 0x55e3075159b0 2024-05-14T15:30:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.864+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3958790375 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8e30022580 con 0x55e3075159b0 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e52f58b80 10 client.26231 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e52f58b80 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e52f58b80 20 client.26231 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e30752d680 0x55e30752fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e52f58b80 4 client.26231 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e307443240 con 0x55e30752d680 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.865+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e30752d680 0x55e30752fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.866+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e30752d680 0x55e30752fb30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37cf15c199ff0afa server_cookie=57e140ae2b375eaf in_seq=0 out_seq=0 2024-05-14T15:30:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.866+0000 7f8e25ffb700 10 client.26231 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.868+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3958790375 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e307443240 con 0x55e30752d680 2024-05-14T15:30:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.868+0000 7f8e25ffb700 10 client.26231 handle_command_reply: tid=0 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.868+0000 7f8e52f58b80 1 client.26231 shutdown 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.868+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 msgr2=0x55e307510f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.868+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307510f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e30004f10 tx=0x7f8e30009e60 comp rx=0 tx=0).stop 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 shutdown_connections 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e30752d680 0x55e30752fb30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307522100 0x55e3075114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e307510a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3958790375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307510f60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 >> 172.21.15.5:0/3958790375 conn(0x55e3073dd600 msgr2=0x55e30751d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e25ffb700 0 client.26231 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 shutdown_connections 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3958790375 wait complete. 2024-05-14T15:30:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.869+0000 7f8e52f58b80 20 client.26231 trim_cache size 0 max 16384 2024-05-14T15:30:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 Processor -- start 2024-05-14T15:30:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 -- start start 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e3075298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e3075087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307522100 0x55e307514e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1b0cc0 con 0x55e3075159b0 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1ae550 con 0x55e307522100 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.910+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1ac3a0 con 0x55e307395e40 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e3075298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e41adf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e3075087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e3075298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52582/0 (socket says 172.21.15.5:52582) 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 -- 172.21.15.5:0/3826923510 learned_addr learned my addr 172.21.15.5:0/3826923510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:16.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307522100 0x55e307514e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 -- 172.21.15.5:0/3826923510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307522100 msgr2=0x55e307514e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307522100 0x55e307514e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 -- 172.21.15.5:0/3826923510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 msgr2=0x55e3075087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e3075087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.911+0000 7f8e422e0700 1 -- 172.21.15.5:0/3826923510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e30750f300 con 0x55e307395e40 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.912+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e3075298e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e38012010 tx=0x7f8e38017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4717eacd4a1215c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.912+0000 7f8e27fff700 1 -- 172.21.15.5:0/3826923510 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38012b00 con 0x55e307395e40 2024-05-14T15:30:16.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.913+0000 7f8e27fff700 1 -- 172.21.15.5:0/3826923510 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e380044d0 con 0x55e307395e40 2024-05-14T15:30:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.913+0000 7f8e27fff700 1 -- 172.21.15.5:0/3826923510 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3804aa30 con 0x55e307395e40 2024-05-14T15:30:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3826923510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 msgr2=0x55e3075298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e3075298e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e38012010 tx=0x7f8e38017000 comp rx=0 tx=0).stop 2024-05-14T15:30:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3826923510 shutdown_connections 2024-05-14T15:30:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e307522100 0x55e307514e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307395e40 0x55e3075298e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3826923510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3075159b0 0x55e3075087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3826923510 >> 172.21.15.5:0/3826923510 conn(0x55e3073dd600 msgr2=0x55e307510e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3826923510 shutdown_connections 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.914+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3826923510 wait complete. 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 -- start start 2024-05-14T15:30:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 Processor -- start 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 0x55e307527050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307527590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307527b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1aca10 con 0x55e307395e40 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1ade80 con 0x55e3075159b0 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.915+0000 7f8e52f58b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1ad0e0 con 0x55e307522100 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e52f58b80 10 client.0 _ll_register_callbacks cb 0x55e3073cd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 0x55e307527050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 0x55e307527050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34798/0 (socket says 172.21.15.5:34798) 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 -- 172.21.15.5:0/3173205416 learned_addr learned my addr 172.21.15.5:0/3173205416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e42ae1700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307527b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e41adf700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307527590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 -- 172.21.15.5:0/3173205416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 msgr2=0x55e307527b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307527b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 -- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 msgr2=0x55e307527590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307527590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 -- 172.21.15.5:0/3173205416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e307597110 con 0x55e307395e40 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e422e0700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 0x55e307527050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8e38011c10 tx=0x7f8e3804a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0ec1b773654500 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.916+0000 7f8e25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.917+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3173205416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38058070 con 0x55e307395e40 2024-05-14T15:30:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.917+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3173205416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f8e3804a670 con 0x55e307395e40 2024-05-14T15:30:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.918+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e380044d0 con 0x55e307395e40 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.918+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3173205416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8e38062050 con 0x55e307395e40 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.918+0000 7f8e52f58b80 10 client.16617 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.918+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e307598380 con 0x55e307395e40 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3173205416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e3804a870 con 0x55e307395e40 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3173205416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8e3804ace0 con 0x55e307395e40 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e52f58b80 10 client.16617 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e52f58b80 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e52f58b80 20 client.16617 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e3075989c0 0x55e30759ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e52f58b80 4 client.16617 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:30:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.919+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8e380044d0 con 0x55e3075989c0 2024-05-14T15:30:16.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.920+0000 7f8e41adf700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e3075989c0 0x55e30759ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.922+0000 7f8e41adf700 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e3075989c0 0x55e30759ade0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d5b4e5815a711c server_cookie=c5f8f6d12047a6f9 in_seq=0 out_seq=0 2024-05-14T15:30:16.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.922+0000 7f8e25ffb700 10 client.16617 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e25ffb700 1 -- 172.21.15.5:0/3173205416 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8e380044d0 con 0x55e3075989c0 2024-05-14T15:30:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e25ffb700 10 client.16617 handle_command_reply: tid=0 2024-05-14T15:30:16.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 client.16617 shutdown 2024-05-14T15:30:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 msgr2=0x55e307527050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 0x55e307527050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8e38011c10 tx=0x7f8e3804a5b0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 shutdown_connections 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e3075989c0 0x55e30759ade0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e25ffb700 0 client.16617 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e307522100 0x55e307527b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e307395e40 0x55e307527050 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 --2- 172.21.15.5:0/3173205416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e3075159b0 0x55e307527590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.925+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 >> 172.21.15.5:0/3173205416 conn(0x55e3073dd600 msgr2=0x55e307510e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:16.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.926+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 shutdown_connections 2024-05-14T15:30:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.927+0000 7f8e52f58b80 1 -- 172.21.15.5:0/3173205416 wait complete. 2024-05-14T15:30:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.927+0000 7f8e52f58b80 20 client.16617 trim_cache size 0 max 16384 2024-05-14T15:30:16.930 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:30:16.930 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:30:16.930 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:30:16.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:30:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8e280b2bc0 msgr2=0x7f8e280b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8e280b2bc0 0x7f8e280b50d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009bf0 tx=0x7f8e38009260 comp rx=0 tx=0).stop 2024-05-14T15:30:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 msgr2=0x7f8e3c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 0x7f8e3c19f720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400b970 tx=0x7f8e3400bd30 comp rx=0 tx=0).stop 2024-05-14T15:30:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 shutdown_connections 2024-05-14T15:30:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8e280b2bc0 0x7f8e280b50d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.929+0000 7f8e44544700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e3c108780 0x7f8e3c19f720 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.930+0000 7f8e44544700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c103550 0x7f8e3c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.930+0000 7f8e44544700 1 --2- 172.21.15.5:0/1387696401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e3c107e30 0x7f8e3c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.930+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 >> 172.21.15.5:0/1387696401 conn(0x7f8e3c078e00 msgr2=0x7f8e3c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.932+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 shutdown_connections 2024-05-14T15:30:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:16.932+0000 7f8e44544700 1 -- 172.21.15.5:0/1387696401 wait complete. 2024-05-14T15:30:17.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 473228 11.07 MB/sec execute 98 sec latency 84.883 ms 2024-05-14T15:30:17.732 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:30:17.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:30:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:18.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 475725 11.09 MB/sec execute 99 sec latency 13.920 ms 2024-05-14T15:30:18.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:17 smithi005 ceph-mon[31559]: pgmap v168: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 365 op/s 2024-05-14T15:30:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:17 smithi144 ceph-mon[35650]: pgmap v168: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 365 op/s 2024-05-14T15:30:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:17 smithi193 ceph-mon[36001]: pgmap v168: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 365 op/s 2024-05-14T15:30:19.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 477594 11.09 MB/sec execute 100 sec latency 16.454 ms 2024-05-14T15:30:20.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:19 smithi005 ceph-mon[31559]: pgmap v169: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-05-14T15:30:20.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 480065 11.12 MB/sec execute 101 sec latency 16.474 ms 2024-05-14T15:30:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:19 smithi144 ceph-mon[35650]: pgmap v169: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-05-14T15:30:20.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:19 smithi193 ceph-mon[36001]: pgmap v169: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 -- 172.21.15.5:0/742334129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc107df0 msgr2=0x7feadc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 --2- 172.21.15.5:0/742334129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc107df0 0x7feadc108200 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fead00099c0 tx=0x7fead0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 -- 172.21.15.5:0/742334129 shutdown_connections 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 --2- 172.21.15.5:0/742334129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc108740 0x7feadc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 --2- 172.21.15.5:0/742334129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 --2- 172.21.15.5:0/742334129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc107df0 0x7feadc108200 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 -- 172.21.15.5:0/742334129 >> 172.21.15.5:0/742334129 conn(0x7feadc0fedc0 msgr2=0x7feadc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 -- 172.21.15.5:0/742334129 shutdown_connections 2024-05-14T15:30:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.305+0000 7feae080d700 1 -- 172.21.15.5:0/742334129 wait complete. 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.306+0000 7feae080d700 1 Processor -- start 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.306+0000 7feae080d700 1 -- start start 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feae080d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feae080d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc107df0 0x7feadc19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feae080d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc108740 0x7feadc19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feae080d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113350 con 0x7feadc108740 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feae080d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1131d0 con 0x7feadc107df0 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feae080d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1134d0 con 0x7feadc1034c0 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feadad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc108740 0x7feadc19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52606/0 (socket says 172.21.15.5:52606) 2024-05-14T15:30:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7fead9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc107df0 0x7feadc19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 -- 172.21.15.5:0/168715730 learned_addr learned my addr 172.21.15.5:0/168715730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 -- 172.21.15.5:0/168715730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc107df0 msgr2=0x7feadc19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc107df0 0x7feadc19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 -- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc108740 msgr2=0x7feadc19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc108740 0x7feadc19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 -- 172.21.15.5:0/168715730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1a6fc0 con 0x7feadc1034c0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7fead9d9b700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc107df0 0x7feadc19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.307+0000 7feada59c700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc19bc70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feacc00d980 tx=0x7feacc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.308+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc016070 con 0x7feadc1034c0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.308+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1a72b0 con 0x7feadc1034c0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.308+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc004830 con 0x7feadc1034c0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.308+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc005540 con 0x7feadc1034c0 2024-05-14T15:30:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.308+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a77c0 con 0x7feadc1034c0 2024-05-14T15:30:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.309+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc0512c0 con 0x7feadc1034c0 2024-05-14T15:30:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.310+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7feacc004220 con 0x7feadc1034c0 2024-05-14T15:30:20.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.311+0000 7feacb7fe700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feac40b29f0 0x7feac40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:20.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.311+0000 7fead9d9b700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feac40b29f0 0x7feac40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:20.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.311+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7feacc0e2d60 con 0x7feadc1034c0 2024-05-14T15:30:20.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.312+0000 7fead9d9b700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feac40b29f0 0x7feac40b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fead0000c00 tx=0x7fead001a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.315+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feacc01f020 con 0x7feadc1034c0 2024-05-14T15:30:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.593+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feadc02e7b0 con 0x7feadc1034c0 2024-05-14T15:30:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.594+0000 7feacb7fe700 1 -- 172.21.15.5:0/168715730 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11864 (secure 0 0 0) 0x7feacc0aa320 con 0x7feadc1034c0 2024-05-14T15:30:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:30:20.596 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:27:33.754380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25691":{"gid":25691,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2401236249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2401236249},{"type":"v1","addr":"172.21.15.144:6835","nonce":2401236249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:30:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.596+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feac40b29f0 msgr2=0x7feac40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.596+0000 7feae080d700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feac40b29f0 0x7feac40b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fead0000c00 tx=0x7fead001a040 comp rx=0 tx=0).stop 2024-05-14T15:30:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.596+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 msgr2=0x7feadc19bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.596+0000 7feae080d700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc19bc70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feacc00d980 tx=0x7feacc00dc90 comp rx=0 tx=0).stop 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 shutdown_connections 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feac40b29f0 0x7feac40b4f00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc108740 0x7feadc19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc1034c0 0x7feadc19bc70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 --2- 172.21.15.5:0/168715730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feadc107df0 0x7feadc19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 >> 172.21.15.5:0/168715730 conn(0x7feadc0fedc0 msgr2=0x7feadc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 shutdown_connections 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:20.597+0000 7feae080d700 1 -- 172.21.15.5:0/168715730 wait complete. 2024-05-14T15:30:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-05-14T15:30:21.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 482046 11.08 MB/sec execute 102 sec latency 20.291 ms 2024-05-14T15:30:21.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:20 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/168715730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:30:21.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:20 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/168715730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:30:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:20 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/168715730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:30:21.318 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:30:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:22.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:21 smithi005 ceph-mon[31559]: pgmap v170: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-05-14T15:30:22.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 483618 11.09 MB/sec execute 103 sec latency 73.394 ms 2024-05-14T15:30:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:21 smithi144 ceph-mon[35650]: pgmap v170: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-05-14T15:30:22.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:21 smithi193 ceph-mon[36001]: pgmap v170: 129 pgs: 129 active+clean; 56 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 2.4 MiB/s wr, 253 op/s 2024-05-14T15:30:23.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 485660 11.05 MB/sec execute 104 sec latency 15.204 ms 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 -- 172.21.15.5:0/1461633180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50741045e0 msgr2=0x7f50741049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 --2- 172.21.15.5:0/1461633180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50741045e0 0x7f50741049f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f50640099c0 tx=0x7f5064009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 -- 172.21.15.5:0/1461633180 shutdown_connections 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 --2- 172.21.15.5:0/1461633180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5074109860 0x7f50741111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 --2- 172.21.15.5:0/1461633180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50741045e0 0x7f50741049f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 --2- 172.21.15.5:0/1461633180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5074108f10 0x7f5074109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 -- 172.21.15.5:0/1461633180 >> 172.21.15.5:0/1461633180 conn(0x7f50740ffed0 msgr2=0x7f5074102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 -- 172.21.15.5:0/1461633180 shutdown_connections 2024-05-14T15:30:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.533+0000 7f507a842700 1 -- 172.21.15.5:0/1461633180 wait complete. 2024-05-14T15:30:23.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.534+0000 7f507a842700 1 Processor -- start 2024-05-14T15:30:23.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.534+0000 7f507a842700 1 -- start start 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f507a842700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 0x7f507419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f507a842700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5074108f10 0x7f507419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f507a842700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074109860 0x7f507419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f507a842700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5074114490 con 0x7f5074109860 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f507a842700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5074114310 con 0x7f5074108f10 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f507a842700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5074114610 con 0x7f50741045e0 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5078ddf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074109860 0x7f507419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 0x7f507419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 0x7f507419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52628/0 (socket says 172.21.15.5:52628) 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 -- 172.21.15.5:0/2905608512 learned_addr learned my addr 172.21.15.5:0/2905608512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5078ddf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074109860 0x7f507419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34838/0 (socket says 172.21.15.5:34838) 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f50737fe700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5074108f10 0x7f507419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 -- 172.21.15.5:0/2905608512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5074108f10 msgr2=0x7f507419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5074108f10 0x7f507419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 -- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074109860 msgr2=0x7f507419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074109860 0x7f507419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 -- 172.21.15.5:0/2905608512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507419fcb0 con 0x7f50741045e0 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.535+0000 7f5073fff700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 0x7f507419ade0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5064006010 tx=0x7f5064005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.536+0000 7f50717fa700 1 -- 172.21.15.5:0/2905608512 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064026070 con 0x7f50741045e0 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.536+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507419ff40 con 0x7f50741045e0 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.536+0000 7f50717fa700 1 -- 172.21.15.5:0/2905608512 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5064005680 con 0x7f50741045e0 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.536+0000 7f50717fa700 1 -- 172.21.15.5:0/2905608512 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506400fb20 con 0x7f50741045e0 2024-05-14T15:30:23.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.536+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50741a78c0 con 0x7f50741045e0 2024-05-14T15:30:23.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.538+0000 7f50717fa700 1 -- 172.21.15.5:0/2905608512 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5064029070 con 0x7f50741045e0 2024-05-14T15:30:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.539+0000 7f50717fa700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f505c0b2a70 0x7f505c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.539+0000 7f50737fe700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f505c0b2a70 0x7f505c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.539+0000 7f50717fa700 1 -- 172.21.15.5:0/2905608512 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f50640e3280 con 0x7f50741045e0 2024-05-14T15:30:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 Processor -- start 2024-05-14T15:30:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 -- start start 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 0x55d643017720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d643017c60 0x55d64301bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f50737fe700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f505c0b2a70 0x7f505c0b4f80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5074103470 tx=0x7f5068009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64301c470 0x55d64301c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d642f39860 con 0x55d64301c470 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d642ed87c0 con 0x55d643017c60 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5089256b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d642f35820 con 0x55d642e99e40 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5078ddf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64301c470 0x55d64301c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5073fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 0x55d643017720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5073fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 0x55d643017720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:52630/0 (socket says 172.21.15.5:52630) 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f50737fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d643017c60 0x55d64301bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5073fff700 1 -- 172.21.15.5:0/1277252119 learned_addr learned my addr 172.21.15.5:0/1277252119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.540+0000 7f5078ddf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64301c470 0x55d64301c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34844/0 (socket says 172.21.15.5:34844) 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f5073fff700 1 -- 172.21.15.5:0/1277252119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d643017c60 msgr2=0x55d64301bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f5073fff700 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d643017c60 0x55d64301bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f5073fff700 1 -- 172.21.15.5:0/1277252119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64301c470 msgr2=0x55d64301c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f5073fff700 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64301c470 0x55d64301c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f5073fff700 1 -- 172.21.15.5:0/1277252119 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d642e868d0 con 0x55d642e99e40 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f5073fff700 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 0x55d643017720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50640a62b0 tx=0x7f5064003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2603f17601b3d867 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f505a7fc700 1 -- 172.21.15.5:0/1277252119 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640aa230 con 0x55d642e99e40 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f505a7fc700 1 -- 172.21.15.5:0/1277252119 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50640ae5e0 con 0x55d642e99e40 2024-05-14T15:30:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.541+0000 7f505a7fc700 1 -- 172.21.15.5:0/1277252119 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640b2d70 con 0x55d642e99e40 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 -- 172.21.15.5:0/1277252119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 msgr2=0x55d643017720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 0x55d643017720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f50640a62b0 tx=0x7f5064003e20 comp rx=0 tx=0).stop 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 -- 172.21.15.5:0/1277252119 shutdown_connections 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64301c470 0x55d64301c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d642e99e40 0x55d643017720 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 --2- 172.21.15.5:0/1277252119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d643017c60 0x55d64301bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 -- 172.21.15.5:0/1277252119 >> 172.21.15.5:0/1277252119 conn(0x55d642ee1600 msgr2=0x55d6430124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 -- 172.21.15.5:0/1277252119 shutdown_connections 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 -- 172.21.15.5:0/1277252119 wait complete. 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.542+0000 7f5089256b80 1 -- start start 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 Processor -- start 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d642e99e40 0x55d643021270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d643017c60 0x55d6430217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d64301c470 0x55d643021cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d6430241e0 con 0x55d643017c60 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d642e9fc30 con 0x55d642e99e40 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d643024360 con 0x55d64301c470 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5089256b80 10 client.0 _ll_register_callbacks cb 0x55d642ed1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f50737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d643017c60 0x55d6430217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5073fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d642e99e40 0x55d643021270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f50737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d643017c60 0x55d6430217b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34852/0 (socket says 172.21.15.5:34852) 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5073fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d642e99e40 0x55d643021270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57546/0 (socket says 172.21.15.5:57546) 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f5078ddf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d64301c470 0x55d643021cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.543+0000 7f50737fe700 1 -- 172.21.15.5:0/2412835492 learned_addr learned my addr 172.21.15.5:0/2412835492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5078ddf700 1 -- 172.21.15.5:0/2412835492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d642e99e40 msgr2=0x55d643021270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5078ddf700 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d642e99e40 0x55d643021270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5078ddf700 1 -- 172.21.15.5:0/2412835492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d643017c60 msgr2=0x55d6430217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5078ddf700 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d643017c60 0x55d6430217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5078ddf700 1 -- 172.21.15.5:0/2412835492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d642ed5690 con 0x55d64301c470 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5078ddf700 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d64301c470 0x55d643021cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f506c011e90 tx=0x7f506c014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f202005efadbbe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f50537fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f50537fe700 1 -- 172.21.15.5:0/2412835492 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c004bb0 con 0x55d64301c470 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f50537fe700 1 -- 172.21.15.5:0/2412835492 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f506c004d50 con 0x55d64301c470 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50640ae5e0 con 0x55d64301c470 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f50537fe700 1 -- 172.21.15.5:0/2412835492 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f506c026050 con 0x55d64301c470 2024-05-14T15:30:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5089256b80 10 client.26242 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.544+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d64302ee80 con 0x55d64301c470 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f50537fe700 1 -- 172.21.15.5:0/2412835492 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50640ae5e0 con 0x55d64301c470 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f50537fe700 1 -- 172.21.15.5:0/2412835492 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f506c0041e0 con 0x55d64301c470 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5089256b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5089256b80 10 client.26242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5089256b80 20 client.26242 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5089256b80 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d643031860 0x55d643033c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5089256b80 4 client.26242 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d642f47240 con 0x55d643031860 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.545+0000 7f5073fff700 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d643031860 0x55d643033c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.546+0000 7f50537fe700 10 client.26242 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.546+0000 7f5073fff700 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d643031860 0x55d643033c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ea86f52ada81881 server_cookie=700b382c20fe7be4 in_seq=0 out_seq=0 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.546+0000 7f50537fe700 1 -- 172.21.15.5:0/2412835492 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d642f47240 con 0x55d643031860 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.546+0000 7f50537fe700 10 client.26242 handle_command_reply: tid=0 2024-05-14T15:30:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.546+0000 7f5089256b80 1 client.26242 shutdown 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d64301c470 msgr2=0x55d643021cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d64301c470 0x55d643021cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f506c011e90 tx=0x7f506c014600 comp rx=0 tx=0).stop 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 shutdown_connections 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d64301c470 0x55d643021cf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d643031860 0x55d643033c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d642e99e40 0x55d643021270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 --2- 172.21.15.5:0/2412835492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d643017c60 0x55d6430217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 >> 172.21.15.5:0/2412835492 conn(0x55d642ee1600 msgr2=0x55d643016450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f50537fe700 0 client.26242 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 shutdown_connections 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 1 -- 172.21.15.5:0/2412835492 wait complete. 2024-05-14T15:30:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.547+0000 7f5089256b80 20 client.26242 trim_cache size 0 max 16384 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.563+0000 7f5089256b80 1 Processor -- start 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.563+0000 7f5089256b80 1 -- start start 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5089256b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d64302d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5089256b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d643021b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5089256b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d64301d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5089256b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50741b0c50 con 0x55d642e99e40 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5089256b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ae4e0 con 0x55d64301c470 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5089256b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ac330 con 0x55d643017c60 2024-05-14T15:30:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d64302d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d64302d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34858/0 (socket says 172.21.15.5:34858) 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 -- 172.21.15.5:0/2200339613 learned_addr learned my addr 172.21.15.5:0/2200339613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f50737fe700 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d643021b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5078ddf700 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d64301d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 -- 172.21.15.5:0/2200339613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 msgr2=0x55d643021b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d643021b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 -- 172.21.15.5:0/2200339613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 msgr2=0x55d64301d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d64301d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 -- 172.21.15.5:0/2200339613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d643022070 con 0x55d642e99e40 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f50737fe700 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d643021b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5073fff700 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d64302d5c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f50640e06c0 tx=0x7f5064009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea906c343521a5c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.564+0000 7f5059ffb700 1 -- 172.21.15.5:0/2200339613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640c3070 con 0x55d642e99e40 2024-05-14T15:30:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5059ffb700 1 -- 172.21.15.5:0/2200339613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50640ae5e0 con 0x55d642e99e40 2024-05-14T15:30:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5059ffb700 1 -- 172.21.15.5:0/2200339613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640b2dd0 con 0x55d642e99e40 2024-05-14T15:30:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 -- 172.21.15.5:0/2200339613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 msgr2=0x55d64302d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d64302d5c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f50640e06c0 tx=0x7f5064009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 -- 172.21.15.5:0/2200339613 shutdown_connections 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d64301d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d64302d5c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 --2- 172.21.15.5:0/2200339613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d643021b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 -- 172.21.15.5:0/2200339613 >> 172.21.15.5:0/2200339613 conn(0x55d642ee1600 msgr2=0x55d643011be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.565+0000 7f5089256b80 1 -- 172.21.15.5:0/2200339613 shutdown_connections 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 -- 172.21.15.5:0/2200339613 wait complete. 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 -- start start 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 Processor -- start 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d643037330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d6430336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d643033cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.566+0000 7f5089256b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ac9a0 con 0x55d642e99e40 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5089256b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ade10 con 0x55d64301c470 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5089256b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ad070 con 0x55d643017c60 2024-05-14T15:30:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5089256b80 10 client.0 _ll_register_callbacks cb 0x55d642ed1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d643037330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d643037330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34870/0 (socket says 172.21.15.5:34870) 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 -- 172.21.15.5:0/1424439871 learned_addr learned my addr 172.21.15.5:0/1424439871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5078ddf700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d643033cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f50737fe700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d6430336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 -- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 msgr2=0x55d6430336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d6430336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 -- 172.21.15.5:0/1424439871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 msgr2=0x55d643033cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d643033cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 -- 172.21.15.5:0/1424439871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d643034470 con 0x55d642e99e40 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5073fff700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d643037330 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5064009fa0 tx=0x7f50640a9a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b648d375f70cba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f50537fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f50537fe700 1 -- 172.21.15.5:0/1424439871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640c8020 con 0x55d642e99e40 2024-05-14T15:30:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f50537fe700 1 -- 172.21.15.5:0/1424439871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f50640b2b70 con 0x55d642e99e40 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.567+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50640ae5e0 con 0x55d642e99e40 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f50537fe700 1 -- 172.21.15.5:0/1424439871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f50640c3030 con 0x55d642e99e40 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 10 client.16641 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d64309adc0 con 0x55d642e99e40 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f50537fe700 1 -- 172.21.15.5:0/1424439871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50640ae5e0 con 0x55d642e99e40 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f50537fe700 1 -- 172.21.15.5:0/1424439871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f50640a5ba0 con 0x55d642e99e40 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 10 client.16641 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 20 client.16641 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d64309b1c0 0x55d64309d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 4 client.16641 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.568+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d642f412e0 con 0x55d64309b1c0 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.569+0000 7f50737fe700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d64309b1c0 0x55d64309d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.569+0000 7f50737fe700 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d64309b1c0 0x55d64309d5e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b4c3c2346d09ef3 server_cookie=f1288eddf6d8afad in_seq=0 out_seq=0 2024-05-14T15:30:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.569+0000 7f50537fe700 10 client.16641 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:23.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.569+0000 7f50537fe700 1 -- 172.21.15.5:0/1424439871 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d642f412e0 con 0x55d64309b1c0 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.569+0000 7f50537fe700 10 client.16641 handle_command_reply: tid=0 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.569+0000 7f5089256b80 1 client.16641 shutdown 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 msgr2=0x55d643037330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d643037330 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5064009fa0 tx=0x7f50640a9a50 comp rx=0 tx=0).stop 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 shutdown_connections 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55d64309b1c0 0x55d64309d5e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d64301c470 0x55d643033cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d642e99e40 0x55d643037330 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f50537fe700 0 client.16641 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 --2- 172.21.15.5:0/1424439871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d643017c60 0x55d6430336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 >> 172.21.15.5:0/1424439871 conn(0x55d642ee1600 msgr2=0x55d64301e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 shutdown_connections 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 1 -- 172.21.15.5:0/1424439871 wait complete. 2024-05-14T15:30:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f5089256b80 20 client.16641 trim_cache size 0 max 16384 2024-05-14T15:30:23.576 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:30:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f505c0b2a70 msgr2=0x7f505c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f507a842700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f505c0b2a70 0x7f505c0b4f80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5074103470 tx=0x7f5068009260 comp rx=0 tx=0).stop 2024-05-14T15:30:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 msgr2=0x7f507419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.570+0000 7f507a842700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 0x7f507419ade0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5064006010 tx=0x7f5064005280 comp rx=0 tx=0).stop 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 shutdown_connections 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f505c0b2a70 0x7f505c0b4f80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074109860 0x7f507419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50741045e0 0x7f507419ade0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 --2- 172.21.15.5:0/2905608512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5074108f10 0x7f507419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 >> 172.21.15.5:0/2905608512 conn(0x7f50740ffed0 msgr2=0x7f50741086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 shutdown_connections 2024-05-14T15:30:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:23.571+0000 7f507a842700 1 -- 172.21.15.5:0/2905608512 wait complete. 2024-05-14T15:30:24.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 487573 11.07 MB/sec execute 105 sec latency 38.874 ms 2024-05-14T15:30:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:23 smithi144 ceph-mon[35650]: pgmap v171: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-05-14T15:30:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:23 smithi193 ceph-mon[36001]: pgmap v171: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-05-14T15:30:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:23 smithi005 ceph-mon[31559]: pgmap v171: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-05-14T15:30:24.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:30:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:25.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 490251 11.12 MB/sec execute 106 sec latency 30.425 ms 2024-05-14T15:30:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:25 smithi144 ceph-mon[35650]: pgmap v172: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2024-05-14T15:30:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:25 smithi193 ceph-mon[36001]: pgmap v172: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2024-05-14T15:30:25.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:25 smithi005 ceph-mon[31559]: pgmap v172: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.8 MiB/s wr, 288 op/s 2024-05-14T15:30:26.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 491867 11.08 MB/sec execute 107 sec latency 16.276 ms 2024-05-14T15:30:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 -- 172.21.15.5:0/920271421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 msgr2=0x7efe84102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 --2- 172.21.15.5:0/920271421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe84102c80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009900 tx=0x7efe6c02a410 comp rx=0 tx=0).stop 2024-05-14T15:30:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 -- 172.21.15.5:0/920271421 shutdown_connections 2024-05-14T15:30:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 --2- 172.21.15.5:0/920271421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 0x7efe8410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 --2- 172.21.15.5:0/920271421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe84102c80 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 --2- 172.21.15.5:0/920271421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 0x7efe84102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 -- 172.21.15.5:0/920271421 >> 172.21.15.5:0/920271421 conn(0x7efe840782e0 msgr2=0x7efe84078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 -- 172.21.15.5:0/920271421 shutdown_connections 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 -- 172.21.15.5:0/920271421 wait complete. 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.470+0000 7efe89bb2640 1 Processor -- start 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 -- start start 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 0x7efe8419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 0x7efe8419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe8419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe84112d90 con 0x7efe84109040 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe84112c10 con 0x7efe84108ba0 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe89bb2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe84112f10 con 0x7efe841032f0 2024-05-14T15:30:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe8419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe8419c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44882/0 (socket says 172.21.15.5:44882) 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe837fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 0x7efe8419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 -- 172.21.15.5:0/1383049845 learned_addr learned my addr 172.21.15.5:0/1383049845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe7bfff640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 0x7efe8419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 -- 172.21.15.5:0/1383049845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 msgr2=0x7efe8419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 0x7efe8419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 -- 172.21.15.5:0/1383049845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 msgr2=0x7efe8419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 0x7efe8419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe83fff640 1 -- 172.21.15.5:0/1383049845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe841a6a50 con 0x7efe84109040 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe7bfff640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 0x7efe8419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.471+0000 7efe837fe640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 0x7efe8419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.472+0000 7efe83fff640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe8419c210 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00eb10 tx=0x7efe7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.472+0000 7efe817fa640 1 -- 172.21.15.5:0/1383049845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c004b20 con 0x7efe84109040 2024-05-14T15:30:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.472+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe841a6d40 con 0x7efe84109040 2024-05-14T15:30:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.472+0000 7efe817fa640 1 -- 172.21.15.5:0/1383049845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7c004cc0 con 0x7efe84109040 2024-05-14T15:30:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.472+0000 7efe817fa640 1 -- 172.21.15.5:0/1383049845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c010690 con 0x7efe84109040 2024-05-14T15:30:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.472+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a71a0 con 0x7efe84109040 2024-05-14T15:30:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.474+0000 7efe90a65740 1 Processor -- start 2024-05-14T15:30:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.474+0000 7efe817fa640 1 -- 172.21.15.5:0/1383049845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7efe7c0108f0 con 0x7efe84109040 2024-05-14T15:30:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.474+0000 7efe90a65740 1 -- start start 2024-05-14T15:30:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe90a65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 0x55fe168afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe90a65740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe164e62f0 0x55fe168b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe90a65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe90a65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe168155d0 con 0x55fe1650f1b0 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe90a65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fe167ce890 con 0x55fe164ee4d0 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe90a65740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe16811590 con 0x55fe164e62f0 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 0x55fe168afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 0x55fe168afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44894/0 (socket says 172.21.15.5:44894) 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 -- 172.21.15.5:0/2417962098 learned_addr learned my addr 172.21.15.5:0/2417962098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe837fe640 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe164e62f0 0x55fe168b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 -- 172.21.15.5:0/2417962098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe164e62f0 msgr2=0x55fe168b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe164e62f0 0x55fe168b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 -- 172.21.15.5:0/2417962098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 msgr2=0x55fe168b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.475+0000 7efe7bfff640 1 -- 172.21.15.5:0/2417962098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe16527210 con 0x55fe1650f1b0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe7bfff640 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 0x55fe168afbe0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efe8406c650 tx=0x7efe6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337618864eb6052 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe817fa640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe700b2950 0x7efe700b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe837fe640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe700b2950 0x7efe700b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe79ffb640 1 -- 172.21.15.5:0/2417962098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c044020 con 0x55fe1650f1b0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe817fa640 1 -- 172.21.15.5:0/1383049845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7efe7c0e2f20 con 0x7efe84109040 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe79ffb640 1 -- 172.21.15.5:0/2417962098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c005040 con 0x55fe1650f1b0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.476+0000 7efe79ffb640 1 -- 172.21.15.5:0/2417962098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c035040 con 0x55fe1650f1b0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 -- 172.21.15.5:0/2417962098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 msgr2=0x55fe168afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe837fe640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe700b2950 0x7efe700b4e60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55fe168b0f90 tx=0x7efe74005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 0x55fe168afbe0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efe8406c650 tx=0x7efe6c000bc0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 -- 172.21.15.5:0/2417962098 shutdown_connections 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe164e62f0 0x55fe168b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 --2- 172.21.15.5:0/2417962098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe1650f1b0 0x55fe168afbe0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 -- 172.21.15.5:0/2417962098 >> 172.21.15.5:0/2417962098 conn(0x55fe16503300 msgr2=0x55fe168ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 -- 172.21.15.5:0/2417962098 shutdown_connections 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 -- 172.21.15.5:0/2417962098 wait complete. 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.477+0000 7efe90a65740 1 -- start start 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 Processor -- start 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168ba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe168bccb0 con 0x55fe164ee4d0 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fe167d2370 con 0x55fe164e62f0 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe168bce30 con 0x55fe1650f1b0 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe90a65740 10 client.0 _ll_register_callbacks cb 0x55fe16526a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe83fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168ba750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44906/0 (socket says 172.21.15.5:44906) 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe7bfff640 1 -- 172.21.15.5:0/3920083770 learned_addr learned my addr 172.21.15.5:0/3920083770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe837fe640 1 -- 172.21.15.5:0/3920083770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 msgr2=0x55fe168c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe837fe640 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe837fe640 1 -- 172.21.15.5:0/3920083770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 msgr2=0x55fe168ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe837fe640 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168ba750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.478+0000 7efe837fe640 1 -- 172.21.15.5:0/3920083770 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe165f3620 con 0x55fe164e62f0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe837fe640 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168ba210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55fe168ad6d0 tx=0x7efe74004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83b56027e2a4548f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe62ffd640 1 -- 172.21.15.5:0/3920083770 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7402e060 con 0x55fe164e62f0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c005040 con 0x55fe164e62f0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe62ffd640 1 -- 172.21.15.5:0/3920083770 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7efe74018030 con 0x55fe164e62f0 2024-05-14T15:30:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe62ffd640 1 -- 172.21.15.5:0/3920083770 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe74035050 con 0x55fe164e62f0 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe90a65740 10 client.26273 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe168c90e0 con 0x55fe164e62f0 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.479+0000 7efe62ffd640 1 -- 172.21.15.5:0/3920083770 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c005040 con 0x55fe164e62f0 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe62ffd640 1 -- 172.21.15.5:0/3920083770 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efe7401c5d0 con 0x55fe164e62f0 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe90a65740 10 client.26273 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe90a65740 10 client.26273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe90a65740 20 client.26273 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe90a65740 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164cdd80 0x55fe168cc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe90a65740 4 client.26273 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe168231e0 con 0x55fe164cdd80 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe7bfff640 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164cdd80 0x55fe168cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe62ffd640 10 client.26273 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.480+0000 7efe7bfff640 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164cdd80 0x55fe168cc560 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f92f07dd1a0f19 server_cookie=5e71d1cb8b437072 in_seq=0 out_seq=0 2024-05-14T15:30:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.482+0000 7efe62ffd640 1 -- 172.21.15.5:0/3920083770 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe168231e0 con 0x55fe164cdd80 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.482+0000 7efe62ffd640 10 client.26273 handle_command_reply: tid=0 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.482+0000 7efe90a65740 1 client.26273 shutdown 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.482+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 msgr2=0x55fe168ba210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.482+0000 7efe90a65740 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168ba210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55fe168ad6d0 tx=0x7efe74004730 comp rx=0 tx=0).stop 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.482+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 shutdown_connections 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164cdd80 0x55fe168cc560 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe62ffd640 0 client.26273 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 --2- 172.21.15.5:0/3920083770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168ba210 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 >> 172.21.15.5:0/3920083770 conn(0x55fe16503300 msgr2=0x55fe168af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 shutdown_connections 2024-05-14T15:30:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 1 -- 172.21.15.5:0/3920083770 wait complete. 2024-05-14T15:30:26.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.483+0000 7efe90a65740 20 client.26273 trim_cache size 0 max 16384 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 Processor -- start 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 -- start start 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164e62f0 0x55fe168b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b3be0 con 0x55fe164e62f0 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b14b0 con 0x55fe164ee4d0 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.499+0000 7efe90a65740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe841af300 con 0x55fe1650f1b0 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164e62f0 0x55fe168b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164e62f0 0x55fe168b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44916/0 (socket says 172.21.15.5:44916) 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe837fe640 1 -- 172.21.15.5:0/655075853 learned_addr learned my addr 172.21.15.5:0/655075853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe83fff640 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 -- 172.21.15.5:0/655075853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 msgr2=0x55fe168bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 -- 172.21.15.5:0/655075853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164e62f0 msgr2=0x55fe168b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164e62f0 0x55fe168b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 -- 172.21.15.5:0/655075853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe168b0fc0 con 0x55fe164ee4d0 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe83fff640 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe7bfff640 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b03a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fe168ccd00 tx=0x7efe6c009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ca60b653ffc9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe797fa640 1 -- 172.21.15.5:0/655075853 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c052070 con 0x55fe164ee4d0 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe797fa640 1 -- 172.21.15.5:0/655075853 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c005590 con 0x55fe164ee4d0 2024-05-14T15:30:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.500+0000 7efe797fa640 1 -- 172.21.15.5:0/655075853 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c044040 con 0x55fe164ee4d0 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 -- 172.21.15.5:0/655075853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 msgr2=0x55fe168b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b03a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55fe168ccd00 tx=0x7efe6c009520 comp rx=0 tx=0).stop 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 -- 172.21.15.5:0/655075853 shutdown_connections 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe168bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164ee4d0 0x55fe168b03a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 --2- 172.21.15.5:0/655075853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164e62f0 0x55fe168b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 -- 172.21.15.5:0/655075853 >> 172.21.15.5:0/655075853 conn(0x55fe16503300 msgr2=0x55fe168bb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 -- 172.21.15.5:0/655075853 shutdown_connections 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.501+0000 7efe90a65740 1 -- 172.21.15.5:0/655075853 wait complete. 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 -- start start 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 Processor -- start 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168c1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe16930d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe841af970 con 0x55fe164ee4d0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b0de0 con 0x55fe164e62f0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe841b0040 con 0x55fe1650f1b0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe90a65740 10 client.0 _ll_register_callbacks cb 0x7efe7400f4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168c1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44922/0 (socket says 172.21.15.5:44922) 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.502+0000 7efe7bfff640 1 -- 172.21.15.5:0/408504251 learned_addr learned my addr 172.21.15.5:0/408504251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe83fff640 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe16930d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe837fe640 1 -- 172.21.15.5:0/408504251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 msgr2=0x55fe16930d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe837fe640 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe16930d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe837fe640 1 -- 172.21.15.5:0/408504251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 msgr2=0x55fe168c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe837fe640 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe837fe640 1 -- 172.21.15.5:0/408504251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe168c7170 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe837fe640 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168c1930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe740092e0 tx=0x7efe7401ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ff50888203792c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe62ffd640 1 -- 172.21.15.5:0/408504251 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7403d070 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe62ffd640 1 -- 172.21.15.5:0/408504251 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7efe7401c5b0 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.503+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c005590 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe62ffd640 1 -- 172.21.15.5:0/408504251 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe7400f930 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 10 client.26279 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe16938370 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe62ffd640 1 -- 172.21.15.5:0/408504251 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6c005590 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe62ffd640 1 -- 172.21.15.5:0/408504251 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efe74004a70 con 0x55fe164e62f0 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 10 client.26279 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 20 client.26279 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164a31c0 0x55fe1693aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 4 client.26279 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:30:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe1681d050 con 0x55fe164a31c0 2024-05-14T15:30:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.504+0000 7efe7bfff640 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164a31c0 0x55fe1693aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.505+0000 7efe62ffd640 10 client.26279 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.505+0000 7efe7bfff640 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164a31c0 0x55fe1693aac0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1469fc3ba2b5b24a server_cookie=910233b60b1eca6 in_seq=0 out_seq=0 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe62ffd640 1 -- 172.21.15.5:0/408504251 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe1681d050 con 0x55fe164a31c0 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe62ffd640 10 client.26279 handle_command_reply: tid=0 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 client.26279 shutdown 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 msgr2=0x55fe168c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168c1930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe740092e0 tx=0x7efe7401ce10 comp rx=0 tx=0).stop 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 shutdown_connections 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe164a31c0 0x55fe1693aac0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe1650f1b0 0x55fe16930d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe62ffd640 0 client.26279 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe164ee4d0 0x55fe168c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 --2- 172.21.15.5:0/408504251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe164e62f0 0x55fe168c1930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.935+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 >> 172.21.15.5:0/408504251 conn(0x55fe16503300 msgr2=0x55fe168b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.936+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 shutdown_connections 2024-05-14T15:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.936+0000 7efe90a65740 1 -- 172.21.15.5:0/408504251 wait complete. 2024-05-14T15:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.936+0000 7efe90a65740 20 client.26279 trim_cache size 0 max 16384 2024-05-14T15:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:30:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe700b2950 msgr2=0x7efe700b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe700b2950 0x7efe700b4e60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55fe168b0f90 tx=0x7efe74005f90 comp rx=0 tx=0).stop 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 msgr2=0x7efe8419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe8419c210 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00eb10 tx=0x7efe7c00c2d0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 shutdown_connections 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efe700b2950 0x7efe700b4e60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe84109040 0x7efe8419c210 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe84108ba0 0x7efe8419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 --2- 172.21.15.5:0/1383049845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe841032f0 0x7efe8419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.937+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 >> 172.21.15.5:0/1383049845 conn(0x7efe840782e0 msgr2=0x7efe841012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.938+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 shutdown_connections 2024-05-14T15:30:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:26.938+0000 7efe89bb2640 1 -- 172.21.15.5:0/1383049845 wait complete. 2024-05-14T15:30:27.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 493483 11.05 MB/sec execute 108 sec latency 187.067 ms 2024-05-14T15:30:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 -- 172.21.15.5:0/3152606637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda50107e10 msgr2=0x7fda50108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 --2- 172.21.15.5:0/3152606637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda50107e10 0x7fda50108220 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fda380099c0 tx=0x7fda38009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 -- 172.21.15.5:0/3152606637 shutdown_connections 2024-05-14T15:30:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 --2- 172.21.15.5:0/3152606637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda50108760 0x7fda501100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 --2- 172.21.15.5:0/3152606637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda501034e0 0x7fda501038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 --2- 172.21.15.5:0/3152606637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda50107e10 0x7fda50108220 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 -- 172.21.15.5:0/3152606637 >> 172.21.15.5:0/3152606637 conn(0x7fda500fef30 msgr2=0x7fda50101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 -- 172.21.15.5:0/3152606637 shutdown_connections 2024-05-14T15:30:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 -- 172.21.15.5:0/3152606637 wait complete. 2024-05-14T15:30:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.463+0000 7fda566ab700 1 Processor -- start 2024-05-14T15:30:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 -- start start 2024-05-14T15:30:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 0x7fda50199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda50107e10 0x7fda50199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda50108760 0x7fda5019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda50113390 con 0x7fda501034e0 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda50113210 con 0x7fda50108760 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda566ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda50113510 con 0x7fda50107e10 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 0x7fda50199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 0x7fda50199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44940/0 (socket says 172.21.15.5:44940) 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 -- 172.21.15.5:0/3543886677 learned_addr learned my addr 172.21.15.5:0/3543886677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda54c48700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda50108760 0x7fda5019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda47fff700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda50107e10 0x7fda50199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 -- 172.21.15.5:0/3543886677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda50107e10 msgr2=0x7fda50199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda50107e10 0x7fda50199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 -- 172.21.15.5:0/3543886677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda50108760 msgr2=0x7fda5019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda50108760 0x7fda5019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.464+0000 7fda4ffff700 1 -- 172.21.15.5:0/3543886677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda501a91a0 con 0x7fda501034e0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.465+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 0x7fda50199aa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9c0 tx=0x7fda4000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.465+0000 7fda4dffb700 1 -- 172.21.15.5:0/3543886677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4000cbd0 con 0x7fda501034e0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.465+0000 7fda4dffb700 1 -- 172.21.15.5:0/3543886677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4000cd70 con 0x7fda501034e0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.465+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda501a9490 con 0x7fda501034e0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.465+0000 7fda4dffb700 1 -- 172.21.15.5:0/3543886677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40005610 con 0x7fda501034e0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.465+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501a9950 con 0x7fda501034e0 2024-05-14T15:30:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.468+0000 7fda4dffb700 1 -- 172.21.15.5:0/3543886677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fda40021440 con 0x7fda501034e0 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda4dffb700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fda3c0b29f0 0x7fda3c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 Processor -- start 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 -- start start 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda47fff700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fda3c0b29f0 0x7fda3c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda4dffb700 1 -- 172.21.15.5:0/3543886677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fda400e2da0 con 0x7fda501034e0 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd40c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd410f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd411880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618dd32e860 con 0x5618dd40cc60 2024-05-14T15:30:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5618dd2cd7c0 con 0x5618dd411470 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda650bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5618dd32a820 con 0x5618dd28ee40 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd410f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd40c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd40c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:49166/0 (socket says 172.21.15.5:49166) 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda4ffff700 1 -- 172.21.15.5:0/3795719583 learned_addr learned my addr 172.21.15.5:0/3795719583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda47fff700 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd410f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44944/0 (socket says 172.21.15.5:44944) 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.469+0000 7fda54c48700 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd411880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda4ffff700 1 -- 172.21.15.5:0/3795719583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 msgr2=0x5618dd411880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd411880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda4ffff700 1 -- 172.21.15.5:0/3795719583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 msgr2=0x5618dd410f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd410f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda4ffff700 1 -- 172.21.15.5:0/3795719583 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd27b8d0 con 0x5618dd28ee40 2024-05-14T15:30:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda54c48700 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd411880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd40c720 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda400a5ce0 tx=0x7fda400a9b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=932e1d7a246c55c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda47fff700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fda3c0b29f0 0x7fda3c0b4f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda38005040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda467fc700 1 -- 172.21.15.5:0/3795719583 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400a9770 con 0x5618dd28ee40 2024-05-14T15:30:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda467fc700 1 -- 172.21.15.5:0/3795719583 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda40091040 con 0x5618dd28ee40 2024-05-14T15:30:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.470+0000 7fda467fc700 1 -- 172.21.15.5:0/3795719583 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda400aa0c0 con 0x5618dd28ee40 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.471+0000 7fda650bfb80 1 -- 172.21.15.5:0/3795719583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 msgr2=0x5618dd40c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.471+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd40c720 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda400a5ce0 tx=0x7fda400a9b60 comp rx=0 tx=0).stop 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.471+0000 7fda650bfb80 1 -- 172.21.15.5:0/3795719583 shutdown_connections 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.471+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd411880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.471+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd40c720 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.471+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3795719583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd410f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.472+0000 7fda650bfb80 1 -- 172.21.15.5:0/3795719583 >> 172.21.15.5:0/3795719583 conn(0x5618dd2d6600 msgr2=0x5618dd4074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.472+0000 7fda650bfb80 1 -- 172.21.15.5:0/3795719583 shutdown_connections 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.472+0000 7fda650bfb80 1 -- 172.21.15.5:0/3795719583 wait complete. 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.472+0000 7fda650bfb80 1 -- start start 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.472+0000 7fda650bfb80 1 Processor -- start 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd416240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd416780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd416240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd416240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44960/0 (socket says 172.21.15.5:44960) 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 -- 172.21.15.5:0/3936281926 learned_addr learned my addr 172.21.15.5:0/3936281926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618dd419420 con 0x5618dd28ee40 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5618dd294c30 con 0x5618dd411470 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5618dd4195a0 con 0x5618dd40cc60 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda54c48700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda650bfb80 10 client.0 _ll_register_callbacks cb 0x5618dd2c6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda47fff700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd416780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 -- 172.21.15.5:0/3936281926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 msgr2=0x5618dd416780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd416780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 -- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 msgr2=0x5618dd416cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.473+0000 7fda4ffff700 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd2ca690 con 0x5618dd28ee40 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd416240 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fda40009fd0 tx=0x7fda400b2860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f883d13b4bacd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3936281926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4006d070 con 0x5618dd28ee40 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3936281926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fda400a90d0 con 0x5618dd28ee40 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda40091040 con 0x5618dd28ee40 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3936281926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda400a9770 con 0x5618dd28ee40 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda650bfb80 10 client.16671 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618dd419420 con 0x5618dd28ee40 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.474+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3936281926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda40091040 con 0x5618dd28ee40 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3936281926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fda4006f310 con 0x5618dd28ee40 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda650bfb80 10 client.16671 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda650bfb80 10 client.16671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda650bfb80 20 client.16671 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd4287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda650bfb80 4 client.16671 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618dd33c240 con 0x5618dd426380 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda47fff700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd4287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda2f7fe700 10 client.16671 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.475+0000 7fda47fff700 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd4287a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f9739ad043ad6b6 server_cookie=3a9fb9faae57f2fd in_seq=0 out_seq=0 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3936281926 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618dd33c240 con 0x5618dd426380 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda2f7fe700 10 client.16671 handle_command_reply: tid=0 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda650bfb80 1 client.16671 shutdown 2024-05-14T15:30:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 msgr2=0x5618dd416240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd416240 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fda40009fd0 tx=0x7fda400b2860 comp rx=0 tx=0).stop 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 shutdown_connections 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.476+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd4287a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda2f7fe700 0 client.16671 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd416240 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3936281926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd416780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 >> 172.21.15.5:0/3936281926 conn(0x5618dd2d6600 msgr2=0x5618dd40b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 shutdown_connections 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 1 -- 172.21.15.5:0/3936281926 wait complete. 2024-05-14T15:30:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.477+0000 7fda650bfb80 20 client.16671 trim_cache size 0 max 16384 2024-05-14T15:30:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.492+0000 7fda650bfb80 1 Processor -- start 2024-05-14T15:30:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 -- start start 2024-05-14T15:30:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd407610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd4017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b5100 con 0x5618dd40cc60 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b2990 con 0x5618dd411470 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda650bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b07e0 con 0x5618dd28ee40 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd4017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd4017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44968/0 (socket says 172.21.15.5:44968) 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 -- 172.21.15.5:0/4033945112 learned_addr learned my addr 172.21.15.5:0/4033945112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda4ffff700 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd407610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda54c48700 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 -- 172.21.15.5:0/4033945112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 msgr2=0x5618dd407610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd407610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 -- 172.21.15.5:0/4033945112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 msgr2=0x5618dd416b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda47fff700 1 -- 172.21.15.5:0/4033945112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd401e30 con 0x5618dd40cc60 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.493+0000 7fda4ffff700 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd407610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.494+0000 7fda47fff700 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd4017f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fda50076ab0 tx=0x7fda38009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506223c4f4ad7eff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.494+0000 7fda45ffb700 1 -- 172.21.15.5:0/4033945112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38003e00 con 0x5618dd40cc60 2024-05-14T15:30:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.494+0000 7fda45ffb700 1 -- 172.21.15.5:0/4033945112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3800bc40 con 0x5618dd40cc60 2024-05-14T15:30:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.494+0000 7fda45ffb700 1 -- 172.21.15.5:0/4033945112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38033750 con 0x5618dd40cc60 2024-05-14T15:30:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 -- 172.21.15.5:0/4033945112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 msgr2=0x5618dd4017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd4017f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fda50076ab0 tx=0x7fda38009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 -- 172.21.15.5:0/4033945112 shutdown_connections 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd416b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd28ee40 0x5618dd407610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 --2- 172.21.15.5:0/4033945112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd40cc60 0x5618dd4017f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 -- 172.21.15.5:0/4033945112 >> 172.21.15.5:0/4033945112 conn(0x5618dd2d6600 msgr2=0x5618dd414a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 -- 172.21.15.5:0/4033945112 shutdown_connections 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 -- 172.21.15.5:0/4033945112 wait complete. 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.495+0000 7fda650bfb80 1 -- start start 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 Processor -- start 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd42bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd429060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd4295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b0e50 con 0x5618dd28ee40 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b22c0 con 0x5618dd411470 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b1520 con 0x5618dd40cc60 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda650bfb80 10 client.0 _ll_register_callbacks cb 0x5618dd2c6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd42bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda54c48700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd4295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd42bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44974/0 (socket says 172.21.15.5:44974) 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda4ffff700 1 -- 172.21.15.5:0/3084138257 learned_addr learned my addr 172.21.15.5:0/3084138257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda47fff700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd429060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda54c48700 1 -- 172.21.15.5:0/3084138257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 msgr2=0x5618dd429060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda54c48700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd429060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda54c48700 1 -- 172.21.15.5:0/3084138257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 msgr2=0x5618dd42bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.496+0000 7fda54c48700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd42bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda54c48700 1 -- 172.21.15.5:0/3084138257 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd429cd0 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd42bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda47fff700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd429060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda54c48700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd4295a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda48007cb0 tx=0x7fda480114d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55872e5fe5cfe614 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3084138257 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48011c60 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3084138257 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fda48011e00 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3800bc40 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3084138257 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda4802a050 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda650bfb80 10 client.26297 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.497+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618dd4907d0 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3084138257 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3800bc40 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3084138257 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fda48021470 con 0x5618dd411470 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda650bfb80 10 client.26297 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda650bfb80 10 client.26297 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda650bfb80 20 client.26297 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd492f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda650bfb80 4 client.26297 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618dd3362e0 con 0x5618dd426380 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd492f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda2f7fe700 10 client.26297 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.498+0000 7fda4ffff700 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd492f20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3216fc0c3d9ff7cb server_cookie=a1c9e19a12cfb7a9 in_seq=0 out_seq=0 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda2f7fe700 1 -- 172.21.15.5:0/3084138257 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618dd3362e0 con 0x5618dd426380 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda2f7fe700 10 client.26297 handle_command_reply: tid=0 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 client.26297 shutdown 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 msgr2=0x5618dd4295a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd4295a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda48007cb0 tx=0x7fda480114d0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 shutdown_connections 2024-05-14T15:30:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5618dd426380 0x5618dd492f20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5618dd411470 0x5618dd4295a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd28ee40 0x5618dd42bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda2f7fe700 0 client.26297 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 --2- 172.21.15.5:0/3084138257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5618dd40cc60 0x5618dd429060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 >> 172.21.15.5:0/3084138257 conn(0x5618dd2d6600 msgr2=0x5618dd407370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 shutdown_connections 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 1 -- 172.21.15.5:0/3084138257 wait complete. 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.499+0000 7fda650bfb80 20 client.26297 trim_cache size 0 max 16384 2024-05-14T15:30:27.505 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fda3c0b29f0 msgr2=0x7fda3c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fda3c0b29f0 0x7fda3c0b4f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda38005040 comp rx=0 tx=0).stop 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 msgr2=0x7fda50199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 0x7fda50199aa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9c0 tx=0x7fda4000ecd0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 shutdown_connections 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fda3c0b29f0 0x7fda3c0b4f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda50108760 0x7fda5019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda501034e0 0x7fda50199aa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 --2- 172.21.15.5:0/3543886677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda50107e10 0x7fda50199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 >> 172.21.15.5:0/3543886677 conn(0x7fda500fef30 msgr2=0x7fda501096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 shutdown_connections 2024-05-14T15:30:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:27.500+0000 7fda566ab700 1 -- 172.21.15.5:0/3543886677 wait complete. 2024-05-14T15:30:28.050 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:27 smithi144 ceph-mon[35650]: pgmap v173: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-05-14T15:30:28.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:27 smithi193 ceph-mon[36001]: pgmap v173: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-05-14T15:30:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:27 smithi005 ceph-mon[31559]: pgmap v173: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-05-14T15:30:28.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 494832 11.01 MB/sec execute 109 sec latency 52.818 ms 2024-05-14T15:30:28.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:30:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:29.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 497535 11.07 MB/sec execute 110 sec latency 13.766 ms 2024-05-14T15:30:30.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:29 smithi144 ceph-mon[35650]: pgmap v174: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-14T15:30:30.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:29 smithi193 ceph-mon[36001]: pgmap v174: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-14T15:30:30.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:29 smithi005 ceph-mon[31559]: pgmap v174: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-14T15:30:30.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 499068 11.03 MB/sec execute 111 sec latency 23.157 ms 2024-05-14T15:30:30.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 -- 172.21.15.5:0/2820010231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988108740 msgr2=0x7f39881100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 --2- 172.21.15.5:0/2820010231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988108740 0x7f39881100a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f398000b0d0 tx=0x7f398000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 -- 172.21.15.5:0/2820010231 shutdown_connections 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 --2- 172.21.15.5:0/2820010231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988108740 0x7f39881100a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 --2- 172.21.15.5:0/2820010231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39881034c0 0x7f39881038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 --2- 172.21.15.5:0/2820010231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988107df0 0x7f3988108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.797+0000 7f398d668700 1 -- 172.21.15.5:0/2820010231 >> 172.21.15.5:0/2820010231 conn(0x7f39880fedf0 msgr2=0x7f3988101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.798+0000 7f398d668700 1 -- 172.21.15.5:0/2820010231 shutdown_connections 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.798+0000 7f398d668700 1 -- 172.21.15.5:0/2820010231 wait complete. 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.798+0000 7f398d668700 1 Processor -- start 2024-05-14T15:30:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 -- start start 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39881034c0 0x7f398819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 0x7f398819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988108740 0x7f398819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39881133c0 con 0x7f3988107df0 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113240 con 0x7f3988108740 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f398d668700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3988113540 con 0x7f39881034c0 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 0x7f398819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 0x7f398819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44998/0 (socket says 172.21.15.5:44998) 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 -- 172.21.15.5:0/1309042828 learned_addr learned my addr 172.21.15.5:0/1309042828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f39877fe700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988108740 0x7f398819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f3986ffd700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39881034c0 0x7f398819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 -- 172.21.15.5:0/1309042828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39881034c0 msgr2=0x7f398819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39881034c0 0x7f398819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 -- 172.21.15.5:0/1309042828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988108740 msgr2=0x7f398819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988108740 0x7f398819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f397e7fc700 1 -- 172.21.15.5:0/1309042828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39881a6fb0 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.799+0000 7f39877fe700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988108740 0x7f398819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.800+0000 7f397e7fc700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 0x7f398819c1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f397000e970 tx=0x7f397000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.800+0000 7f3984ff9700 1 -- 172.21.15.5:0/1309042828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970004a30 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.800+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39881a72a0 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.800+0000 7f3984ff9700 1 -- 172.21.15.5:0/1309042828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397000cc90 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.800+0000 7f3984ff9700 1 -- 172.21.15.5:0/1309042828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970005610 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.800+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39881a7830 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.804+0000 7f3984ff9700 1 -- 172.21.15.5:0/1309042828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3970021440 con 0x7f3988107df0 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.804+0000 7f399c07cb80 1 Processor -- start 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 -- start start 2024-05-14T15:30:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f3984ff9700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f39740b2a70 0x7f39740b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f39877fe700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f39740b2a70 0x7f39740b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f3984ff9700 1 -- 172.21.15.5:0/1309042828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f39700e2aa0 con 0x7f3988107df0 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b7441d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74421f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 0x564b74422880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b7433f860 con 0x564b7441dc60 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564b742de7c0 con 0x564b7429fe40 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f399c07cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b7433b820 con 0x564b74422470 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f3986ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b7441d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74421f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f3986ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b7441d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53016/0 (socket says 172.21.15.5:53016) 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f3986ffd700 1 -- 172.21.15.5:0/1692695393 learned_addr learned my addr 172.21.15.5:0/1692695393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.805+0000 7f397e7fc700 1 -- 172.21.15.5:0/1692695393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 msgr2=0x564b74422880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:30:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397e7fc700 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 0x564b74422880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397e7fc700 1 -- 172.21.15.5:0/1692695393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 msgr2=0x564b7441d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397e7fc700 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b7441d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397e7fc700 1 -- 172.21.15.5:0/1692695393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b7428c8d0 con 0x564b7441dc60 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f3986ffd700 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b7441d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f39877fe700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f39740b2a70 0x7f39740b4f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f398000bfd0 tx=0x7f398000b040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397e7fc700 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74421f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f39700dfe30 tx=0x7f39700050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096687bc2f407b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397d7fa700 1 -- 172.21.15.5:0/1692695393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700b2170 con 0x564b7441dc60 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397d7fa700 1 -- 172.21.15.5:0/1692695393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397000ce40 con 0x564b7441dc60 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.806+0000 7f397d7fa700 1 -- 172.21.15.5:0/1692695393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700add70 con 0x564b7441dc60 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 -- 172.21.15.5:0/1692695393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 msgr2=0x564b74421f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74421f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f39700dfe30 tx=0x7f39700050b0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 -- 172.21.15.5:0/1692695393 shutdown_connections 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 0x564b74422880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b7441d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 --2- 172.21.15.5:0/1692695393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74421f30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 -- 172.21.15.5:0/1692695393 >> 172.21.15.5:0/1692695393 conn(0x564b742e7600 msgr2=0x564b744184b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 -- 172.21.15.5:0/1692695393 shutdown_connections 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 -- 172.21.15.5:0/1692695393 wait complete. 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.807+0000 7f399c07cb80 1 -- start start 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 Processor -- start 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b744271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74427730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 0x564b74427c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b7442e830 con 0x564b7441dc60 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564b742a5c30 con 0x564b7429fe40 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b7442e9b0 con 0x564b74422470 2024-05-14T15:30:30.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f399c07cb80 10 client.0 _ll_register_callbacks cb 0x564b742d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:30.944 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:30:30.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74427730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74427730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45024/0 (socket says 172.21.15.5:45024) 2024-05-14T15:30:30.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 -- 172.21.15.5:0/2635690185 learned_addr learned my addr 172.21.15.5:0/2635690185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:30.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f3986ffd700 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b744271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 -- 172.21.15.5:0/2635690185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 msgr2=0x564b74427c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 0x564b74427c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 -- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 msgr2=0x564b744271f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b744271f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.808+0000 7f397e7fc700 1 -- 172.21.15.5:0/2635690185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b742db690 con 0x564b7441dc60 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f397e7fc700 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74427730 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f39700dfe90 tx=0x7f397000c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f23dba2b9c402d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f396e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f396e7fc700 1 -- 172.21.15.5:0/2635690185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700a99c0 con 0x564b7441dc60 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f396e7fc700 1 -- 172.21.15.5:0/2635690185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f39700a9c20 con 0x564b7441dc60 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564b7434d240 con 0x564b7441dc60 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f396e7fc700 1 -- 172.21.15.5:0/2635690185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f39700aa0c0 con 0x564b7441dc60 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 10 client.16689 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b744394b0 con 0x564b7441dc60 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f396e7fc700 1 -- 172.21.15.5:0/2635690185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3970087020 con 0x564b7441dc60 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f396e7fc700 1 -- 172.21.15.5:0/2635690185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f39700b2390 con 0x564b7441dc60 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 10 client.16689 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 20 client.16689 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b7443b980 0x564b7443dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 4 client.16689 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.809+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b743472e0 con 0x564b7443b980 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.810+0000 7f3986ffd700 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b7443b980 0x564b7443dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.810+0000 7f3986ffd700 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b7443b980 0x564b7443dda0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c10f128c888c8d server_cookie=bd17499293644ca5 in_seq=0 out_seq=0 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.810+0000 7f396e7fc700 10 client.16689 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f396e7fc700 1 -- 172.21.15.5:0/2635690185 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b743472e0 con 0x564b7443b980 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f396e7fc700 10 client.16689 handle_command_reply: tid=0 2024-05-14T15:30:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 client.16689 shutdown 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 msgr2=0x564b74427730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74427730 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f39700dfe90 tx=0x7f397000c3d0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 shutdown_connections 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b7443b980 0x564b7443dda0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b74422470 0x564b74427c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7429fe40 0x564b744271f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2635690185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b7441dc60 0x564b74427730 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 >> 172.21.15.5:0/2635690185 conn(0x564b742e7600 msgr2=0x564b74422880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f396e7fc700 0 client.16689 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 shutdown_connections 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 1 -- 172.21.15.5:0/2635690185 wait complete. 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.811+0000 7f399c07cb80 20 client.16689 trim_cache size 0 max 16384 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.827+0000 7f399c07cb80 1 Processor -- start 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.827+0000 7f399c07cb80 1 -- start start 2024-05-14T15:30:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b74423890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b744182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b74427760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f399c07cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39881b0d60 con 0x564b74422470 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f399c07cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39881ae5f0 con 0x564b7441dc60 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f399c07cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39881ac440 con 0x564b7429fe40 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f39877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b74427760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f39877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b74427760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45032/0 (socket says 172.21.15.5:45032) 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b744182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f3986ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b74423890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f39877fe700 1 -- 172.21.15.5:0/4052642044 learned_addr learned my addr 172.21.15.5:0/4052642044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f3986ffd700 1 -- 172.21.15.5:0/4052642044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 msgr2=0x564b744182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f3986ffd700 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b744182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f3986ffd700 1 -- 172.21.15.5:0/4052642044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 msgr2=0x564b74427760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f3986ffd700 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b74427760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.828+0000 7f3986ffd700 1 -- 172.21.15.5:0/4052642044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b74414ae0 con 0x564b7429fe40 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.829+0000 7f3986ffd700 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b74423890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564b7442e9d0 tx=0x7f3978004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b0ad3285b33be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.829+0000 7f397cff9700 1 -- 172.21.15.5:0/4052642044 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978023070 con 0x564b7429fe40 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.829+0000 7f397cff9700 1 -- 172.21.15.5:0/4052642044 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397801ee50 con 0x564b7429fe40 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.829+0000 7f397cff9700 1 -- 172.21.15.5:0/4052642044 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978010690 con 0x564b7429fe40 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 -- 172.21.15.5:0/4052642044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 msgr2=0x564b74423890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b74423890 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x564b7442e9d0 tx=0x7f3978004750 comp rx=0 tx=0).stop 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 -- 172.21.15.5:0/4052642044 shutdown_connections 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b74427760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b74423890 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 --2- 172.21.15.5:0/4052642044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b744182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 -- 172.21.15.5:0/4052642044 >> 172.21.15.5:0/4052642044 conn(0x564b742e7600 msgr2=0x564b74417be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 -- 172.21.15.5:0/4052642044 shutdown_connections 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 -- 172.21.15.5:0/4052642044 wait complete. 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 -- start start 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.830+0000 7f399c07cb80 1 Processor -- start 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b7443dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b7443e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b744a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39881acab0 con 0x564b74422470 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39881adf20 con 0x564b7441dc60 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39881ad180 con 0x564b7429fe40 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f399c07cb80 10 client.0 _ll_register_callbacks cb 0x564b742d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f39877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b744a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b7443e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f39877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b744a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45036/0 (socket says 172.21.15.5:45036) 2024-05-14T15:30:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f39877fe700 1 -- 172.21.15.5:0/2428679585 learned_addr learned my addr 172.21.15.5:0/2428679585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f3986ffd700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b7443dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 -- 172.21.15.5:0/2428679585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 msgr2=0x564b7443dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b7443dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 -- 172.21.15.5:0/2428679585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 msgr2=0x564b744a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b744a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 -- 172.21.15.5:0/2428679585 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b744a4430 con 0x564b7441dc60 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f39877fe700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b744a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f397e7fc700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b7443e290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39700a6c30 tx=0x7f39700a9840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8400ebc8a4d00a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.831+0000 7f396e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f396e7fc700 1 -- 172.21.15.5:0/2428679585 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700c2040 con 0x564b7441dc60 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f396e7fc700 1 -- 172.21.15.5:0/2428679585 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f397000c7e0 con 0x564b7441dc60 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397801ee50 con 0x564b7441dc60 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f396e7fc700 1 -- 172.21.15.5:0/2428679585 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f39700b2170 con 0x564b7441dc60 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 10 client.26315 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b744a4430 con 0x564b7441dc60 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f396e7fc700 1 -- 172.21.15.5:0/2428679585 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397801ee50 con 0x564b7441dc60 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f396e7fc700 1 -- 172.21.15.5:0/2428679585 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f397008d070 con 0x564b7441dc60 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 10 client.26315 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 20 client.26315 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b744a5a00 0x564b744a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 4 client.26315 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b741bf250 con 0x564b744a5a00 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.832+0000 7f3986ffd700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b744a5a00 0x564b744a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f396e7fc700 10 client.26315 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f3986ffd700 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b744a5a00 0x564b744a7e20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c96563e51979a9a server_cookie=7800a2d10381c6c0 in_seq=0 out_seq=0 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f396e7fc700 1 -- 172.21.15.5:0/2428679585 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b741bf250 con 0x564b744a5a00 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f396e7fc700 10 client.26315 handle_command_reply: tid=0 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f399c07cb80 1 client.26315 shutdown 2024-05-14T15:30:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 msgr2=0x564b7443e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b7443e290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39700a6c30 tx=0x7f39700a9840 comp rx=0 tx=0).stop 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.833+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 shutdown_connections 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b744a5a00 0x564b744a7e20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b74422470 0x564b744a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b7429fe40 0x564b7443dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f396e7fc700 0 client.26315 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 --2- 172.21.15.5:0/2428679585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b7441dc60 0x564b7443e290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 >> 172.21.15.5:0/2428679585 conn(0x564b742e7600 msgr2=0x564b74418370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 shutdown_connections 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 1 -- 172.21.15.5:0/2428679585 wait complete. 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f399c07cb80 20 client.26315 trim_cache size 0 max 16384 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f39740b2a70 msgr2=0x7f39740b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f39740b2a70 0x7f39740b4f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f398000bfd0 tx=0x7f398000b040 comp rx=0 tx=0).stop 2024-05-14T15:30:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 msgr2=0x7f398819c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 0x7f398819c1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f397000e970 tx=0x7f397000ec80 comp rx=0 tx=0).stop 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 shutdown_connections 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f39740b2a70 0x7f39740b4f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3988108740 0x7f398819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39881034c0 0x7f398819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 --2- 172.21.15.5:0/1309042828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3988107df0 0x7f398819c1a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.834+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 >> 172.21.15.5:0/1309042828 conn(0x7f39880fedf0 msgr2=0x7f3988107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.835+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 shutdown_connections 2024-05-14T15:30:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:30.835+0000 7f398d668700 1 -- 172.21.15.5:0/1309042828 wait complete. 2024-05-14T15:30:31.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 501483 11.05 MB/sec execute 112 sec latency 18.433 ms 2024-05-14T15:30:32.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:31 smithi144 ceph-mon[35650]: pgmap v175: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-14T15:30:32.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:31 smithi193 ceph-mon[36001]: pgmap v175: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-14T15:30:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:31 smithi005 ceph-mon[31559]: pgmap v175: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 2.3 MiB/s wr, 300 op/s 2024-05-14T15:30:32.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 503036 11.00 MB/sec execute 113 sec latency 121.961 ms 2024-05-14T15:30:32.640 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:30:32.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:30:32.915 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:33.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 505202 11.03 MB/sec execute 114 sec latency 13.496 ms 2024-05-14T15:30:34.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:33 smithi144 ceph-mon[35650]: pgmap v176: 129 pgs: 129 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 484 op/s 2024-05-14T15:30:34.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:33 smithi193 ceph-mon[36001]: pgmap v176: 129 pgs: 129 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 484 op/s 2024-05-14T15:30:34.064 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:33 smithi005 ceph-mon[31559]: pgmap v176: 129 pgs: 129 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 484 op/s 2024-05-14T15:30:34.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 507385 11.03 MB/sec execute 115 sec latency 18.433 ms 2024-05-14T15:30:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 -- 172.21.15.5:0/2169344938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 msgr2=0x7f0e8c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 --2- 172.21.15.5:0/2169344938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c104a60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0099c0 tx=0x7f0e7c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 -- 172.21.15.5:0/2169344938 shutdown_connections 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 --2- 172.21.15.5:0/2169344938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c109880 0x7f0e8c1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 --2- 172.21.15.5:0/2169344938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c104a60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 --2- 172.21.15.5:0/2169344938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8c108f30 0x7f0e8c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.658+0000 7f0e91922700 1 -- 172.21.15.5:0/2169344938 >> 172.21.15.5:0/2169344938 conn(0x7f0e8c079f00 msgr2=0x7f0e8c07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.659+0000 7f0e91922700 1 -- 172.21.15.5:0/2169344938 shutdown_connections 2024-05-14T15:30:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.659+0000 7f0e91922700 1 -- 172.21.15.5:0/2169344938 wait complete. 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.659+0000 7f0e91922700 1 Processor -- start 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 -- start start 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108f30 0x7f0e8c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8c109880 0x7f0e8c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1144b0 con 0x7f0e8c104650 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c114330 con 0x7f0e8c109880 2024-05-14T15:30:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e91922700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c114630 con 0x7f0e8c108f30 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45044/0 (socket says 172.21.15.5:45044) 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e8a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108f30 0x7f0e8c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e8affd700 1 -- 172.21.15.5:0/2831336373 learned_addr learned my addr 172.21.15.5:0/2831336373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.660+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8c109880 0x7f0e8c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8a7fc700 1 -- 172.21.15.5:0/2831336373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8c109880 msgr2=0x7f0e8c076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8c109880 0x7f0e8c076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8a7fc700 1 -- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 msgr2=0x7f0e8c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8a7fc700 1 -- 172.21.15.5:0/2831336373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c1a91c0 con 0x7f0e8c108f30 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8affd700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108f30 0x7f0e8c076000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000e9c0 tx=0x7f0e8000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e90920700 1 -- 172.21.15.5:0/2831336373 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000c9c0 con 0x7f0e8c108f30 2024-05-14T15:30:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e90920700 1 -- 172.21.15.5:0/2831336373 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e8000cb60 con 0x7f0e8c108f30 2024-05-14T15:30:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c1a94b0 con 0x7f0e8c108f30 2024-05-14T15:30:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e90920700 1 -- 172.21.15.5:0/2831336373 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80018680 con 0x7f0e8c108f30 2024-05-14T15:30:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.661+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1a99f0 con 0x7f0e8c108f30 2024-05-14T15:30:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.664+0000 7f0e90920700 1 -- 172.21.15.5:0/2831336373 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0e80010430 con 0x7f0e8c108f30 2024-05-14T15:30:34.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.665+0000 7f0e90920700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e740b29f0 0x7f0e740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.665+0000 7f0e90920700 1 -- 172.21.15.5:0/2831336373 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0e800e2b50 con 0x7f0e8c108f30 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.665+0000 7f0e8affd700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e740b29f0 0x7f0e740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.665+0000 7f0ea0336b80 1 Processor -- start 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.665+0000 7f0ea0336b80 1 -- start start 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.665+0000 7f0e8affd700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e740b29f0 0x7f0e740b4f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009990 tx=0x7f0e7c01a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 0x55fca0158900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fca00758e0 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fca0014840 con 0x55fca01584f0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fca00718a0 con 0x55fca0153ce0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 0x55fca0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 0x55fca0158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53066/0 (socket says 172.21.15.5:53066) 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/3912815183 learned_addr learned my addr 172.21.15.5:0/3912815183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45050/0 (socket says 172.21.15.5:45050) 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 -- 172.21.15.5:0/3912815183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 msgr2=0x55fca0158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 0x55fca0158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 -- 172.21.15.5:0/3912815183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 msgr2=0x55fca01537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 -- 172.21.15.5:0/3912815183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9ffc2950 con 0x55fca0153ce0 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 0x55fca0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:34.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca0157fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e800dfee0 tx=0x7f0e800a98c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787a605172ab4fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:34.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.666+0000 7f0e71ffb700 1 -- 172.21.15.5:0/3912815183 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e800a9a70 con 0x55fca0153ce0 2024-05-14T15:30:34.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0e71ffb700 1 -- 172.21.15.5:0/3912815183 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e800ade20 con 0x55fca0153ce0 2024-05-14T15:30:34.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0e71ffb700 1 -- 172.21.15.5:0/3912815183 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e800b26a0 con 0x55fca0153ce0 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 -- 172.21.15.5:0/3912815183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 msgr2=0x55fca0157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca0157fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e800dfee0 tx=0x7f0e800a98c0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 -- 172.21.15.5:0/3912815183 shutdown_connections 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca01584f0 0x55fca0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/3912815183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca0157fb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.667+0000 7f0ea0336b80 1 -- 172.21.15.5:0/3912815183 >> 172.21.15.5:0/3912815183 conn(0x55fca001d680 msgr2=0x55fca014e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.668+0000 7f0ea0336b80 1 -- 172.21.15.5:0/3912815183 shutdown_connections 2024-05-14T15:30:34.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.668+0000 7f0ea0336b80 1 -- 172.21.15.5:0/3912815183 wait complete. 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.668+0000 7f0ea0336b80 1 -- start start 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.668+0000 7f0ea0336b80 1 Processor -- start 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.668+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.668+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fca01604a0 con 0x55fca01584f0 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fc9ffdbcb0 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fca0160620 con 0x55fca0153ce0 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0ea0336b80 10 client.0 _ll_register_callbacks cb 0x55fca000d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45062/0 (socket says 172.21.15.5:45062) 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/1564816274 learned_addr learned my addr 172.21.15.5:0/1564816274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8affd700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/1564816274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 msgr2=0x55fca015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 msgr2=0x55fca015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/1564816274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca0011710 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015dd40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0e84004340 tx=0x7f0e84004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4c84f5ec29a4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.669+0000 7f0e6affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0e6affd700 1 -- 172.21.15.5:0/1564816274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401a070 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0e6affd700 1 -- 172.21.15.5:0/1564816274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0e840079d0 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e800ade20 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0e6affd700 1 -- 172.21.15.5:0/1564816274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e84026050 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 10 client.16707 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fca016b1d0 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0e6affd700 1 -- 172.21.15.5:0/1564816274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e800ade20 con 0x55fca01584f0 2024-05-14T15:30:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0e6affd700 1 -- 172.21.15.5:0/1564816274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0e8401e3b0 con 0x55fca01584f0 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 20 client.16707 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca016d760 0x55fca016fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 4 client.16707 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.670+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca00832c0 con 0x55fca016d760 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.671+0000 7f0e8affd700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca016d760 0x55fca016fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.671+0000 7f0e6affd700 10 client.16707 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.671+0000 7f0e8affd700 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca016d760 0x55fca016fba0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e32acfea92eec87 server_cookie=35cd4c5a9b6b1c25 in_seq=0 out_seq=0 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0e6affd700 1 -- 172.21.15.5:0/1564816274 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fca00832c0 con 0x55fca016d760 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0e6affd700 10 client.16707 handle_command_reply: tid=0 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 client.16707 shutdown 2024-05-14T15:30:34.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 msgr2=0x55fca015dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015dd40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0e84004340 tx=0x7f0e84004370 comp rx=0 tx=0).stop 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 shutdown_connections 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca016d760 0x55fca016fba0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015dd40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/1564816274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 >> 172.21.15.5:0/1564816274 conn(0x55fca001d680 msgr2=0x55fca01524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0e6affd700 0 client.16707 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 shutdown_connections 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 1 -- 172.21.15.5:0/1564816274 wait complete. 2024-05-14T15:30:34.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.672+0000 7f0ea0336b80 20 client.16707 trim_cache size 0 max 16384 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 Processor -- start 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 -- start start 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca014e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b51c0 con 0x55fca01584f0 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b2a50 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.692+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b08a0 con 0x55fca0153ce0 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45074/0 (socket says 172.21.15.5:45074) 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/672864375 learned_addr learned my addr 172.21.15.5:0/672864375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:34.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/672864375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 msgr2=0x55fca015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/672864375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 msgr2=0x55fca014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 -- 172.21.15.5:0/672864375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca0148e70 con 0x55fca01584f0 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8affd700 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015d280 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0e84002230 tx=0x7f0e84004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4fd71a2029dc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e717fa700 1 -- 172.21.15.5:0/672864375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8402f070 con 0x55fca01584f0 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e717fa700 1 -- 172.21.15.5:0/672864375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e84023070 con 0x55fca01584f0 2024-05-14T15:30:34.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.693+0000 7f0e717fa700 1 -- 172.21.15.5:0/672864375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401a050 con 0x55fca01584f0 2024-05-14T15:30:34.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 -- 172.21.15.5:0/672864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 msgr2=0x55fca015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015d280 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0e84002230 tx=0x7f0e84004850 comp rx=0 tx=0).stop 2024-05-14T15:30:34.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 -- 172.21.15.5:0/672864375 shutdown_connections 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca01584f0 0x55fca015d280 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fc9ffd5ec0 0x55fca014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/672864375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca0153ce0 0x55fca015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 -- 172.21.15.5:0/672864375 >> 172.21.15.5:0/672864375 conn(0x55fca001d680 msgr2=0x55fca015ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 -- 172.21.15.5:0/672864375 shutdown_connections 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.694+0000 7f0ea0336b80 1 -- 172.21.15.5:0/672864375 wait complete. 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 -- start start 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 Processor -- start 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca0153ce0 0x55fca0165df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca01584f0 0x55fca0162880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b0f10 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b2380 con 0x55fca0153ce0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1b15e0 con 0x55fca01584f0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0ea0336b80 10 client.0 _ll_register_callbacks cb 0x55fca000d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0e8a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca0153ce0 0x55fca0165df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0e8affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45086/0 (socket says 172.21.15.5:45086) 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.695+0000 7f0e8affd700 1 -- 172.21.15.5:0/91408661 learned_addr learned my addr 172.21.15.5:0/91408661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8b7fe700 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca01584f0 0x55fca0162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8affd700 1 -- 172.21.15.5:0/91408661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca01584f0 msgr2=0x55fca0162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8affd700 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca01584f0 0x55fca0162880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8affd700 1 -- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca0153ce0 msgr2=0x55fca0165df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8affd700 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca0153ce0 0x55fca0165df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8affd700 1 -- 172.21.15.5:0/91408661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca0162dc0 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e8affd700 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01658b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00b580 tx=0x7f0e7c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dcd802a2606da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e6affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e6affd700 1 -- 172.21.15.5:0/91408661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c005580 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e84023070 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e6affd700 1 -- 172.21.15.5:0/91408661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0e7c0323b0 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0e6affd700 1 -- 172.21.15.5:0/91408661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c041050 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0ea0336b80 10 client.16713 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.696+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fca01d8720 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0e6affd700 1 -- 172.21.15.5:0/91408661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e84023070 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0e6affd700 1 -- 172.21.15.5:0/91408661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0e7c00f7d0 con 0x55fc9ffd5ec0 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0ea0336b80 10 client.16713 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0ea0336b80 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0ea0336b80 20 client.16713 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca01d8e60 0x55fca01db280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0ea0336b80 4 client.16713 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fca007d360 con 0x55fca01d8e60 2024-05-14T15:30:34.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.697+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca01d8e60 0x55fca01db280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:34.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.698+0000 7f0e8a7fc700 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca01d8e60 0x55fca01db280 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40b2f5be47adc87 server_cookie=1c03a540769e2488 in_seq=0 out_seq=0 2024-05-14T15:30:34.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.698+0000 7f0e6affd700 10 client.16713 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.698+0000 7f0e6affd700 1 -- 172.21.15.5:0/91408661 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fca007d360 con 0x55fca01d8e60 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.698+0000 7f0e6affd700 10 client.16713 handle_command_reply: tid=0 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.698+0000 7f0ea0336b80 1 client.16713 shutdown 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 msgr2=0x55fca01658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01658b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00b580 tx=0x7f0e7c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 shutdown_connections 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fca01d8e60 0x55fca01db280 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fca01584f0 0x55fca0162880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9ffd5ec0 0x55fca01658b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0e6affd700 0 client.16713 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 --2- 172.21.15.5:0/91408661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fca0153ce0 0x55fca0165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 >> 172.21.15.5:0/91408661 conn(0x55fca001d680 msgr2=0x55fca014e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 shutdown_connections 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 1 -- 172.21.15.5:0/91408661 wait complete. 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.699+0000 7f0ea0336b80 20 client.16713 trim_cache size 0 max 16384 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9220e780-26a3-47f5-9059-4374104a9692", 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:30:34.706 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e740b29f0 msgr2=0x7f0e740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e740b29f0 0x7f0e740b4f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009990 tx=0x7f0e7c01a040 comp rx=0 tx=0).stop 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108f30 msgr2=0x7f0e8c076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108f30 0x7f0e8c076000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000e9c0 tx=0x7f0e8000ecd0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 shutdown_connections 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0e740b29f0 0x7f0e740b4f00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8c109880 0x7f0e8c076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c104650 0x7f0e8c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 --2- 172.21.15.5:0/2831336373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e8c108f30 0x7f0e8c076000 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 >> 172.21.15.5:0/2831336373 conn(0x7f0e8c079f00 msgr2=0x7f0e8c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 shutdown_connections 2024-05-14T15:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:34.700+0000 7f0e91922700 1 -- 172.21.15.5:0/2831336373 wait complete. 2024-05-14T15:30:35.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 508899 11.01 MB/sec execute 116 sec latency 55.089 ms 2024-05-14T15:30:35.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:30:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:36.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:35 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:30:36.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:35 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:36.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:35 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:30:36.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:35 smithi144 ceph-mon[35650]: pgmap v177: 129 pgs: 129 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.3 MiB/s wr, 381 op/s 2024-05-14T15:30:36.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:35 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:36.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:35 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:30:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:35 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:30:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:35 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:35 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:30:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:35 smithi193 ceph-mon[36001]: pgmap v177: 129 pgs: 129 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.3 MiB/s wr, 381 op/s 2024-05-14T15:30:36.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:35 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:36.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:35 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:30:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:35 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:30:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:35 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:35 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:30:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:35 smithi005 ceph-mon[31559]: pgmap v177: 129 pgs: 129 active+clean; 54 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.3 MiB/s wr, 381 op/s 2024-05-14T15:30:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:35 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:36.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:35 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:30:36.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 511591 11.06 MB/sec execute 117 sec latency 15.006 ms 2024-05-14T15:30:37.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 512965 11.02 MB/sec execute 118 sec latency 90.725 ms 2024-05-14T15:30:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.484+0000 7fa038ed6700 1 -- 172.21.15.5:0/1702355602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 msgr2=0x7fa0341049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.484+0000 7fa038ed6700 1 --2- 172.21.15.5:0/1702355602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa0341049f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa0240099c0 tx=0x7fa024009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 -- 172.21.15.5:0/1702355602 shutdown_connections 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 --2- 172.21.15.5:0/1702355602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034109860 0x7fa0341111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 --2- 172.21.15.5:0/1702355602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa0341049f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 --2- 172.21.15.5:0/1702355602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa034108f10 0x7fa034109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 -- 172.21.15.5:0/1702355602 >> 172.21.15.5:0/1702355602 conn(0x7fa0340ffe70 msgr2=0x7fa0341022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 -- 172.21.15.5:0/1702355602 shutdown_connections 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.485+0000 7fa038ed6700 1 -- 172.21.15.5:0/1702355602 wait complete. 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 Processor -- start 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 -- start start 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa03419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034108f10 0x7fa03419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa034109860 0x7fa03419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341144e0 con 0x7fa0341045e0 2024-05-14T15:30:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa034114360 con 0x7fa034108f10 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa038ed6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa034114660 con 0x7fa034109860 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa03259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa03419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa03259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa03419ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54412/0 (socket says 172.21.15.5:54412) 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa03259c700 1 -- 172.21.15.5:0/776751583 learned_addr learned my addr 172.21.15.5:0/776751583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa029d9b700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034108f10 0x7fa03419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.486+0000 7fa032d9d700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa034109860 0x7fa03419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa03259c700 1 -- 172.21.15.5:0/776751583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa034109860 msgr2=0x7fa03419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa03259c700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa034109860 0x7fa03419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa03259c700 1 -- 172.21.15.5:0/776751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034108f10 msgr2=0x7fa03419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa03259c700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034108f10 0x7fa03419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa03259c700 1 -- 172.21.15.5:0/776751583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03419fd40 con 0x7fa0341045e0 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa029d9b700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034108f10 0x7fa03419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa03259c700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa03419ae70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa024005910 tx=0x7fa024004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa02bfff700 1 -- 172.21.15.5:0/776751583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024026070 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa02bfff700 1 -- 172.21.15.5:0/776751583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa024003c70 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03419ffd0 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa02bfff700 1 -- 172.21.15.5:0/776751583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400fa40 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.487+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0341a04b0 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.490+0000 7fa02bfff700 1 -- 172.21.15.5:0/776751583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa024029070 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.490+0000 7fa02bfff700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0200b29f0 0x7fa0200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 Processor -- start 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa029d9b700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0200b29f0 0x7fa0200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa02bfff700 1 -- 172.21.15.5:0/776751583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fa0240e3150 con 0x7fa0341045e0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 -- start start 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2eef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa029d9b700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0200b29f0 0x7fa0200b4f00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa0341088b0 tx=0x7fa01c009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e9c20c860 con 0x555e9c2eac60 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e9c1ab7c0 con 0x555e9c2ef470 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa0478eab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e9c208820 con 0x555e9c16ce40 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa029d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa029d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2eef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54426/0 (socket says 172.21.15.5:54426) 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.491+0000 7fa032d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa029d9b700 1 -- 172.21.15.5:0/2984071833 learned_addr learned my addr 172.21.15.5:0/2984071833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa03259c700 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa032d9d700 1 -- 172.21.15.5:0/2984071833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 msgr2=0x555e9c2ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa032d9d700 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa032d9d700 1 -- 172.21.15.5:0/2984071833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 msgr2=0x555e9c2eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa032d9d700 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa032d9d700 1 -- 172.21.15.5:0/2984071833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9c1598d0 con 0x555e9c2ef470 2024-05-14T15:30:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa032d9d700 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2ef880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa02c011580 tx=0x7fa02c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71056e4e7305ca5e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa028d99700 1 -- 172.21.15.5:0/2984071833 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c011e80 con 0x555e9c2ef470 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa028d99700 1 -- 172.21.15.5:0/2984071833 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c004bb0 con 0x555e9c2ef470 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.492+0000 7fa028d99700 1 -- 172.21.15.5:0/2984071833 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c004720 con 0x555e9c2ef470 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 -- 172.21.15.5:0/2984071833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 msgr2=0x555e9c2ef880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2ef880 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa02c011580 tx=0x7fa02c009920 comp rx=0 tx=0).stop 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 -- 172.21.15.5:0/2984071833 shutdown_connections 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2ef880 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2984071833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 -- 172.21.15.5:0/2984071833 >> 172.21.15.5:0/2984071833 conn(0x555e9c1b4600 msgr2=0x555e9c2e54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 -- 172.21.15.5:0/2984071833 shutdown_connections 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.493+0000 7fa0478eab80 1 -- 172.21.15.5:0/2984071833 wait complete. 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 -- start start 2024-05-14T15:30:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 Processor -- start 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e9c2f7420 con 0x555e9c2eac60 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e9c172c30 con 0x555e9c2ef470 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e9c2f75a0 con 0x555e9c16ce40 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa0478eab80 10 client.0 _ll_register_callbacks cb 0x555e9c1a4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa029d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa029d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54442/0 (socket says 172.21.15.5:54442) 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa029d9b700 1 -- 172.21.15.5:0/1058727810 learned_addr learned my addr 172.21.15.5:0/1058727810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa03259c700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.494+0000 7fa032d9d700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa029d9b700 1 -- 172.21.15.5:0/1058727810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 msgr2=0x555e9c2f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa029d9b700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa029d9b700 1 -- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 msgr2=0x555e9c2f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa029d9b700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa029d9b700 1 -- 172.21.15.5:0/1058727810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9c1a8690 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa029d9b700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2f4600 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009f10 tx=0x7fa01c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441362f99941950c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa019ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa019ffb700 1 -- 172.21.15.5:0/1058727810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c01e6b0 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa019ffb700 1 -- 172.21.15.5:0/1058727810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fa01c01e850 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa02c004bb0 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa019ffb700 1 -- 172.21.15.5:0/1058727810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa01c030050 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa0478eab80 10 client.16725 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e9c301fc0 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.495+0000 7fa019ffb700 1 -- 172.21.15.5:0/1058727810 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c004bb0 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa019ffb700 1 -- 172.21.15.5:0/1058727810 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa01c028ba0 con 0x555e9c2eac60 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa0478eab80 10 client.16725 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa0478eab80 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa0478eab80 20 client.16725 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c304650 0x555e9c306a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa0478eab80 4 client.16725 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e9c21a240 con 0x555e9c304650 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa03259c700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c304650 0x555e9c306a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa019ffb700 10 client.16725 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.496+0000 7fa03259c700 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c304650 0x555e9c306a70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd233925326f848a server_cookie=c8c0af0e214a2275 in_seq=0 out_seq=0 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa019ffb700 1 -- 172.21.15.5:0/1058727810 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e9c21a240 con 0x555e9c304650 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa019ffb700 10 client.16725 handle_command_reply: tid=0 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 client.16725 shutdown 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 msgr2=0x555e9c2f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2f4600 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009f10 tx=0x7fa01c0047c0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 shutdown_connections 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c304650 0x555e9c306a70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa019ffb700 0 client.16725 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2ef470 0x555e9c2f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1058727810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2eac60 0x555e9c2f4600 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 >> 172.21.15.5:0/1058727810 conn(0x555e9c1b4600 msgr2=0x555e9c2e9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 shutdown_connections 2024-05-14T15:30:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 1 -- 172.21.15.5:0/1058727810 wait complete. 2024-05-14T15:30:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.497+0000 7fa0478eab80 20 client.16725 trim_cache size 0 max 16384 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 Processor -- start 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 -- start start 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2eac60 0x555e9c3004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c30c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341b2e10 con 0x555e9c2ef470 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341b06a0 con 0x555e9c2eac60 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa0478eab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341ae4f0 con 0x555e9c16ce40 2024-05-14T15:30:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa03259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa032d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c30c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa03259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2eee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:34776/0 (socket says 172.21.15.5:34776) 2024-05-14T15:30:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa03259c700 1 -- 172.21.15.5:0/2606755915 learned_addr learned my addr 172.21.15.5:0/2606755915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa032d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c30c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54444/0 (socket says 172.21.15.5:54444) 2024-05-14T15:30:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.519+0000 7fa029d9b700 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2eac60 0x555e9c3004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa03259c700 1 -- 172.21.15.5:0/2606755915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2eac60 msgr2=0x555e9c3004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa03259c700 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2eac60 0x555e9c3004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa03259c700 1 -- 172.21.15.5:0/2606755915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 msgr2=0x555e9c30c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa03259c700 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c30c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa03259c700 1 -- 172.21.15.5:0/2606755915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9c2eb740 con 0x555e9c16ce40 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa032d9d700 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c30c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa03259c700 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2eee30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0240aa090 tx=0x7fa0240036f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c94d4d94bf1e6e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa01bfff700 1 -- 172.21.15.5:0/2606755915 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240aa800 con 0x555e9c16ce40 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa01bfff700 1 -- 172.21.15.5:0/2606755915 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0240aa100 con 0x555e9c16ce40 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.520+0000 7fa01bfff700 1 -- 172.21.15.5:0/2606755915 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240aa5c0 con 0x555e9c16ce40 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 -- 172.21.15.5:0/2606755915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 msgr2=0x555e9c2eee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2eee30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0240aa090 tx=0x7fa0240036f0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 -- 172.21.15.5:0/2606755915 shutdown_connections 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c30c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c16ce40 0x555e9c2eee30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 --2- 172.21.15.5:0/2606755915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c2eac60 0x555e9c3004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 -- 172.21.15.5:0/2606755915 >> 172.21.15.5:0/2606755915 conn(0x555e9c1b4600 msgr2=0x555e9c2ee720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 -- 172.21.15.5:0/2606755915 shutdown_connections 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 -- 172.21.15.5:0/2606755915 wait complete. 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.521+0000 7fa0478eab80 1 -- start start 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 Processor -- start 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 0x555e9c2fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c2eac60 0x555e9c2fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c36ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341aeb60 con 0x555e9c2ef470 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341affd0 con 0x555e9c16ce40 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341af230 con 0x555e9c2eac60 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa0478eab80 10 client.0 _ll_register_callbacks cb 0x555e9c1a4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c36ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa03259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 0x555e9c2fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa03259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 0x555e9c2fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:54152/0 (socket says 172.21.15.5:54152) 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c36ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54454/0 (socket says 172.21.15.5:54454) 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa03259c700 1 -- 172.21.15.5:0/1558482796 learned_addr learned my addr 172.21.15.5:0/1558482796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa029d9b700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c2eac60 0x555e9c2fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 -- 172.21.15.5:0/1558482796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c2eac60 msgr2=0x555e9c2fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c2eac60 0x555e9c2fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 -- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 msgr2=0x555e9c2fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 0x555e9c2fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa032d9d700 1 -- 172.21.15.5:0/1558482796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9c36e330 con 0x555e9c2ef470 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa029d9b700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c2eac60 0x555e9c2fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.522+0000 7fa03259c700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 0x555e9c2fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa032d9d700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c36ddf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa02c000f80 tx=0x7fa02c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3ef8a8b77632c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa019ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa019ffb700 1 -- 172.21.15.5:0/1558482796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c02c060 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0240aa100 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa019ffb700 1 -- 172.21.15.5:0/1558482796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fa02c003cc0 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa019ffb700 1 -- 172.21.15.5:0/1558482796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa02c033050 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa0478eab80 10 client.16731 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e9c36f410 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.523+0000 7fa019ffb700 1 -- 172.21.15.5:0/1558482796 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0240aa100 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa019ffb700 1 -- 172.21.15.5:0/1558482796 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa02c007620 con 0x555e9c2ef470 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa0478eab80 10 client.16731 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa0478eab80 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa0478eab80 20 client.16731 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c36f790 0x555e9c371bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa0478eab80 4 client.16731 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e9c2142e0 con 0x555e9c36f790 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa03259c700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c36f790 0x555e9c371bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa03259c700 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c36f790 0x555e9c371bb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf7f10020020cbc server_cookie=3f3ac0fa57bcbc11 in_seq=0 out_seq=0 2024-05-14T15:30:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.524+0000 7fa019ffb700 10 client.16731 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa019ffb700 1 -- 172.21.15.5:0/1558482796 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e9c2142e0 con 0x555e9c36f790 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa019ffb700 10 client.16731 handle_command_reply: tid=0 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 client.16731 shutdown 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 msgr2=0x555e9c36ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c36ddf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa02c000f80 tx=0x7fa02c009510 comp rx=0 tx=0).stop 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 shutdown_connections 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555e9c36f790 0x555e9c371bb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9c2ef470 0x555e9c36ddf0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e9c16ce40 0x555e9c2fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 --2- 172.21.15.5:0/1558482796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e9c2eac60 0x555e9c2fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 >> 172.21.15.5:0/1558482796 conn(0x555e9c1b4600 msgr2=0x555e9c2e4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa019ffb700 0 client.16731 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 shutdown_connections 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 1 -- 172.21.15.5:0/1558482796 wait complete. 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.525+0000 7fa0478eab80 20 client.16731 trim_cache size 0 max 16384 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:30:37.533 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0200b29f0 msgr2=0x7fa0200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0200b29f0 0x7fa0200b4f00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa0341088b0 tx=0x7fa01c009260 comp rx=0 tx=0).stop 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 msgr2=0x7fa03419ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa03419ae70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa024005910 tx=0x7fa024004270 comp rx=0 tx=0).stop 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 shutdown_connections 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa0200b29f0 0x7fa0200b4f00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa034109860 0x7fa03419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0341045e0 0x7fa03419ae70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 --2- 172.21.15.5:0/776751583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa034108f10 0x7fa03419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 >> 172.21.15.5:0/776751583 conn(0x7fa0340ffe70 msgr2=0x7fa0341071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 shutdown_connections 2024-05-14T15:30:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:37.526+0000 7fa038ed6700 1 -- 172.21.15.5:0/776751583 wait complete. 2024-05-14T15:30:38.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:37 smithi144 ceph-mon[35650]: pgmap v178: 129 pgs: 129 active+clean; 59 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-05-14T15:30:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:37 smithi193 ceph-mon[36001]: pgmap v178: 129 pgs: 129 active+clean; 59 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-05-14T15:30:38.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:37 smithi005 ceph-mon[31559]: pgmap v178: 129 pgs: 129 active+clean; 59 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 447 op/s 2024-05-14T15:30:38.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 515138 11.03 MB/sec execute 119 sec latency 18.060 ms 2024-05-14T15:30:38.286 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:30:38.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:30:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:39.138 INFO:tasks.workunit.client.0.smithi005.stdout: 1 516683 11.00 MB/sec execute 120 sec latency 13.964 ms 2024-05-14T15:30:39.998 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:39 smithi005 ceph-mon[31559]: pgmap v179: 129 pgs: 129 active+clean; 57 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 2.5 MiB/s wr, 367 op/s 2024-05-14T15:30:40.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:39 smithi144 ceph-mon[35650]: pgmap v179: 129 pgs: 129 active+clean; 57 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 2.5 MiB/s wr, 367 op/s 2024-05-14T15:30:40.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:39 smithi193 ceph-mon[36001]: pgmap v179: 129 pgs: 129 active+clean; 57 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 2.5 MiB/s wr, 367 op/s 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 -- 172.21.15.5:0/1934145704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc109ca0 msgr2=0x7fb1bc103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1934145704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc109ca0 0x7fb1bc103840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009900 tx=0x7fb1b802a410 comp rx=0 tx=0).stop 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 -- 172.21.15.5:0/1934145704 shutdown_connections 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1934145704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1bc10a140 0x7fb1bc103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1934145704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1bc1043f0 0x7fb1bc1107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1934145704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc109ca0 0x7fb1bc103840 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 -- 172.21.15.5:0/1934145704 >> 172.21.15.5:0/1934145704 conn(0x7fb1bc0ff340 msgr2=0x7fb1bc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 -- 172.21.15.5:0/1934145704 shutdown_connections 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.052+0000 7fb1c3376640 1 -- 172.21.15.5:0/1934145704 wait complete. 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 Processor -- start 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 -- start start 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1bc1043f0 0x7fb1bc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1bc109ca0 0x7fb1bc1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 0x7fb1bc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc113ef0 con 0x7fb1bc10a140 2024-05-14T15:30:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc113d70 con 0x7fb1bc109ca0 2024-05-14T15:30:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c3376640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc114070 con 0x7fb1bc1043f0 2024-05-14T15:30:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c1913640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 0x7fb1bc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c1913640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 0x7fb1bc19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54480/0 (socket says 172.21.15.5:54480) 2024-05-14T15:30:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.053+0000 7fb1c1913640 1 -- 172.21.15.5:0/1093025226 learned_addr learned my addr 172.21.15.5:0/1093025226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1bc1043f0 0x7fb1bc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1bc109ca0 0x7fb1bc1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1913640 1 -- 172.21.15.5:0/1093025226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1bc1043f0 msgr2=0x7fb1bc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1913640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1bc1043f0 0x7fb1bc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1913640 1 -- 172.21.15.5:0/1093025226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1bc109ca0 msgr2=0x7fb1bc1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1913640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1bc109ca0 0x7fb1bc1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1913640 1 -- 172.21.15.5:0/1093025226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc19aed0 con 0x7fb1bc10a140 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c1913640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 0x7fb1bc19a830 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009fd0 tx=0x7fb1b400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1b27fc640 1 -- 172.21.15.5:0/1093025226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4017070 con 0x7fb1bc10a140 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1b27fc640 1 -- 172.21.15.5:0/1093025226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b4003e20 con 0x7fb1bc10a140 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1b27fc640 1 -- 172.21.15.5:0/1093025226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4010460 con 0x7fb1bc10a140 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1bc19b1c0 con 0x7fb1bc10a140 2024-05-14T15:30:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.054+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1bc1a7450 con 0x7fb1bc10a140 2024-05-14T15:30:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 Processor -- start 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1b27fc640 1 -- 172.21.15.5:0/1093025226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb1b4003970 con 0x7fb1bc10a140 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 -- start start 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f4717a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f4717fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f4718510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557f467d5d0 con 0x5557f434e2f0 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5557f4636890 con 0x5557f43564d0 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557f4679590 con 0x5557f43771b0 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1c1112640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f4717fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1c1112640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f4717fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54490/0 (socket says 172.21.15.5:54490) 2024-05-14T15:30:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.058+0000 7fb1c1112640 1 -- 172.21.15.5:0/1267421862 learned_addr learned my addr 172.21.15.5:0/1267421862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f4718510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f4717a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c1112640 1 -- 172.21.15.5:0/1267421862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 msgr2=0x5557f4717a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f4717a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c1112640 1 -- 172.21.15.5:0/1267421862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 msgr2=0x5557f4718510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f4718510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c1112640 1 -- 172.21.15.5:0/1267421862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f438f210 con 0x5557f434e2f0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1b27fc640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1a80b29d0 0x7fb1a80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f4717fd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009900 tx=0x7fb1b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4a48f246d04cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1a80b29d0 0x7fb1a80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1b27fc640 1 -- 172.21.15.5:0/1093025226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb1b40e26d0 con 0x7fb1bc10a140 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1a77fe640 1 -- 172.21.15.5:0/1267421862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8040070 con 0x5557f434e2f0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1a77fe640 1 -- 172.21.15.5:0/1267421862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b8005db0 con 0x5557f434e2f0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.059+0000 7fb1a77fe640 1 -- 172.21.15.5:0/1267421862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8035040 con 0x5557f434e2f0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1267421862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 msgr2=0x5557f4717fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1a80b29d0 0x7fb1a80b4ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00a010 tx=0x7fb1ac008040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f4717fd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009900 tx=0x7fb1b8000bc0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1267421862 shutdown_connections 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f4718510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f4717fd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1267421862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f4717a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1267421862 >> 172.21.15.5:0/1267421862 conn(0x5557f436b300 msgr2=0x5557f4714340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1267421862 shutdown_connections 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.060+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1267421862 wait complete. 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 -- start start 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 Processor -- start 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4726620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f4726b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f47325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557f47290c0 con 0x5557f43564d0 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5557f463a370 con 0x5557f434e2f0 2024-05-14T15:30:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557f4729240 con 0x5557f43771b0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1ca22e740 10 client.0 _ll_register_callbacks cb 0x5557f438ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1c0911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f4726b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1c0911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f4726b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54494/0 (socket says 172.21.15.5:54494) 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1c0911640 1 -- 172.21.15.5:0/175836817 learned_addr learned my addr 172.21.15.5:0/175836817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1c1913640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f47325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.061+0000 7fb1c1112640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4726620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1c0911640 1 -- 172.21.15.5:0/175836817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 msgr2=0x5557f47325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1c0911640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f47325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1c0911640 1 -- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 msgr2=0x5557f4726620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1c0911640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4726620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1c0911640 1 -- 172.21.15.5:0/175836817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f445b620 con 0x5557f43564d0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1c0911640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f4726b60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac016c00 tx=0x7fb1ac0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72f1bf30eef3ced server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/175836817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac028070 con 0x5557f43564d0 2024-05-14T15:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/175836817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb1ac0243f0 con 0x5557f43564d0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b8005db0 con 0x5557f43564d0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/175836817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1ac034050 con 0x5557f43564d0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1ca22e740 10 client.16743 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.062+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557f4735440 con 0x5557f43564d0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/175836817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b8005db0 con 0x5557f43564d0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/175836817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb1ac023070 con 0x5557f43564d0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1ca22e740 10 client.16743 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1ca22e740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1ca22e740 20 client.16743 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f4335d80 0x5557f4738980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1ca22e740 4 client.16743 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f468b1e0 con 0x5557f4335d80 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1c1112640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f4335d80 0x5557f4738980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1a4ff9640 10 client.16743 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.063+0000 7fb1c1112640 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f4335d80 0x5557f4738980 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ffbb64ccd4483f server_cookie=c605db483cea966c in_seq=0 out_seq=0 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/175836817 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557f468b1e0 con 0x5557f4335d80 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1a4ff9640 10 client.16743 handle_command_reply: tid=0 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 client.16743 shutdown 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 msgr2=0x5557f4726b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f4726b60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac016c00 tx=0x7fb1ac0046e0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 shutdown_connections 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f4335d80 0x5557f4738980 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f43771b0 0x5557f47325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f4726b60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/175836817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4726620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1a4ff9640 0 client.16743 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 >> 172.21.15.5:0/175836817 conn(0x5557f436b300 msgr2=0x5557f4719890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.064+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 shutdown_connections 2024-05-14T15:30:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.065+0000 7fb1ca22e740 1 -- 172.21.15.5:0/175836817 wait complete. 2024-05-14T15:30:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.065+0000 7fb1ca22e740 20 client.16743 trim_cache size 0 max 16384 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 Processor -- start 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 -- start start 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f4725fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4719fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f473e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1b1a40 con 0x5557f43564d0 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1af310 con 0x5557f434e2f0 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1ad160 con 0x5557f4335d80 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f473e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1112640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f4725fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f473e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54500/0 (socket says 172.21.15.5:54500) 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1112640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f4725fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:34836/0 (socket says 172.21.15.5:34836) 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 -- 172.21.15.5:0/68366708 learned_addr learned my addr 172.21.15.5:0/68366708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c0911640 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4719fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 -- 172.21.15.5:0/68366708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 msgr2=0x5557f4725fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f4725fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 -- 172.21.15.5:0/68366708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 msgr2=0x5557f4719fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4719fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.082+0000 7fb1c1913640 1 -- 172.21.15.5:0/68366708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f4714f60 con 0x5557f43564d0 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.083+0000 7fb1c1913640 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f473e180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400dfd0 tx=0x7fb1b40099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9485639eb08b8cda server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.083+0000 7fb1a6ffd640 1 -- 172.21.15.5:0/68366708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4020030 con 0x5557f43564d0 2024-05-14T15:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.083+0000 7fb1a6ffd640 1 -- 172.21.15.5:0/68366708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b40ad940 con 0x5557f43564d0 2024-05-14T15:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.083+0000 7fb1a6ffd640 1 -- 172.21.15.5:0/68366708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40e2290 con 0x5557f43564d0 2024-05-14T15:30:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 -- 172.21.15.5:0/68366708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 msgr2=0x5557f473e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f473e180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400dfd0 tx=0x7fb1b40099f0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 -- 172.21.15.5:0/68366708 shutdown_connections 2024-05-14T15:30:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f43564d0 0x5557f473e180 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f434e2f0 0x5557f4719fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/68366708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f4725fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 -- 172.21.15.5:0/68366708 >> 172.21.15.5:0/68366708 conn(0x5557f436b300 msgr2=0x5557f4719890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 -- 172.21.15.5:0/68366708 shutdown_connections 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 -- 172.21.15.5:0/68366708 wait complete. 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 -- start start 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.084+0000 7fb1ca22e740 1 Processor -- start 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f471e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f471aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f471b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1ad7d0 con 0x5557f434e2f0 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1aec40 con 0x5557f43564d0 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc1adea0 con 0x5557f4335d80 2024-05-14T15:30:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1ca22e740 10 client.0 _ll_register_callbacks cb 0x7fb1ac009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c0911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f471aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f471e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f471e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:34850/0 (socket says 172.21.15.5:34850) 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c0911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f471aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54510/0 (socket says 172.21.15.5:54510) 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 -- 172.21.15.5:0/1963966383 learned_addr learned my addr 172.21.15.5:0/1963966383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1913640 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f471b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 -- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 msgr2=0x5557f471b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f471b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 -- 172.21.15.5:0/1963966383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 msgr2=0x5557f471aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f471aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.085+0000 7fb1c1112640 1 -- 172.21.15.5:0/1963966383 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f471b770 con 0x5557f4335d80 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1c1112640 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f471e760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009850 tx=0x7fb1b8005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f103e212a7e303c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:30:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/1963966383 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8035040 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b40ad940 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/1963966383 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fb1b8005040 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/1963966383 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1b8060050 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1ca22e740 10 client.26344 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557f472ee40 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.086+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/1963966383 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b40ad940 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/1963966383 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb1b80046c0 con 0x5557f4335d80 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1ca22e740 10 client.26344 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1ca22e740 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1ca22e740 20 client.26344 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f43771b0 0x5557f47a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1ca22e740 4 client.26344 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557f4685050 con 0x5557f43771b0 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f43771b0 0x5557f47a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1c0911640 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f43771b0 0x5557f47a7530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4741b70739a8ccb6 server_cookie=95fa6eba810fd6cb in_seq=0 out_seq=0 2024-05-14T15:30:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.087+0000 7fb1a4ff9640 10 client.26344 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:40.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 518939 11.02 MB/sec execute 121 sec latency 40.644 ms 2024-05-14T15:30:40.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.344+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/2503453603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc811d930 msgr2=0x7fcbc811fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.344+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/2503453603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc811d930 0x7fcbc811fd50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000b0a0 tx=0x7fcbc000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.346+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/2503453603 shutdown_connections 2024-05-14T15:30:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.346+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/2503453603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc811d930 0x7fcbc811fd50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.346+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/2503453603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc810a930 0x7fcbc810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.346+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/2503453603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.346+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/2503453603 >> 172.21.15.5:0/2503453603 conn(0x7fcbc806f4d0 msgr2=0x7fcbc806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.348+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/2503453603 shutdown_connections 2024-05-14T15:30:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.348+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/2503453603 wait complete. 2024-05-14T15:30:40.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.350+0000 7fcbcd0ca700 1 Processor -- start 2024-05-14T15:30:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.351+0000 7fcbcd0ca700 1 -- start start 2024-05-14T15:30:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.351+0000 7fcbcd0ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc810a930 0x7fcbc81a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.351+0000 7fcbcd0ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc81aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbcd0ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc811d930 0x7fcbc81ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbcd0ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8123350 con 0x7fcbc810a930 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbcd0ca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81231d0 con 0x7fcbc811d930 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbcd0ca700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81234d0 con 0x7fcbc810b280 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc81aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc810a930 0x7fcbc81a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc81aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:34878/0 (socket says 172.21.15.5:34878) 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 -- 172.21.15.5:0/1539009826 learned_addr learned my addr 172.21.15.5:0/1539009826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc759e700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc811d930 0x7fcbc81ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 -- 172.21.15.5:0/1539009826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc811d930 msgr2=0x7fcbc81ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc811d930 0x7fcbc81ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 -- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc810a930 msgr2=0x7fcbc81a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc810a930 0x7fcbc81a9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.352+0000 7fcbc659c700 1 -- 172.21.15.5:0/1539009826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc81aec30 con 0x7fcbc810b280 2024-05-14T15:30:40.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.353+0000 7fcbc659c700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc81aa300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d950 tx=0x7fcbb800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.353+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8016070 con 0x7fcbc810b280 2024-05-14T15:30:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.353+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc81aef20 con 0x7fcbc810b280 2024-05-14T15:30:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.354+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81b6840 con 0x7fcbc810b280 2024-05-14T15:30:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.354+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb800fd40 con 0x7fcbc810b280 2024-05-14T15:30:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.354+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800e3b0 con 0x7fcbc810b280 2024-05-14T15:30:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.355+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc80512c0 con 0x7fcbc810b280 2024-05-14T15:30:40.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.358+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcbb8019070 con 0x7fcbc810b280 2024-05-14T15:30:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.359+0000 7fcbaffff700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcbb00b2af0 0x7fcbb00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.360+0000 7fcbc6d9d700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcbb00b2af0 0x7fcbb00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:40.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.360+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7fcbb80e2880 con 0x7fcbc810b280 2024-05-14T15:30:40.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.361+0000 7fcbc6d9d700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcbb00b2af0 0x7fcbb00b5000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc006fd0 tx=0x7fcbbc009190 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:40.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.362+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbb80a9e40 con 0x7fcbc810b280 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.372+0000 7fb1a4ff9640 1 -- 172.21.15.5:0/1963966383 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557f4685050 con 0x5557f43771b0 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.372+0000 7fb1a4ff9640 10 client.26344 handle_command_reply: tid=0 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.372+0000 7fb1ca22e740 1 client.26344 shutdown 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 msgr2=0x5557f471e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f471e760 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009850 tx=0x7fb1b8005db0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 shutdown_connections 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5557f43771b0 0x5557f47a7530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557f43564d0 0x5557f471b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1a4ff9640 0 client.26344 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557f434e2f0 0x5557f471aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 --2- 172.21.15.5:0/1963966383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557f4335d80 0x5557f471e760 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 >> 172.21.15.5:0/1963966383 conn(0x5557f436b300 msgr2=0x5557f4717970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 shutdown_connections 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 1 -- 172.21.15.5:0/1963966383 wait complete. 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.373+0000 7fb1ca22e740 20 client.26344 trim_cache size 0 max 16384 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:30:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1a80b29d0 msgr2=0x7fb1a80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1a80b29d0 0x7fb1a80b4ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00a010 tx=0x7fb1ac008040 comp rx=0 tx=0).stop 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 msgr2=0x7fb1bc19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 0x7fb1bc19a830 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009fd0 tx=0x7fb1b400b260 comp rx=0 tx=0).stop 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 shutdown_connections 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb1a80b29d0 0x7fb1a80b4ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc10a140 0x7fb1bc19a830 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1bc109ca0 0x7fb1bc1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 --2- 172.21.15.5:0/1093025226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1bc1043f0 0x7fb1bc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 >> 172.21.15.5:0/1093025226 conn(0x7fb1bc0ff340 msgr2=0x7fb1bc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 shutdown_connections 2024-05-14T15:30:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:40.374+0000 7fb1c3376640 1 -- 172.21.15.5:0/1093025226 wait complete. 2024-05-14T15:30:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.662+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbc81ab0c0 con 0x7fcbc810b280 2024-05-14T15:30:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.663+0000 7fcbaffff700 1 -- 172.21.15.5:0/1539009826 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11864 (secure 0 0 0) 0x7fcbb80a9740 con 0x7fcbc810b280 2024-05-14T15:30:40.665 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:30:40.666 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:27:33.754380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25691":{"gid":25691,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2401236249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2401236249},{"type":"v1","addr":"172.21.15.144:6835","nonce":2401236249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcbb00b2af0 msgr2=0x7fcbb00b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcbb00b2af0 0x7fcbb00b5000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc006fd0 tx=0x7fcbbc009190 comp rx=0 tx=0).stop 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 msgr2=0x7fcbc81aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc81aa300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d950 tx=0x7fcbb800dc60 comp rx=0 tx=0).stop 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 shutdown_connections 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcbb00b2af0 0x7fcbb00b5000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbc811d930 0x7fcbc81ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc810a930 0x7fcbc81a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 --2- 172.21.15.5:0/1539009826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc810b280 0x7fcbc81aa300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 >> 172.21.15.5:0/1539009826 conn(0x7fcbc806f4d0 msgr2=0x7fcbc811b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 shutdown_connections 2024-05-14T15:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:40.665+0000 7fcbcd0ca700 1 -- 172.21.15.5:0/1539009826 wait complete. 2024-05-14T15:30:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-05-14T15:30:40.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:40 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1539009826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:30:41.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:40 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1539009826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:30:41.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 520161 10.97 MB/sec execute 122 sec latency 233.345 ms 2024-05-14T15:30:41.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:40 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1539009826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:30:41.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:30:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:42.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:41 smithi193 ceph-mon[36001]: pgmap v180: 129 pgs: 129 active+clean; 57 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-05-14T15:30:42.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:41 smithi005 ceph-mon[31559]: pgmap v180: 129 pgs: 129 active+clean; 57 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-05-14T15:30:42.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 522459 10.99 MB/sec execute 123 sec latency 19.780 ms 2024-05-14T15:30:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:41 smithi144 ceph-mon[35650]: pgmap v180: 129 pgs: 129 active+clean; 57 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 2.3 MiB/s wr, 276 op/s 2024-05-14T15:30:43.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 524085 10.95 MB/sec execute 124 sec latency 32.711 ms 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 -- 172.21.15.5:0/2431297553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c1034c0 msgr2=0x7f9b5c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 --2- 172.21.15.5:0/2431297553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c1038d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0099c0 tx=0x7f9b4c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 -- 172.21.15.5:0/2431297553 shutdown_connections 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 --2- 172.21.15.5:0/2431297553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c108740 0x7f9b5c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 --2- 172.21.15.5:0/2431297553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c1038d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 --2- 172.21.15.5:0/2431297553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5c107df0 0x7f9b5c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 -- 172.21.15.5:0/2431297553 >> 172.21.15.5:0/2431297553 conn(0x7f9b5c0fedc0 msgr2=0x7f9b5c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 -- 172.21.15.5:0/2431297553 shutdown_connections 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.387+0000 7f9b61845700 1 -- 172.21.15.5:0/2431297553 wait complete. 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.388+0000 7f9b61845700 1 Processor -- start 2024-05-14T15:30:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.388+0000 7f9b61845700 1 -- start start 2024-05-14T15:30:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b61845700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b61845700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 0x7f9b5c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b61845700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5c108740 0x7f9b5c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b61845700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c116670 con 0x7f9b5c107df0 2024-05-14T15:30:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b61845700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1164f0 con 0x7f9b5c1034c0 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b61845700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1167f0 con 0x7f9b5c108740 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 0x7f9b5c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 0x7f9b5c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54530/0 (socket says 172.21.15.5:54530) 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:54242/0 (socket says 172.21.15.5:54242) 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3430226356 learned_addr learned my addr 172.21.15.5:0/3430226356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5b7fe700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5c108740 0x7f9b5c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3430226356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5c108740 msgr2=0x7f9b5c1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5c108740 0x7f9b5c1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3430226356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 msgr2=0x7f9b5c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3430226356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c1a1e90 con 0x7f9b5c107df0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.389+0000 7f9b5affd700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.390+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 0x7f9b5c19d560 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e970 tx=0x7f9b5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.390+0000 7f9b60843700 1 -- 172.21.15.5:0/3430226356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50004a30 con 0x7f9b5c107df0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.390+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c1a2180 con 0x7f9b5c107df0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.390+0000 7f9b60843700 1 -- 172.21.15.5:0/3430226356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b5000cc90 con 0x7f9b5c107df0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.390+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c1a9aa0 con 0x7f9b5c107df0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.390+0000 7f9b60843700 1 -- 172.21.15.5:0/3430226356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500055b0 con 0x7f9b5c107df0 2024-05-14T15:30:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.393+0000 7f9b60843700 1 -- 172.21.15.5:0/3430226356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9b50010430 con 0x7f9b5c107df0 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 Processor -- start 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 -- start start 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b60843700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9b440b29e0 0x7f9b440b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b5b7fe700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9b440b29e0 0x7f9b440b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b60843700 1 -- 172.21.15.5:0/3430226356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9b500e2cd0 con 0x7f9b5c107df0 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff724be40 0x558ff73cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 0x558ff73c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ff72eb940 con 0x558ff73c9e10 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558ff7250da0 con 0x558ff73cfd80 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b70259b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ff72e7900 con 0x558ff724be40 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 0x558ff73c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.394+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 0x558ff73c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:54256/0 (socket says 172.21.15.5:54256) 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3405423246 learned_addr learned my addr 172.21.15.5:0/3405423246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5affd700 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff724be40 0x558ff73cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3405423246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff724be40 msgr2=0x558ff73cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff724be40 0x558ff73cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3405423246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 msgr2=0x558ff73ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/3405423246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff72388d0 con 0x558ff73cfd80 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5b7fe700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9b440b29e0 0x7f9b440b4ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b54013de0 tx=0x7f9b54013460 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 0x558ff73c98d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b500a98f0 tx=0x7f9b50000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7a0779031d5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b41ffb700 1 -- 172.21.15.5:0/3405423246 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500a5380 con 0x558ff73cfd80 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b41ffb700 1 -- 172.21.15.5:0/3405423246 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b500b23a0 con 0x558ff73cfd80 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.395+0000 7f9b41ffb700 1 -- 172.21.15.5:0/3405423246 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500b2a50 con 0x558ff73cfd80 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 -- 172.21.15.5:0/3405423246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 msgr2=0x558ff73c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 0x558ff73c98d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b500a98f0 tx=0x7f9b50000bc0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 -- 172.21.15.5:0/3405423246 shutdown_connections 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff724be40 0x558ff73cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3405423246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73cfd80 0x558ff73c98d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 -- 172.21.15.5:0/3405423246 >> 172.21.15.5:0/3405423246 conn(0x558ff7293630 msgr2=0x558ff73c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 -- 172.21.15.5:0/3405423246 shutdown_connections 2024-05-14T15:30:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.396+0000 7f9b70259b80 1 -- 172.21.15.5:0/3405423246 wait complete. 2024-05-14T15:30:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 -- start start 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 Processor -- start 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff724be40 0x558ff73d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ff73d67c0 con 0x558ff73c9e10 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558ff72a4ed0 con 0x558ff724be40 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.397+0000 7f9b70259b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ff73d6940 con 0x558ff73cfd80 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b70259b80 10 client.0 _ll_register_callbacks cb 0x558ff7283950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73d3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54548/0 (socket says 172.21.15.5:54548) 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/1487405926 learned_addr learned my addr 172.21.15.5:0/1487405926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5affd700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff724be40 0x558ff73d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5b7fe700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 msgr2=0x558ff73d3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/1487405926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff724be40 msgr2=0x558ff73d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff724be40 0x558ff73d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 -- 172.21.15.5:0/1487405926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff72a2760 con 0x558ff73c9e10 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73d3a10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9b500e2ff0 tx=0x7f9b50004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63444bc42484e0b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b3affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b3affd700 1 -- 172.21.15.5:0/1487405926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500c9060 con 0x558ff73c9e10 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b3affd700 1 -- 172.21.15.5:0/1487405926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9b500a9e50 con 0x558ff73c9e10 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.398+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b500b23a0 con 0x558ff73c9e10 2024-05-14T15:30:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b3affd700 1 -- 172.21.15.5:0/1487405926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b500aa2f0 con 0x558ff73c9e10 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 10 client.16767 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558ff73e13e0 con 0x558ff73c9e10 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b3affd700 1 -- 172.21.15.5:0/1487405926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b500b23a0 con 0x558ff73c9e10 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b3affd700 1 -- 172.21.15.5:0/1487405926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9b500b25c0 con 0x558ff73c9e10 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 20 client.16767 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff73e38b0 0x558ff73e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 4 client.16767 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff72f9320 con 0x558ff73e38b0 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.399+0000 7f9b5affd700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff73e38b0 0x558ff73e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.400+0000 7f9b3affd700 10 client.16767 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.400+0000 7f9b5affd700 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff73e38b0 0x558ff73e5cd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7937e26f23084751 server_cookie=163af1a696b26615 in_seq=0 out_seq=0 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.400+0000 7f9b3affd700 1 -- 172.21.15.5:0/1487405926 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ff72f9320 con 0x558ff73e38b0 2024-05-14T15:30:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.400+0000 7f9b3affd700 10 client.16767 handle_command_reply: tid=0 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.400+0000 7f9b70259b80 1 client.16767 shutdown 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 msgr2=0x558ff73d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73d3a10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9b500e2ff0 tx=0x7f9b50004ff0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 shutdown_connections 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff73e38b0 0x558ff73e5cd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff724be40 0x558ff73d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1487405926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff73c9e10 0x558ff73d3a10 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 >> 172.21.15.5:0/1487405926 conn(0x558ff7293630 msgr2=0x558ff73c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b3affd700 0 client.16767 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 shutdown_connections 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 1 -- 172.21.15.5:0/1487405926 wait complete. 2024-05-14T15:30:43.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.401+0000 7f9b70259b80 20 client.16767 trim_cache size 0 max 16384 2024-05-14T15:30:43.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 Processor -- start 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 -- start start 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1b51e0 con 0x558ff724be40 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1b2a70 con 0x558ff73c9e10 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b70259b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1b08c0 con 0x558ff73cfd80 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54564/0 (socket says 172.21.15.5:54564) 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b5affd700 1 -- 172.21.15.5:0/3518379670 learned_addr learned my addr 172.21.15.5:0/3518379670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.418+0000 7f9b5b7fe700 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5affd700 1 -- 172.21.15.5:0/3518379670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 msgr2=0x558ff73d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5affd700 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5affd700 1 -- 172.21.15.5:0/3518379670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 msgr2=0x558ff73dfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5affd700 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73dfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5affd700 1 -- 172.21.15.5:0/3518379670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff73d6700 con 0x558ff724be40 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b5affd700 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73c98d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098e0 tx=0x7f9b4c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb4b4631a8d8aef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b417fa700 1 -- 172.21.15.5:0/3518379670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0050b0 con 0x558ff724be40 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b417fa700 1 -- 172.21.15.5:0/3518379670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4c02a070 con 0x558ff724be40 2024-05-14T15:30:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.419+0000 7f9b417fa700 1 -- 172.21.15.5:0/3518379670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c02a480 con 0x558ff724be40 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 -- 172.21.15.5:0/3518379670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 msgr2=0x558ff73c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73c98d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098e0 tx=0x7f9b4c01a040 comp rx=0 tx=0).stop 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 -- 172.21.15.5:0/3518379670 shutdown_connections 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff73d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73c98d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 --2- 172.21.15.5:0/3518379670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 -- 172.21.15.5:0/3518379670 >> 172.21.15.5:0/3518379670 conn(0x558ff7293630 msgr2=0x558ff73d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 -- 172.21.15.5:0/3518379670 shutdown_connections 2024-05-14T15:30:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 -- 172.21.15.5:0/3518379670 wait complete. 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.420+0000 7f9b70259b80 1 -- start start 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 Processor -- start 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff744bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1b0f30 con 0x558ff724be40 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1b23a0 con 0x558ff73c9e10 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1b1600 con 0x558ff73cfd80 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b70259b80 10 client.0 _ll_register_callbacks cb 0x558ff7283950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b5affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73e5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54574/0 (socket says 172.21.15.5:54574) 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b5affd700 1 -- 172.21.15.5:0/1906941259 learned_addr learned my addr 172.21.15.5:0/1906941259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.421+0000 7f9b5b7fe700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff744bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5affd700 1 -- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 msgr2=0x558ff744bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5affd700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff744bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5affd700 1 -- 172.21.15.5:0/1906941259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 msgr2=0x558ff73e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5affd700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73e60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5affd700 1 -- 172.21.15.5:0/1906941259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff744c330 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b5affd700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73e5b60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000c00 tx=0x7f9b4c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacf5fddfe030c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b3affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b3affd700 1 -- 172.21.15.5:0/1906941259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c003e60 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b3affd700 1 -- 172.21.15.5:0/1906941259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9b4c0050b0 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4c02a070 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b3affd700 1 -- 172.21.15.5:0/1906941259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b4c0052b0 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b70259b80 10 client.16773 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558ff744d460 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.422+0000 7f9b3affd700 1 -- 172.21.15.5:0/1906941259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4c02a070 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b3affd700 1 -- 172.21.15.5:0/1906941259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9b4c032020 con 0x558ff724be40 2024-05-14T15:30:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b70259b80 10 client.16773 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b70259b80 10 client.16773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b70259b80 20 client.16773 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff744da00 0x558ff744fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b70259b80 4 client.16773 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff72f33c0 con 0x558ff744da00 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff744da00 0x558ff744fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b3affd700 10 client.16773 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.423+0000 7f9b5a7fc700 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff744da00 0x558ff744fe20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2146a24f4379fd44 server_cookie=957c0a992f4a01c0 in_seq=0 out_seq=0 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b3affd700 1 -- 172.21.15.5:0/1906941259 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff72f33c0 con 0x558ff744da00 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b3affd700 10 client.16773 handle_command_reply: tid=0 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 client.16773 shutdown 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 msgr2=0x558ff73e5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73e5b60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c000c00 tx=0x7f9b4c00b8a0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 shutdown_connections 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ff73cfd80 0x558ff744bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ff724be40 0x558ff73e5b60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558ff744da00 0x558ff744fe20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 --2- 172.21.15.5:0/1906941259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558ff73c9e10 0x558ff73e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 >> 172.21.15.5:0/1906941259 conn(0x558ff7293630 msgr2=0x558ff73cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b3affd700 0 client.16773 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 shutdown_connections 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 1 -- 172.21.15.5:0/1906941259 wait complete. 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.424+0000 7f9b70259b80 20 client.16773 trim_cache size 0 max 16384 2024-05-14T15:30:43.431 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9b440b29e0 msgr2=0x7f9b440b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9b440b29e0 0x7f9b440b4ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b54013de0 tx=0x7f9b54013460 comp rx=0 tx=0).stop 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 msgr2=0x7f9b5c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 0x7f9b5c19d560 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000e970 tx=0x7f9b5000ec80 comp rx=0 tx=0).stop 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 shutdown_connections 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9b440b29e0 0x7f9b440b4ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5c108740 0x7f9b5c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c1034c0 0x7f9b5c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 --2- 172.21.15.5:0/3430226356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c107df0 0x7f9b5c19d560 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 >> 172.21.15.5:0/3430226356 conn(0x7f9b5c0fedc0 msgr2=0x7f9b5c101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 shutdown_connections 2024-05-14T15:30:43.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:43.425+0000 7f9b61845700 1 -- 172.21.15.5:0/3430226356 wait complete. 2024-05-14T15:30:44.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:43 smithi144 ceph-mon[35650]: pgmap v181: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-05-14T15:30:44.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:43 smithi193 ceph-mon[36001]: pgmap v181: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-05-14T15:30:44.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:43 smithi005 ceph-mon[31559]: pgmap v181: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-05-14T15:30:44.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 526227 10.97 MB/sec execute 125 sec latency 26.761 ms 2024-05-14T15:30:44.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:30:44.524 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:45.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 528218 10.94 MB/sec execute 126 sec latency 23.227 ms 2024-05-14T15:30:46.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:45 smithi005 ceph-mon[31559]: pgmap v182: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2024-05-14T15:30:46.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 530096 10.95 MB/sec execute 127 sec latency 25.992 ms 2024-05-14T15:30:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:45 smithi144 ceph-mon[35650]: pgmap v182: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2024-05-14T15:30:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:45 smithi193 ceph-mon[36001]: pgmap v182: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 346 op/s 2024-05-14T15:30:46.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 -- 172.21.15.5:0/538682986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102950 msgr2=0x7f15a4102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 --2- 172.21.15.5:0/538682986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102950 0x7f15a4102d60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1594009990 tx=0x7f1594009ca0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 -- 172.21.15.5:0/538682986 shutdown_connections 2024-05-14T15:30:46.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 --2- 172.21.15.5:0/538682986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a41096c0 0x7f15a4111000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 --2- 172.21.15.5:0/538682986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102950 0x7f15a4102d60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 --2- 172.21.15.5:0/538682986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a4108d70 0x7f15a4109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 -- 172.21.15.5:0/538682986 >> 172.21.15.5:0/538682986 conn(0x7f15a40783b0 msgr2=0x7f15a40787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 -- 172.21.15.5:0/538682986 shutdown_connections 2024-05-14T15:30:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.331+0000 7f15a939e700 1 -- 172.21.15.5:0/538682986 wait complete. 2024-05-14T15:30:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.332+0000 7f15a939e700 1 Processor -- start 2024-05-14T15:30:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.332+0000 7f15a939e700 1 -- start start 2024-05-14T15:30:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a939e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102950 0x7f15a41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a939e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 0x7f15a41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a939e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a41096c0 0x7f15a419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a939e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41142b0 con 0x7f15a4108d70 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a939e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4114130 con 0x7f15a4102950 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a939e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4114430 con 0x7f15a41096c0 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 0x7f15a41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 0x7f15a41a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43930/0 (socket says 172.21.15.5:43930) 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102950 0x7f15a41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 -- 172.21.15.5:0/3621305831 learned_addr learned my addr 172.21.15.5:0/3621305831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a41096c0 0x7f15a419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 -- 172.21.15.5:0/3621305831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a41096c0 msgr2=0x7f15a419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a41096c0 0x7f15a419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 -- 172.21.15.5:0/3621305831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102950 msgr2=0x7f15a41a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102950 0x7f15a41a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a37fe700 1 -- 172.21.15.5:0/3621305831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a419b160 con 0x7f15a4108d70 2024-05-14T15:30:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a41096c0 0x7f15a419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.333+0000 7f15a3fff700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102950 0x7f15a41a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.334+0000 7f15a37fe700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 0x7f15a41a0dd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f159800e940 tx=0x7f159800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.334+0000 7f15a17fa700 1 -- 172.21.15.5:0/3621305831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159800cbd0 con 0x7f15a4108d70 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.334+0000 7f15a17fa700 1 -- 172.21.15.5:0/3621305831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159800ee90 con 0x7f15a4108d70 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.334+0000 7f15a17fa700 1 -- 172.21.15.5:0/3621305831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598018680 con 0x7f15a4108d70 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.334+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a419b450 con 0x7f15a4108d70 2024-05-14T15:30:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.334+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41a7720 con 0x7f15a4108d70 2024-05-14T15:30:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.338+0000 7f15a17fa700 1 -- 172.21.15.5:0/3621305831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1598010430 con 0x7f15a4108d70 2024-05-14T15:30:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.339+0000 7f15a17fa700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f158c0b29f0 0x7f158c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.339+0000 7f15a3fff700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f158c0b29f0 0x7f158c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.339+0000 7f15a17fa700 1 -- 172.21.15.5:0/3621305831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f15980e2c40 con 0x7f15a4108d70 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.339+0000 7f15b9014b80 1 Processor -- start 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.339+0000 7f15b9014b80 1 -- start start 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 0x55f77dd65bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd660f0 0x55f77dd66500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a3fff700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f158c0b29f0 0x7f158c0b4f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f1594005960 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15b9014b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f77dc87940 con 0x55f77dbe7e40 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15b9014b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f77dbecda0 con 0x55f77dd660f0 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15b9014b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f77dc83900 con 0x55f77dd73e90 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 0x55f77dd65bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd660f0 0x55f77dd66500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 0x55f77dd65bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43936/0 (socket says 172.21.15.5:43936) 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a3fff700 1 -- 172.21.15.5:0/2156530166 learned_addr learned my addr 172.21.15.5:0/2156530166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a37fe700 1 -- 172.21.15.5:0/2156530166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 msgr2=0x55f77dd762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a37fe700 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a37fe700 1 -- 172.21.15.5:0/2156530166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 msgr2=0x55f77dd65bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a37fe700 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 0x55f77dd65bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a37fe700 1 -- 172.21.15.5:0/2156530166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77dbd48d0 con 0x55f77dd660f0 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.340+0000 7f15a3fff700 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 0x55f77dd65bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.341+0000 7f15a37fe700 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd660f0 0x55f77dd66500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f15980dffd0 tx=0x7f15980a9580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a194a8e4c768727b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.341+0000 7f158a7fc700 1 -- 172.21.15.5:0/2156530166 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980a9670 con 0x55f77dd660f0 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.341+0000 7f158a7fc700 1 -- 172.21.15.5:0/2156530166 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15980aa260 con 0x55f77dd660f0 2024-05-14T15:30:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.341+0000 7f158a7fc700 1 -- 172.21.15.5:0/2156530166 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980aa020 con 0x55f77dd660f0 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 -- 172.21.15.5:0/2156530166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd660f0 msgr2=0x55f77dd66500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd660f0 0x55f77dd66500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f15980dffd0 tx=0x7f15980a9580 comp rx=0 tx=0).stop 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 -- 172.21.15.5:0/2156530166 shutdown_connections 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dbe7e40 0x55f77dd65bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2156530166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd660f0 0x55f77dd66500 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 -- 172.21.15.5:0/2156530166 >> 172.21.15.5:0/2156530166 conn(0x55f77dc2f630 msgr2=0x55f77dd60cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 -- 172.21.15.5:0/2156530166 shutdown_connections 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 -- 172.21.15.5:0/2156530166 wait complete. 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.342+0000 7f15b9014b80 1 -- start start 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 Processor -- start 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd70fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd660f0 0x55f77dd714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd73e90 0x55f77dd71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f77dd76cf0 con 0x55f77dd73e90 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f77dc40ed0 con 0x55f77dbe7e40 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f77dd76e70 con 0x55f77dd660f0 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15b9014b80 10 client.0 _ll_register_callbacks cb 0x55f77dc1f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd70fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd660f0 0x55f77dd714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd70fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:60954/0 (socket says 172.21.15.5:60954) 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15a3fff700 1 -- 172.21.15.5:0/4022877280 learned_addr learned my addr 172.21.15.5:0/4022877280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.343+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd73e90 0x55f77dd71a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15a3fff700 1 -- 172.21.15.5:0/4022877280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd660f0 msgr2=0x55f77dd714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15a3fff700 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd660f0 0x55f77dd714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15a3fff700 1 -- 172.21.15.5:0/4022877280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd73e90 msgr2=0x55f77dd71a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15a3fff700 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd73e90 0x55f77dd71a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15a3fff700 1 -- 172.21.15.5:0/4022877280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77dc3e760 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15a3fff700 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd70fa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15940059d0 tx=0x7f1594009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00529330596ca14 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15837fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15837fe700 1 -- 172.21.15.5:0/4022877280 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594035060 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15837fe700 1 -- 172.21.15.5:0/4022877280 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f15940259b0 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.344+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15980aa260 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15837fe700 1 -- 172.21.15.5:0/4022877280 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1594025bb0 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 10 client.26402 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f77dd81a70 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15837fe700 1 -- 172.21.15.5:0/4022877280 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15980aa260 con 0x55f77dbe7e40 2024-05-14T15:30:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15837fe700 1 -- 172.21.15.5:0/4022877280 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1594022070 con 0x55f77dbe7e40 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 10 client.26402 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 10 client.26402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 20 client.26402 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77dd840b0 0x55f77dd864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 4 client.26402 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f77dc95320 con 0x55f77dd840b0 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.345+0000 7f15a37fe700 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77dd840b0 0x55f77dd864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.346+0000 7f15837fe700 10 client.26402 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.346+0000 7f15a37fe700 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77dd840b0 0x55f77dd864d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c1b453c34516e7a server_cookie=89e7f93b1199950e in_seq=0 out_seq=0 2024-05-14T15:30:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15837fe700 1 -- 172.21.15.5:0/4022877280 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f77dc95320 con 0x55f77dd840b0 2024-05-14T15:30:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15837fe700 10 client.26402 handle_command_reply: tid=0 2024-05-14T15:30:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 client.26402 shutdown 2024-05-14T15:30:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 msgr2=0x55f77dd70fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd70fa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15940059d0 tx=0x7f1594009ca0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 shutdown_connections 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77dd840b0 0x55f77dd864d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd73e90 0x55f77dd71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd70fa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 --2- 172.21.15.5:0/4022877280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd660f0 0x55f77dd714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 >> 172.21.15.5:0/4022877280 conn(0x55f77dc2f630 msgr2=0x55f77dd628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15837fe700 0 client.26402 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 shutdown_connections 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 1 -- 172.21.15.5:0/4022877280 wait complete. 2024-05-14T15:30:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.347+0000 7f15b9014b80 20 client.26402 trim_cache size 0 max 16384 2024-05-14T15:30:46.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 Processor -- start 2024-05-14T15:30:46.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 -- start start 2024-05-14T15:30:46.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 0x55f77dd8bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd73e90 0x55f77dd6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b0be0 con 0x55f77dd660f0 2024-05-14T15:30:46.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ae470 con 0x55f77dd73e90 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.380+0000 7f15b9014b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ac2c0 con 0x55f77dbe7e40 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 0x55f77dd8bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 0x55f77dd8bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33834/0 (socket says 172.21.15.5:33834) 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 -- 172.21.15.5:0/2659046535 learned_addr learned my addr 172.21.15.5:0/2659046535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd73e90 0x55f77dd6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 -- 172.21.15.5:0/2659046535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd73e90 msgr2=0x55f77dd6e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd73e90 0x55f77dd6e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 -- 172.21.15.5:0/2659046535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 msgr2=0x55f77dd8c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd8c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 -- 172.21.15.5:0/2659046535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77dd615e0 con 0x55f77dbe7e40 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f15a3fff700 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 0x55f77dd8bc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f15940095e0 tx=0x7f159400f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123ca2ff4cf030b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.381+0000 7f1589ffb700 1 -- 172.21.15.5:0/2659046535 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594035040 con 0x55f77dbe7e40 2024-05-14T15:30:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.382+0000 7f1589ffb700 1 -- 172.21.15.5:0/2659046535 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400fba0 con 0x55f77dbe7e40 2024-05-14T15:30:46.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.382+0000 7f1589ffb700 1 -- 172.21.15.5:0/2659046535 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594040430 con 0x55f77dbe7e40 2024-05-14T15:30:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 -- 172.21.15.5:0/2659046535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 msgr2=0x55f77dd8bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 0x55f77dd8bc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f15940095e0 tx=0x7f159400f8b0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 -- 172.21.15.5:0/2659046535 shutdown_connections 2024-05-14T15:30:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dd73e90 0x55f77dd6e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dbe7e40 0x55f77dd8bc40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 --2- 172.21.15.5:0/2659046535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd8c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 -- 172.21.15.5:0/2659046535 >> 172.21.15.5:0/2659046535 conn(0x55f77dc2f630 msgr2=0x55f77dd66500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 -- 172.21.15.5:0/2659046535 shutdown_connections 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 -- 172.21.15.5:0/2659046535 wait complete. 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.383+0000 7f15b9014b80 1 -- start start 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 Processor -- start 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd7bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ac930 con 0x55f77dd660f0 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41adda0 con 0x55f77dbe7e40 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ad000 con 0x55f77dd73e90 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15b9014b80 10 client.0 _ll_register_callbacks cb 0x55f77dc1f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd7bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd7bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:60984/0 (socket says 172.21.15.5:60984) 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15a3fff700 1 -- 172.21.15.5:0/1152804579 learned_addr learned my addr 172.21.15.5:0/1152804579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.384+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a3fff700 1 -- 172.21.15.5:0/1152804579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 msgr2=0x55f77dd774c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a3fff700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd774c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a3fff700 1 -- 172.21.15.5:0/1152804579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 msgr2=0x55f77dd7c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a3fff700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd7c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a3fff700 1 -- 172.21.15.5:0/1152804579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77dd77bb0 con 0x55f77dbe7e40 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a8b9d700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15a3fff700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd7bfb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1594009990 tx=0x7f1594025420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=782b95a80a2ffe12 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15837fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15837fe700 1 -- 172.21.15.5:0/1152804579 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594035040 con 0x55f77dbe7e40 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159400fba0 con 0x55f77dbe7e40 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15837fe700 1 -- 172.21.15.5:0/1152804579 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f1594030680 con 0x55f77dbe7e40 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15837fe700 1 -- 172.21.15.5:0/1152804579 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1594030880 con 0x55f77dbe7e40 2024-05-14T15:30:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15b9014b80 10 client.26408 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.385+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f77dd77bb0 con 0x55f77dbe7e40 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15837fe700 1 -- 172.21.15.5:0/1152804579 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400fba0 con 0x55f77dbe7e40 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15837fe700 1 -- 172.21.15.5:0/1152804579 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1594022040 con 0x55f77dbe7e40 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15b9014b80 10 client.26408 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15b9014b80 10 client.26408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15b9014b80 20 client.26408 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15b9014b80 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77ddee410 0x55f77ddf0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15b9014b80 4 client.26408 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f77dc8f3c0 con 0x55f77ddee410 2024-05-14T15:30:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15a37fe700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77ddee410 0x55f77ddf0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.386+0000 7f15837fe700 10 client.26408 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15a37fe700 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77ddee410 0x55f77ddf0830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec51f3f9bf8cf729 server_cookie=b3678c2458a7f544 in_seq=0 out_seq=0 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15837fe700 1 -- 172.21.15.5:0/1152804579 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f77dc8f3c0 con 0x55f77ddee410 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15837fe700 10 client.26408 handle_command_reply: tid=0 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 client.26408 shutdown 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 msgr2=0x55f77dd7bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd7bfb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1594009990 tx=0x7f1594025420 comp rx=0 tx=0).stop 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 shutdown_connections 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55f77ddee410 0x55f77ddf0830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f77dd73e90 0x55f77dd774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f77dbe7e40 0x55f77dd7bfb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 --2- 172.21.15.5:0/1152804579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f77dd660f0 0x55f77dd7c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 >> 172.21.15.5:0/1152804579 conn(0x55f77dc2f630 msgr2=0x55f77dd66500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:46.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15837fe700 0 client.26408 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:30:46.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 shutdown_connections 2024-05-14T15:30:46.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 1 -- 172.21.15.5:0/1152804579 wait complete. 2024-05-14T15:30:46.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.387+0000 7f15b9014b80 20 client.26408 trim_cache size 0 max 16384 2024-05-14T15:30:46.393 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:30:46.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f158c0b29f0 msgr2=0x7f158c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f158c0b29f0 0x7f158c0b4f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f1594005960 comp rx=0 tx=0).stop 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 msgr2=0x7f15a41a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 0x7f15a41a0dd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f159800e940 tx=0x7f159800ec50 comp rx=0 tx=0).stop 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 shutdown_connections 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f158c0b29f0 0x7f158c0b4f00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a41096c0 0x7f15a419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102950 0x7f15a41a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 --2- 172.21.15.5:0/3621305831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4108d70 0x7f15a41a0dd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 >> 172.21.15.5:0/3621305831 conn(0x7f15a40783b0 msgr2=0x7f15a4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 shutdown_connections 2024-05-14T15:30:46.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:46.388+0000 7f15a939e700 1 -- 172.21.15.5:0/3621305831 wait complete. 2024-05-14T15:30:47.141 INFO:tasks.workunit.client.0.smithi005.stdout: 1 532337 10.96 MB/sec execute 128 sec latency 52.874 ms 2024-05-14T15:30:47.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:30:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:48.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:47 smithi005 ceph-mon[31559]: pgmap v183: 129 pgs: 129 active+clean; 67 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-14T15:30:48.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 533762 10.93 MB/sec execute 129 sec latency 94.859 ms 2024-05-14T15:30:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:47 smithi144 ceph-mon[35650]: pgmap v183: 129 pgs: 129 active+clean; 67 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-14T15:30:48.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:47 smithi193 ceph-mon[36001]: pgmap v183: 129 pgs: 129 active+clean; 67 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 366 op/s 2024-05-14T15:30:49.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 536244 10.96 MB/sec execute 130 sec latency 15.232 ms 2024-05-14T15:30:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 -- 172.21.15.5:0/1847231380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4107e30 msgr2=0x7f1ca4108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 --2- 172.21.15.5:0/1847231380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4107e30 0x7f1ca4108240 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0099c0 tx=0x7f1c8c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 -- 172.21.15.5:0/1847231380 shutdown_connections 2024-05-14T15:30:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 --2- 172.21.15.5:0/1847231380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ca4108780 0x7f1ca41100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 --2- 172.21.15.5:0/1847231380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca4103550 0x7f1ca4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 --2- 172.21.15.5:0/1847231380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4107e30 0x7f1ca4108240 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 -- 172.21.15.5:0/1847231380 >> 172.21.15.5:0/1847231380 conn(0x7f1ca4078e00 msgr2=0x7f1ca4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 -- 172.21.15.5:0/1847231380 shutdown_connections 2024-05-14T15:30:49.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.563+0000 7f1caa575700 1 -- 172.21.15.5:0/1847231380 wait complete. 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 Processor -- start 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 -- start start 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 0x7f1ca419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ca4107e30 0x7f1ca419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca4108780 0x7f1ca419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4113390 con 0x7f1ca4103550 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4113210 con 0x7f1ca4108780 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.564+0000 7f1caa575700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4113510 con 0x7f1ca4107e30 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 0x7f1ca419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 0x7f1ca419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43956/0 (socket says 172.21.15.5:43956) 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 -- 172.21.15.5:0/1689768763 learned_addr learned my addr 172.21.15.5:0/1689768763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca8b12700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca4108780 0x7f1ca419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ca4107e30 0x7f1ca419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 -- 172.21.15.5:0/1689768763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ca4107e30 msgr2=0x7f1ca419c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ca4107e30 0x7f1ca419c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 -- 172.21.15.5:0/1689768763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca4108780 msgr2=0x7f1ca419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca4108780 0x7f1ca419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 -- 172.21.15.5:0/1689768763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca41a6fa0 con 0x7f1ca4103550 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 0x7f1ca419bc50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e9a0 tx=0x7f1c9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1ca1ffb700 1 -- 172.21.15.5:0/1689768763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400ca10 con 0x7f1ca4103550 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca41a7290 con 0x7f1ca4103550 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.565+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca41a77f0 con 0x7f1ca4103550 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.566+0000 7f1ca1ffb700 1 -- 172.21.15.5:0/1689768763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c9400cbb0 con 0x7f1ca4103550 2024-05-14T15:30:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.566+0000 7f1ca1ffb700 1 -- 172.21.15.5:0/1689768763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94026060 con 0x7f1ca4103550 2024-05-14T15:30:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.569+0000 7f1ca1ffb700 1 -- 172.21.15.5:0/1689768763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1c94018420 con 0x7f1ca4103550 2024-05-14T15:30:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.570+0000 7f1ca1ffb700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1c900b2a70 0x7f1c900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.570+0000 7f1cb8f89b80 1 Processor -- start 2024-05-14T15:30:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.570+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1c900b2a70 0x7f1c900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.570+0000 7f1ca1ffb700 1 -- 172.21.15.5:0/1689768763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f1c940e2b70 con 0x7f1ca4103550 2024-05-14T15:30:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.570+0000 7f1cb8f89b80 1 -- start start 2024-05-14T15:30:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 0x5574a307a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a307ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307f470 0x5574a307f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574a2f9c860 con 0x5574a307ac60 2024-05-14T15:30:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574a2f3b7c0 con 0x5574a2efce40 2024-05-14T15:30:49.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574a2f98820 con 0x5574a307f470 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1c900b2a70 0x7f1c900b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c000c00 tx=0x7f1c8c01a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 0x5574a307a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 0x5574a307a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:32778/0 (socket says 172.21.15.5:32778) 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 -- 172.21.15.5:0/4251751500 learned_addr learned my addr 172.21.15.5:0/4251751500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a307ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca8b12700 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307f470 0x5574a307f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 -- 172.21.15.5:0/4251751500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307f470 msgr2=0x5574a307f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307f470 0x5574a307f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 -- 172.21.15.5:0/4251751500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 msgr2=0x5574a307ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a307ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 -- 172.21.15.5:0/4251751500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a2ee98d0 con 0x5574a2efce40 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.571+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 0x5574a307a720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c940a5b10 tx=0x7f1c940aa2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a0b8d49f0a5185a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.572+0000 7f1c9a7fc700 1 -- 172.21.15.5:0/4251751500 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940b28f0 con 0x5574a2efce40 2024-05-14T15:30:49.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.572+0000 7f1c9a7fc700 1 -- 172.21.15.5:0/4251751500 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c940a9cf0 con 0x5574a2efce40 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.572+0000 7f1c9a7fc700 1 -- 172.21.15.5:0/4251751500 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940a8f50 con 0x5574a2efce40 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.572+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/4251751500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 msgr2=0x5574a307a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.572+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 0x5574a307a720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c940a5b10 tx=0x7f1c940aa2e0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/4251751500 shutdown_connections 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307f470 0x5574a307f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a2efce40 0x5574a307a720 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/4251751500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a307ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/4251751500 >> 172.21.15.5:0/4251751500 conn(0x5574a2f44600 msgr2=0x5574a30754b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/4251751500 shutdown_connections 2024-05-14T15:30:49.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/4251751500 wait complete. 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 -- start start 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.573+0000 7f1cb8f89b80 1 Processor -- start 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 0x5574a30841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307ac60 0x5574a3084730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a3084c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 0x5574a30841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 0x5574a30841f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43978/0 (socket says 172.21.15.5:43978) 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 -- 172.21.15.5:0/2160096338 learned_addr learned my addr 172.21.15.5:0/2160096338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574a3087420 con 0x5574a2efce40 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574a2f02c30 con 0x5574a307f470 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574a30875a0 con 0x5574a307ac60 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca8b12700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a3084c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307ac60 0x5574a3084730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1cb8f89b80 10 client.0 _ll_register_callbacks cb 0x5574a2f34920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 -- 172.21.15.5:0/2160096338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307ac60 msgr2=0x5574a3084730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307ac60 0x5574a3084730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 -- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 msgr2=0x5574a3084c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a3084c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a2f38690 con 0x5574a2efce40 2024-05-14T15:30:49.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca8b12700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a3084c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 0x5574a30841f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1c940e2e90 tx=0x7f1c940cc300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d874b4d64dd47f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.574+0000 7f1c837fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1c837fe700 1 -- 172.21.15.5:0/2160096338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940b28f0 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1c837fe700 1 -- 172.21.15.5:0/2160096338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f1c940a8f00 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c940a9cf0 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1c837fe700 1 -- 172.21.15.5:0/2160096338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1c940a5400 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 10 client.16803 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574a3092070 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1c837fe700 1 -- 172.21.15.5:0/2160096338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c940a9cf0 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1c837fe700 1 -- 172.21.15.5:0/2160096338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1c940aefd0 con 0x5574a2efce40 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:49.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 20 client.16803 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.575+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3096960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.576+0000 7f1cb8f89b80 4 client.16803 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.576+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a2faa240 con 0x5574a3094540 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.576+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3096960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.576+0000 7f1c837fe700 10 client.16803 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.576+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3096960 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c33e8383ed61a745 server_cookie=d97f14a1664e0ac5 in_seq=0 out_seq=0 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1c837fe700 1 -- 172.21.15.5:0/2160096338 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574a2faa240 con 0x5574a3094540 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1c837fe700 10 client.16803 handle_command_reply: tid=0 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 client.16803 shutdown 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 msgr2=0x5574a30841f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 0x5574a30841f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1c940e2e90 tx=0x7f1c940cc300 comp rx=0 tx=0).stop 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 shutdown_connections 2024-05-14T15:30:49.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3096960 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1c837fe700 0 client.16803 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a3084c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a2efce40 0x5574a30841f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2160096338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a307ac60 0x5574a3084730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 >> 172.21.15.5:0/2160096338 conn(0x5574a2f44600 msgr2=0x5574a307f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.577+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 shutdown_connections 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.578+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2160096338 wait complete. 2024-05-14T15:30:49.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.578+0000 7f1cb8f89b80 20 client.16803 trim_cache size 0 max 16384 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 Processor -- start 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 -- start start 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3080890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 0x5574a3090870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307f470 0x5574a3084ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84011660 con 0x5574a307f470 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8400eef0 con 0x5574a307ac60 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.593+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8400cd40 con 0x5574a2efce40 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1ca8b12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307f470 0x5574a3084ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 0x5574a3090870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 0x5574a3090870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:32790/0 (socket says 172.21.15.5:32790) 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 -- 172.21.15.5:0/338366420 learned_addr learned my addr 172.21.15.5:0/338366420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3080890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33886/0 (socket says 172.21.15.5:33886) 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 -- 172.21.15.5:0/338366420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 msgr2=0x5574a3080890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3080890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 -- 172.21.15.5:0/338366420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307f470 msgr2=0x5574a3084ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307f470 0x5574a3084ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 -- 172.21.15.5:0/338366420 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a307f280 con 0x5574a307ac60 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 0x5574a3090870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0092e0 tx=0x7f1c8c009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cc2108a3ecb124 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c99ffb700 1 -- 172.21.15.5:0/338366420 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00b990 con 0x5574a307ac60 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c99ffb700 1 -- 172.21.15.5:0/338366420 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c8c0333b0 con 0x5574a307ac60 2024-05-14T15:30:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.594+0000 7f1c99ffb700 1 -- 172.21.15.5:0/338366420 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00f6c0 con 0x5574a307ac60 2024-05-14T15:30:49.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/338366420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 msgr2=0x5574a3090870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 0x5574a3090870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0092e0 tx=0x7f1c8c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/338366420 shutdown_connections 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307f470 0x5574a3084ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3080890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/338366420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307ac60 0x5574a3090870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/338366420 >> 172.21.15.5:0/338366420 conn(0x5574a2f44600 msgr2=0x5574a307eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/338366420 shutdown_connections 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.595+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/338366420 wait complete. 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 -- start start 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 Processor -- start 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3096960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a3096ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a30973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8400d3b0 con 0x5574a307ac60 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8400e820 con 0x5574a307f470 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.596+0000 7f1cb8f89b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8400da80 con 0x5574a2efce40 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1cb8f89b80 10 client.0 _ll_register_callbacks cb 0x5574a2f34920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a3096ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1ca3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3096960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a3096ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44004/0 (socket says 172.21.15.5:44004) 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 -- 172.21.15.5:0/2203432741 learned_addr learned my addr 172.21.15.5:0/2203432741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1ca8b12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a30973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 -- 172.21.15.5:0/2203432741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 msgr2=0x5574a3096960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3096960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 -- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 msgr2=0x5574a30973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a30973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 -- 172.21.15.5:0/2203432741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a30fcbb0 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c9bfff700 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a3096ea0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009350 tx=0x7f1c8c0053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0be43c853dbf1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c837fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c837fe700 1 -- 172.21.15.5:0/2203432741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c033830 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1c837fe700 1 -- 172.21.15.5:0/2203432741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f1c8c0339d0 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.597+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c8c0333b0 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1c837fe700 1 -- 172.21.15.5:0/2203432741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1c8c049050 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 10 client.16809 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574a30fde30 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1c837fe700 1 -- 172.21.15.5:0/2203432741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c8c0333b0 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1c837fe700 1 -- 172.21.15.5:0/2203432741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1c8c021070 con 0x5574a307ac60 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 10 client.16809 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 20 client.16809 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3100430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 4 client.16809 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574a2fa42e0 con 0x5574a3094540 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.598+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1c837fe700 10 client.16809 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1ca3fff700 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3100430 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febb1bc4614f6bba server_cookie=55f54800dcc0b20f in_seq=0 out_seq=0 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1c837fe700 1 -- 172.21.15.5:0/2203432741 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574a2fa42e0 con 0x5574a3094540 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1c837fe700 10 client.16809 handle_command_reply: tid=0 2024-05-14T15:30:49.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 client.16809 shutdown 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 msgr2=0x5574a3096ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a3096ea0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009350 tx=0x7f1c8c0053d0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 shutdown_connections 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a3094540 0x5574a3100430 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a307f470 0x5574a30973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a2efce40 0x5574a3096960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 --2- 172.21.15.5:0/2203432741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a307ac60 0x5574a3096ea0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.599+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 >> 172.21.15.5:0/2203432741 conn(0x5574a2f44600 msgr2=0x5574a307eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1c837fe700 0 client.16809 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 shutdown_connections 2024-05-14T15:30:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1cb8f89b80 1 -- 172.21.15.5:0/2203432741 wait complete. 2024-05-14T15:30:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1cb8f89b80 20 client.16809 trim_cache size 0 max 16384 2024-05-14T15:30:49.606 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1c900b2a70 msgr2=0x7f1c900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1c900b2a70 0x7f1c900b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c000c00 tx=0x7f1c8c01a040 comp rx=0 tx=0).stop 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 msgr2=0x7f1ca419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 0x7f1ca419bc50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e9a0 tx=0x7f1c9400ecb0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 shutdown_connections 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1c900b2a70 0x7f1c900b4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca4108780 0x7f1ca419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca4103550 0x7f1ca419bc50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 --2- 172.21.15.5:0/1689768763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ca4107e30 0x7f1ca419c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:49.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.600+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 >> 172.21.15.5:0/1689768763 conn(0x7f1ca4078e00 msgr2=0x7f1ca4100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:49.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.601+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 shutdown_connections 2024-05-14T15:30:49.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:49.601+0000 7f1caa575700 1 -- 172.21.15.5:0/1689768763 wait complete. 2024-05-14T15:30:49.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:49 smithi005 ceph-mon[31559]: pgmap v184: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 362 op/s 2024-05-14T15:30:50.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 537681 10.93 MB/sec execute 131 sec latency 20.320 ms 2024-05-14T15:30:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:49 smithi144 ceph-mon[35650]: pgmap v184: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 362 op/s 2024-05-14T15:30:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:49 smithi193 ceph-mon[36001]: pgmap v184: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 362 op/s 2024-05-14T15:30:51.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 540113 10.96 MB/sec execute 132 sec latency 23.125 ms 2024-05-14T15:30:51.407 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:30:51.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:30:51.671 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:52.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:51 smithi005 ceph-mon[31559]: pgmap v185: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 335 op/s 2024-05-14T15:30:52.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 541617 10.93 MB/sec execute 133 sec latency 36.298 ms 2024-05-14T15:30:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:51 smithi144 ceph-mon[35650]: pgmap v185: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 335 op/s 2024-05-14T15:30:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:51 smithi193 ceph-mon[36001]: pgmap v185: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.9 MiB/s wr, 335 op/s 2024-05-14T15:30:53.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 543655 10.94 MB/sec execute 134 sec latency 121.995 ms 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1418039584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8108ba0 msgr2=0x7f9ce8102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1418039584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8108ba0 0x7f9ce8102740 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009900 tx=0x7f9cd802a410 comp rx=0 tx=0).stop 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1418039584 shutdown_connections 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1418039584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ce8109040 0x7f9ce8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1418039584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce81032f0 0x7f9ce810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1418039584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8108ba0 0x7f9ce8102740 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1418039584 >> 172.21.15.5:0/1418039584 conn(0x7f9ce80782e0 msgr2=0x7f9ce8078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1418039584 shutdown_connections 2024-05-14T15:30:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.485+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1418039584 wait complete. 2024-05-14T15:30:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 Processor -- start 2024-05-14T15:30:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 -- start start 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ce81032f0 0x7f9ce81a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8108ba0 0x7f9ce81a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 0x7f9ce819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8112d90 con 0x7f9ce8109040 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8112c10 con 0x7f9ce8108ba0 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9cecf5e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8112f10 con 0x7f9ce81032f0 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9ce6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 0x7f9ce819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9ce6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 0x7f9ce819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44016/0 (socket says 172.21.15.5:44016) 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9ce6d9d640 1 -- 172.21.15.5:0/1045427089 learned_addr learned my addr 172.21.15.5:0/1045427089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.486+0000 7f9ce659c640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ce81032f0 0x7f9ce81a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8108ba0 0x7f9ce81a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce6d9d640 1 -- 172.21.15.5:0/1045427089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ce81032f0 msgr2=0x7f9ce81a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce6d9d640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ce81032f0 0x7f9ce81a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce6d9d640 1 -- 172.21.15.5:0/1045427089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8108ba0 msgr2=0x7f9ce81a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce6d9d640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8108ba0 0x7f9ce81a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce6d9d640 1 -- 172.21.15.5:0/1045427089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce819cfb0 con 0x7f9ce8109040 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9ce6d9d640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 0x7f9ce819c910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eb10 tx=0x7f9ce000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9cd77fe640 1 -- 172.21.15.5:0/1045427089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004b20 con 0x7f9ce8109040 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce819d2a0 con 0x7f9ce8109040 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9cd77fe640 1 -- 172.21.15.5:0/1045427089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce0004cc0 con 0x7f9ce8109040 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9cd77fe640 1 -- 172.21.15.5:0/1045427089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0010670 con 0x7f9ce8109040 2024-05-14T15:30:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.487+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81a9580 con 0x7f9ce8109040 2024-05-14T15:30:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.490+0000 7f9cf3e0f740 1 Processor -- start 2024-05-14T15:30:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.490+0000 7f9cd77fe640 1 -- 172.21.15.5:0/1045427089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9ce0005020 con 0x7f9ce8109040 2024-05-14T15:30:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.490+0000 7f9cf3e0f740 1 -- start start 2024-05-14T15:30:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac39d1b0 0x55b3ac73dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac73e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac37c4d0 0x55b3ac73e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3ac6a35d0 con 0x55b3ac39d1b0 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3ac65c890 con 0x55b3ac37c4d0 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b3ac69f590 con 0x55b3ac3742f0 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac39d1b0 0x55b3ac73dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac39d1b0 0x55b3ac73dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44024/0 (socket says 172.21.15.5:44024) 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce5d9b640 1 -- 172.21.15.5:0/210744069 learned_addr learned my addr 172.21.15.5:0/210744069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce659c640 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac73e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac37c4d0 0x55b3ac73e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce659c640 1 -- 172.21.15.5:0/210744069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac37c4d0 msgr2=0x55b3ac73e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce659c640 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac37c4d0 0x55b3ac73e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce659c640 1 -- 172.21.15.5:0/210744069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac39d1b0 msgr2=0x55b3ac73dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce659c640 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac39d1b0 0x55b3ac73dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9cd77fe640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9cd00b2950 0x7f9cd00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce659c640 1 -- 172.21.15.5:0/210744069 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ac3b5210 con 0x55b3ac3742f0 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.491+0000 7f9ce6d9d640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9cd00b2950 0x7f9cd00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.492+0000 7f9ce659c640 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac73e120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8006010 tx=0x7f9cd8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e46e4f127a6336e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.492+0000 7f9cd77fe640 1 -- 172.21.15.5:0/1045427089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9ce00e2fb0 con 0x7f9ce8109040 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.492+0000 7f9cd4ff9640 1 -- 172.21.15.5:0/210744069 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8040070 con 0x55b3ac3742f0 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.492+0000 7f9cd4ff9640 1 -- 172.21.15.5:0/210744069 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd8032e20 con 0x55b3ac3742f0 2024-05-14T15:30:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.492+0000 7f9cd4ff9640 1 -- 172.21.15.5:0/210744069 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8035040 con 0x55b3ac3742f0 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/210744069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 msgr2=0x55b3ac73e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac73e120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8006010 tx=0x7f9cd8000bc0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9ce6d9d640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9cd00b2950 0x7f9cd00b4e60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00e03f0 tx=0x7f9ce0077aa0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/210744069 shutdown_connections 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac73e120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac37c4d0 0x55b3ac73e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/210744069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac39d1b0 0x55b3ac73dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/210744069 >> 172.21.15.5:0/210744069 conn(0x55b3ac391300 msgr2=0x55b3ac73a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/210744069 shutdown_connections 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/210744069 wait complete. 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 -- start start 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.493+0000 7f9cf3e0f740 1 Processor -- start 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac748200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 0x55b3ac748740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3ac74acb0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3ac660370 con 0x55b3ac39d1b0 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b3ac74ae30 con 0x55b3ac3742f0 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9cf3e0f740 10 client.0 _ll_register_callbacks cb 0x55b3ac3b4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 0x55b3ac748740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac748200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 0x55b3ac748740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44034/0 (socket says 172.21.15.5:44034) 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 -- 172.21.15.5:0/862927633 learned_addr learned my addr 172.21.15.5:0/862927633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 -- 172.21.15.5:0/862927633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 msgr2=0x55b3ac748200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac748200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 -- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 msgr2=0x55b3ac754160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 -- 172.21.15.5:0/862927633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ac481620 con 0x55b3ac37c4d0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac748200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ce659c640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 0x55b3ac748740 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004650 tx=0x7f9cd80316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a626ed8b72d1c472 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.494+0000 7f9ccdffb640 1 -- 172.21.15.5:0/862927633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8035070 con 0x55b3ac37c4d0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd8032e20 con 0x55b3ac37c4d0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9ccdffb640 1 -- 172.21.15.5:0/862927633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9cd80318c0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9ccdffb640 1 -- 172.21.15.5:0/862927633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9cd8058050 con 0x55b3ac37c4d0 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 10 client.16821 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3ac757020 con 0x55b3ac37c4d0 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9ccdffb640 1 -- 172.21.15.5:0/862927633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd8032e20 con 0x55b3ac37c4d0 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9ccdffb640 1 -- 172.21.15.5:0/862927633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9cd803c440 con 0x55b3ac37c4d0 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 10 client.16821 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 20 client.16821 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac35bd80 0x55b3ac75a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 4 client.16821 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3ac6b11e0 con 0x55b3ac35bd80 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.495+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac35bd80 0x55b3ac75a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.496+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac35bd80 0x55b3ac75a380 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f507df5bdbbba12 server_cookie=c4645d47a0c1e71b in_seq=0 out_seq=0 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.496+0000 7f9ccdffb640 10 client.16821 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9ccdffb640 1 -- 172.21.15.5:0/862927633 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3ac6b11e0 con 0x55b3ac35bd80 2024-05-14T15:30:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9ccdffb640 10 client.16821 handle_command_reply: tid=0 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 client.16821 shutdown 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 msgr2=0x55b3ac748740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 0x55b3ac748740 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004650 tx=0x7f9cd80316d0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 shutdown_connections 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac35bd80 0x55b3ac75a380 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac37c4d0 0x55b3ac748740 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9ccdffb640 0 client.16821 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/862927633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac3742f0 0x55b3ac748200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.497+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 >> 172.21.15.5:0/862927633 conn(0x55b3ac391300 msgr2=0x55b3ac73f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.498+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 shutdown_connections 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.498+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/862927633 wait complete. 2024-05-14T15:30:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.498+0000 7f9cf3e0f740 20 client.16821 trim_cache size 0 max 16384 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 Processor -- start 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 -- start start 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac73a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac7540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b5d50 con 0x55b3ac3742f0 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b3620 con 0x55b3ac39d1b0 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b1470 con 0x55b3ac37c4d0 2024-05-14T15:30:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac73a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac7540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac73a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44046/0 (socket says 172.21.15.5:44046) 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce5d9b640 1 -- 172.21.15.5:0/1401773707 learned_addr learned my addr 172.21.15.5:0/1401773707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce659c640 1 -- 172.21.15.5:0/1401773707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 msgr2=0x55b3ac754610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce659c640 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce659c640 1 -- 172.21.15.5:0/1401773707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 msgr2=0x55b3ac73a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce659c640 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac73a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce659c640 1 -- 172.21.15.5:0/1401773707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ac7472f0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.513+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.514+0000 7f9ce659c640 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac7540d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cd803c810 tx=0x7f9cd8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7434dfb0a854c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.514+0000 7f9ccffff640 1 -- 172.21.15.5:0/1401773707 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8035070 con 0x55b3ac37c4d0 2024-05-14T15:30:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.514+0000 7f9ccffff640 1 -- 172.21.15.5:0/1401773707 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd8032ba0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.514+0000 7f9ccffff640 1 -- 172.21.15.5:0/1401773707 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8040040 con 0x55b3ac37c4d0 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/1401773707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 msgr2=0x55b3ac7540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac7540d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9cd803c810 tx=0x7f9cd8000bc0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/1401773707 shutdown_connections 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac7540d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac754610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/1401773707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac73a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/1401773707 >> 172.21.15.5:0/1401773707 conn(0x55b3ac391300 msgr2=0x55b3ac739cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/1401773707 shutdown_connections 2024-05-14T15:30:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.515+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/1401773707 wait complete. 2024-05-14T15:30:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.516+0000 7f9cf3e0f740 1 -- start start 2024-05-14T15:30:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.516+0000 7f9cf3e0f740 1 Processor -- start 2024-05-14T15:30:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac75a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac75abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac7c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b1ae0 con 0x55b3ac3742f0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b2f50 con 0x55b3ac39d1b0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81b21b0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9cf3e0f740 10 client.0 _ll_register_callbacks cb 0x7f9cdc0099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac75a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac75a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44048/0 (socket says 172.21.15.5:44048) 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce5d9b640 1 -- 172.21.15.5:0/2026766277 learned_addr learned my addr 172.21.15.5:0/2026766277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac75abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac7c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 -- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 msgr2=0x55b3ac7c04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac7c04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 -- 172.21.15.5:0/2026766277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 msgr2=0x55b3ac75a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac75a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 -- 172.21.15.5:0/2026766277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ac7c09e0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac75a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ce659c640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac75abd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004620 tx=0x7f9cd803ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b169d92572b9b883 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.517+0000 7f9ccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9ccdffb640 1 -- 172.21.15.5:0/2026766277 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8035070 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd8032ba0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9ccdffb640 1 -- 172.21.15.5:0/2026766277 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f9cd803cc10 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9ccdffb640 1 -- 172.21.15.5:0/2026766277 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9cd806d050 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9cf3e0f740 10 client.26422 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3ac7c1b80 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9ccdffb640 1 -- 172.21.15.5:0/2026766277 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd8032ba0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9ccdffb640 1 -- 172.21.15.5:0/2026766277 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9cd80046c0 con 0x55b3ac37c4d0 2024-05-14T15:30:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9cf3e0f740 10 client.26422 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.518+0000 7f9cf3e0f740 10 client.26422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9cf3e0f740 20 client.26422 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac3311c0 0x55b3ac7c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9cf3e0f740 4 client.26422 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3ac6ab050 con 0x55b3ac3311c0 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac3311c0 0x55b3ac7c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9ccdffb640 10 client.26422 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.519+0000 7f9ce5d9b640 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac3311c0 0x55b3ac7c4070 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9197a845b0f49579 server_cookie=4fef29e27d925b90 in_seq=0 out_seq=0 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9ccdffb640 1 -- 172.21.15.5:0/2026766277 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3ac6ab050 con 0x55b3ac3311c0 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9ccdffb640 10 client.26422 handle_command_reply: tid=0 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 client.26422 shutdown 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 msgr2=0x55b3ac75abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac75abd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004620 tx=0x7f9cd803ceb0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 shutdown_connections 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55b3ac3311c0 0x55b3ac7c4070 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b3ac37c4d0 0x55b3ac75abd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3ac39d1b0 0x55b3ac7c04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 --2- 172.21.15.5:0/2026766277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ac3742f0 0x55b3ac75a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 >> 172.21.15.5:0/2026766277 conn(0x55b3ac391300 msgr2=0x55b3ac749630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9ccdffb640 0 client.26422 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.838+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 shutdown_connections 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.839+0000 7f9cf3e0f740 1 -- 172.21.15.5:0/2026766277 wait complete. 2024-05-14T15:30:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.839+0000 7f9cf3e0f740 20 client.26422 trim_cache size 0 max 16384 2024-05-14T15:30:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:30:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:30:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:30:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.839+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9cd00b2950 msgr2=0x7f9cd00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.839+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9cd00b2950 0x7f9cd00b4e60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00e03f0 tx=0x7f9ce0077aa0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 msgr2=0x7f9ce819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 0x7f9ce819c910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eb10 tx=0x7f9ce000c2d0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 shutdown_connections 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9cd00b2950 0x7f9cd00b4e60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce8109040 0x7f9ce819c910 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8108ba0 0x7f9ce81a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 --2- 172.21.15.5:0/1045427089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ce81032f0 0x7f9ce81a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.840+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 >> 172.21.15.5:0/1045427089 conn(0x7f9ce80782e0 msgr2=0x7f9ce806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.841+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 shutdown_connections 2024-05-14T15:30:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:30:53.841+0000 7f9cecf5e640 1 -- 172.21.15.5:0/1045427089 wait complete. 2024-05-14T15:30:53.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 -- 172.21.15.5:0/2359987587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb41034c0 msgr2=0x7f0cb41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 --2- 172.21.15.5:0/2359987587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb41034c0 0x7f0cb41038d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40099c0 tx=0x7f0ca4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 -- 172.21.15.5:0/2359987587 shutdown_connections 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 --2- 172.21.15.5:0/2359987587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb4108740 0x7f0cb41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 --2- 172.21.15.5:0/2359987587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb41034c0 0x7f0cb41038d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 --2- 172.21.15.5:0/2359987587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb4107df0 0x7f0cb4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 -- 172.21.15.5:0/2359987587 >> 172.21.15.5:0/2359987587 conn(0x7f0cb40fedc0 msgr2=0x7f0cb41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 -- 172.21.15.5:0/2359987587 shutdown_connections 2024-05-14T15:30:53.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.885+0000 7f0cb894a700 1 -- 172.21.15.5:0/2359987587 wait complete. 2024-05-14T15:30:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.886+0000 7f0cb894a700 1 Processor -- start 2024-05-14T15:30:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.886+0000 7f0cb894a700 1 -- start start 2024-05-14T15:30:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb894a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 0x7f0cb41a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb894a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4107df0 0x7f0cb41a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb894a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb4108740 0x7f0cb41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb894a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4113370 con 0x7f0cb4107df0 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb894a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41131f0 con 0x7f0cb41034c0 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb894a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41134f0 con 0x7f0cb4108740 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 0x7f0cb41a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb2d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb4108740 0x7f0cb41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 0x7f0cb41a70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:32876/0 (socket says 172.21.15.5:32876) 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4107df0 0x7f0cb41a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb259c700 1 -- 172.21.15.5:0/1063779204 learned_addr learned my addr 172.21.15.5:0/1063779204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb259c700 1 -- 172.21.15.5:0/1063779204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb4108740 msgr2=0x7f0cb41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb4108740 0x7f0cb41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.887+0000 7f0cb259c700 1 -- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4107df0 msgr2=0x7f0cb41a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4107df0 0x7f0cb41a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0cb259c700 1 -- 172.21.15.5:0/1063779204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41a19d0 con 0x7f0cb41034c0 2024-05-14T15:30:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 0x7f0cb41a70b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4006010 tx=0x7f0ca4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0ca37fe700 1 -- 172.21.15.5:0/1063779204 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4026070 con 0x7f0cb41034c0 2024-05-14T15:30:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb41a1c60 con 0x7f0cb41034c0 2024-05-14T15:30:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0ca37fe700 1 -- 172.21.15.5:0/1063779204 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca4004630 con 0x7f0cb41034c0 2024-05-14T15:30:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0ca37fe700 1 -- 172.21.15.5:0/1063779204 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca400fad0 con 0x7f0cb41034c0 2024-05-14T15:30:53.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.888+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb41ade60 con 0x7f0cb41034c0 2024-05-14T15:30:53.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.892+0000 7f0ca37fe700 1 -- 172.21.15.5:0/1063779204 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0ca4029070 con 0x7f0cb41034c0 2024-05-14T15:30:53.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.893+0000 7f0ca37fe700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c9c0b29f0 0x7f0c9c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.894+0000 7f0ca37fe700 1 -- 172.21.15.5:0/1063779204 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0ca40e31b0 con 0x7f0cb41034c0 2024-05-14T15:30:53.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.894+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c9c0b29f0 0x7f0c9c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.894+0000 7f0cc735eb80 1 Processor -- start 2024-05-14T15:30:53.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.895+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c9c0b29f0 0x7f0c9c0b4f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4101230 tx=0x7f0ca8009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.895+0000 7f0cc735eb80 1 -- start start 2024-05-14T15:30:53.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.895+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36e47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:33990/0 (socket says 172.21.15.5:33990) 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e94f0 0x55fac36e9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb259c700 1 -- 172.21.15.5:0/1014814910 learned_addr learned my addr 172.21.15.5:0/1014814910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fac36068e0 con 0x55fac36e94f0 2024-05-14T15:30:53.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fac35a5840 con 0x55fac36e4ce0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fac36028a0 con 0x55fac3566ec0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1014814910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 msgr2=0x55fac36e47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36e47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1014814910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e94f0 msgr2=0x55fac36e9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e94f0 0x55fac36e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.896+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1014814910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac3553950 con 0x55fac36e4ce0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.897+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36e8fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4076ab0 tx=0x7f0ca8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757a18e2f0091b5d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.897+0000 7f0ca0ff9700 1 -- 172.21.15.5:0/1014814910 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8004230 con 0x55fac36e4ce0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.897+0000 7f0ca0ff9700 1 -- 172.21.15.5:0/1014814910 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca800ecf0 con 0x55fac36e4ce0 2024-05-14T15:30:53.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.897+0000 7f0ca0ff9700 1 -- 172.21.15.5:0/1014814910 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8003dd0 con 0x55fac36e4ce0 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 msgr2=0x55fac36e8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36e8fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4076ab0 tx=0x7f0ca8014000 comp rx=0 tx=0).stop 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 shutdown_connections 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e94f0 0x55fac36e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1014814910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36e8fb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.898+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 >> 172.21.15.5:0/1014814910 conn(0x55fac35ae680 msgr2=0x55fac36df530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.899+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 shutdown_connections 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.899+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1014814910 wait complete. 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.899+0000 7f0cc735eb80 1 -- start start 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.899+0000 7f0cc735eb80 1 Processor -- start 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac36ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac36eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fac36f14a0 con 0x55fac3566ec0 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fac356ccb0 con 0x55fac36e4ce0 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fac36f1620 con 0x55fac36e94f0 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cc735eb80 10 client.0 _ll_register_callbacks cb 0x55fac359e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cb1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cb1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36ee730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:32894/0 (socket says 172.21.15.5:32894) 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1561682550 learned_addr learned my addr 172.21.15.5:0/1561682550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.900+0000 7f0cb2d9d700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac36eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1561682550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 msgr2=0x55fac36eec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac36ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac36eec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1561682550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 msgr2=0x55fac36ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac36ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1561682550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac35a2710 con 0x55fac36e4ce0 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac36ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36ee730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8003b70 tx=0x7f0ca8003c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f808b51227690b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0c99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.901+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1561682550 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80054d0 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1561682550 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0ca8026510 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca800ecf0 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1561682550 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ca8005670 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0cc735eb80 10 client.26456 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fac36fbec0 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1561682550 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca800ecf0 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1561682550 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0ca801d740 con 0x55fac36e4ce0 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.902+0000 7f0cc735eb80 10 client.26456 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cc735eb80 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cc735eb80 20 client.26456 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac36fe390 0x55fac37007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cc735eb80 4 client.26456 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:53.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac36142c0 con 0x55fac36fe390 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac36fe390 0x55fac37007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0c99ffb700 10 client.26456 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.903+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac36fe390 0x55fac37007b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb64f2a7d35ce9f9 server_cookie=109d9bfa7460989b in_seq=0 out_seq=0 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1561682550 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fac36142c0 con 0x55fac36fe390 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0c99ffb700 10 client.26456 handle_command_reply: tid=0 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 client.26456 shutdown 2024-05-14T15:30:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 msgr2=0x55fac36ee730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36ee730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8003b70 tx=0x7f0ca8003c50 comp rx=0 tx=0).stop 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 shutdown_connections 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac36fe390 0x55fac37007b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0c99ffb700 0 client.26456 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac36eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac36ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1561682550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac36ee730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.905+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 >> 172.21.15.5:0/1561682550 conn(0x55fac35ae680 msgr2=0x55fac36e34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.906+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 shutdown_connections 2024-05-14T15:30:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.906+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1561682550 wait complete. 2024-05-14T15:30:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.906+0000 7f0cc735eb80 20 client.26456 trim_cache size 0 max 16384 2024-05-14T15:30:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.931+0000 7f0cc735eb80 1 Processor -- start 2024-05-14T15:30:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.931+0000 7f0cc735eb80 1 -- start start 2024-05-14T15:30:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 0x55fac36e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e94f0 0x55fac36eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b93f0 con 0x55fac36e4ce0 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b6c80 con 0x55fac36e94f0 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b4ad0 con 0x55fac3566ec0 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 0x55fac36e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 0x55fac36e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44078/0 (socket says 172.21.15.5:44078) 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1199720062 learned_addr learned my addr 172.21.15.5:0/1199720062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb2d9d700 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e94f0 0x55fac36eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb259c700 1 -- 172.21.15.5:0/1199720062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e94f0 msgr2=0x55fac36eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e94f0 0x55fac36eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb259c700 1 -- 172.21.15.5:0/1199720062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 msgr2=0x55fac36e8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 0x55fac36e8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb259c700 1 -- 172.21.15.5:0/1199720062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac36e57c0 con 0x55fac3566ec0 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.932+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 0x55fac36e8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.933+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36df690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40a61e0 tx=0x7f0ca4009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c3f6e91202061c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.933+0000 7f0c9bfff700 1 -- 172.21.15.5:0/1199720062 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40aa160 con 0x55fac3566ec0 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.933+0000 7f0c9bfff700 1 -- 172.21.15.5:0/1199720062 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca40ae510 con 0x55fac3566ec0 2024-05-14T15:30:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.933+0000 7f0c9bfff700 1 -- 172.21.15.5:0/1199720062 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40aa620 con 0x55fac3566ec0 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1199720062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 msgr2=0x55fac36df690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36df690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40a61e0 tx=0x7f0ca4009d30 comp rx=0 tx=0).stop 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1199720062 shutdown_connections 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e94f0 0x55fac36eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac3566ec0 0x55fac36df690 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1199720062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac36e4ce0 0x55fac36e8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1199720062 >> 172.21.15.5:0/1199720062 conn(0x55fac35ae680 msgr2=0x55fac36e34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1199720062 shutdown_connections 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1199720062 wait complete. 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 -- start start 2024-05-14T15:30:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.934+0000 7f0cc735eb80 1 Processor -- start 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac3762210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac3762750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac375c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b5140 con 0x55fac3566ec0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b65b0 con 0x55fac36e4ce0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41b5810 con 0x55fac36e94f0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cc735eb80 10 client.0 _ll_register_callbacks cb 0x55fac359e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac3762210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac3762750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac3762210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44080/0 (socket says 172.21.15.5:44080) 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb259c700 1 -- 172.21.15.5:0/1691186899 learned_addr learned my addr 172.21.15.5:0/1691186899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb2d9d700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac375c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1691186899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 msgr2=0x55fac375c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac375c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1691186899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 msgr2=0x55fac3762210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac3762210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 -- 172.21.15.5:0/1691186899 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac375ceb0 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac3762210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0cb1d9b700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac3762750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005830 tx=0x7f0ca80037b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0f9675747ffffa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.935+0000 7f0c99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1691186899 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca801d610 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1691186899 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9474+0+0 (secure 0 0 0) 0x7f0ca801d7b0 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca40ae510 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1691186899 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ca8026510 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 10 client.26462 fetch_fsmap learned FSMap version 30 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fac3769950 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1691186899 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca40ae510 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1691186899 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0ca803d460 con 0x55fac36e4ce0 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 10 client.26462 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 10 client.26462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 20 client.26462 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.936+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac3769ff0 0x55fac376c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.937+0000 7f0cc735eb80 4 client.26462 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:30:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.937+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fac360e360 con 0x55fac3769ff0 2024-05-14T15:30:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.937+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac3769ff0 0x55fac376c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.937+0000 7f0c99ffb700 10 client.26462 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.937+0000 7f0cb259c700 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac3769ff0 0x55fac376c410 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c1e6807b020f2a server_cookie=e760e87cf8feea9b in_seq=0 out_seq=0 2024-05-14T15:30:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0c99ffb700 1 -- 172.21.15.5:0/1691186899 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fac360e360 con 0x55fac3769ff0 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0c99ffb700 10 client.26462 handle_command_reply: tid=0 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0cc735eb80 1 client.26462 shutdown 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 msgr2=0x55fac3762750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac3762750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005830 tx=0x7f0ca80037b0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 shutdown_connections 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.938+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fac3769ff0 0x55fac376c410 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0c99ffb700 0 client.26462 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fac36e94f0 0x55fac375c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fac3566ec0 0x55fac3762210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 1 --2- 172.21.15.5:0/1691186899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fac36e4ce0 0x55fac3762750 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8005830 tx=0x7f0ca80037b0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 >> 172.21.15.5:0/1691186899 conn(0x55fac35ae680 msgr2=0x55fac36df3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 shutdown_connections 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 1 -- 172.21.15.5:0/1691186899 wait complete. 2024-05-14T15:30:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.939+0000 7f0cc735eb80 20 client.26462 trim_cache size 0 max 16384 2024-05-14T15:30:53.945 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:30:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:30:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "f9ac374f-7d8f-489c-abe5-ff862b6f3ad1", 2024-05-14T15:30:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:30:53.945 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c9c0b29f0 msgr2=0x7f0c9c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c9c0b29f0 0x7f0c9c0b4f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4101230 tx=0x7f0ca8009260 comp rx=0 tx=0).stop 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 msgr2=0x7f0cb41a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 0x7f0cb41a70b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4006010 tx=0x7f0ca4004270 comp rx=0 tx=0).stop 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 shutdown_connections 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c9c0b29f0 0x7f0c9c0b4f00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cb4108740 0x7f0cb41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cb41034c0 0x7f0cb41a70b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 --2- 172.21.15.5:0/1063779204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4107df0 0x7f0cb41a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 >> 172.21.15.5:0/1063779204 conn(0x7f0cb40fedc0 msgr2=0x7f0cb406b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 shutdown_connections 2024-05-14T15:30:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:53.940+0000 7f0cb894a700 1 -- 172.21.15.5:0/1063779204 wait complete. 2024-05-14T15:30:53.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:53 smithi005 ceph-mon[31559]: pgmap v186: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 429 op/s 2024-05-14T15:30:54.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 544675 10.88 MB/sec execute 135 sec latency 181.500 ms 2024-05-14T15:30:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:53 smithi144 ceph-mon[35650]: pgmap v186: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 429 op/s 2024-05-14T15:30:54.298 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:30:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setuser ceph since I am not root 2024-05-14T15:30:54.298 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:30:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setgroup ceph since I am not root 2024-05-14T15:30:54.298 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mds[65770]: -- 172.21.15.144:0/2572897040 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e149df44e0 con 0x55e149e3f180 2024-05-14T15:30:54.299 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mds[65770]: -- 172.21.15.144:0/2572897040 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e149df7a40 con 0x55e149e3f180 2024-05-14T15:30:54.299 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:30:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: starting mds.e at 2024-05-14T15:30:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:53 smithi193 ceph-mon[36001]: pgmap v186: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.1 MiB/s wr, 429 op/s 2024-05-14T15:30:55.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:54 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:30:55.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:54 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:55.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:54 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:30:55.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:54 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:55.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:54 smithi005 ceph-mon[31559]: daemon mds.e restarted 2024-05-14T15:30:55.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:30:55.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 547049 10.92 MB/sec execute 136 sec latency 46.974 ms 2024-05-14T15:30:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:30:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:30:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:55.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:54 smithi144 ceph-mon[35650]: daemon mds.e restarted 2024-05-14T15:30:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:54 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:30:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:54 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:54 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:30:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:54 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:30:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:54 smithi193 ceph-mon[36001]: daemon mds.e restarted 2024-05-14T15:30:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:30:56.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 548523 10.89 MB/sec execute 137 sec latency 15.333 ms 2024-05-14T15:30:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: pgmap v187: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-05-14T15:30:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T15:30:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6834/1180616641,v1:172.21.15.144:6835/1180616641] up:boot 2024-05-14T15:30:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:30:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:30:56.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:30:56.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:55 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:30:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: pgmap v187: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-05-14T15:30:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T15:30:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6834/1180616641,v1:172.21.15.144:6835/1180616641] up:boot 2024-05-14T15:30:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:30:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:30:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:30:56.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:55 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: pgmap v187: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6834/1180616641,v1:172.21.15.144:6835/1180616641] up:boot 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:30:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:55 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:30:57.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 550847 10.91 MB/sec execute 138 sec latency 46.993 ms 2024-05-14T15:30:58.122 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:57 smithi005 ceph-mon[31559]: pgmap v189: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:30:58.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 551783 10.85 MB/sec execute 139 sec latency 173.740 ms 2024-05-14T15:30:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:57 smithi144 ceph-mon[35650]: pgmap v189: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:30:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:57 smithi193 ceph-mon[36001]: pgmap v189: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 -- 172.21.15.5:0/2108594595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c81034c0 msgr2=0x7f25c81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 --2- 172.21.15.5:0/2108594595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c81034c0 0x7f25c81038d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f25b80099c0 tx=0x7f25b8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 -- 172.21.15.5:0/2108594595 shutdown_connections 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 --2- 172.21.15.5:0/2108594595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c8108740 0x7f25c81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 --2- 172.21.15.5:0/2108594595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c81034c0 0x7f25c81038d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 --2- 172.21.15.5:0/2108594595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25c8107df0 0x7f25c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 -- 172.21.15.5:0/2108594595 >> 172.21.15.5:0/2108594595 conn(0x7f25c80fedc0 msgr2=0x7f25c81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 -- 172.21.15.5:0/2108594595 shutdown_connections 2024-05-14T15:30:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.733+0000 7f25cccb3700 1 -- 172.21.15.5:0/2108594595 wait complete. 2024-05-14T15:30:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 Processor -- start 2024-05-14T15:30:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 -- start start 2024-05-14T15:30:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 0x7f25c819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25c8107df0 0x7f25c819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c8108740 0x7f25c819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8113370 con 0x7f25c8108740 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81131f0 con 0x7f25c81034c0 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25cccb3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81134f0 con 0x7f25c8107df0 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.734+0000 7f25c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 0x7f25c819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 0x7f25c819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53746/0 (socket says 172.21.15.5:53746) 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c659c700 1 -- 172.21.15.5:0/1621792739 learned_addr learned my addr 172.21.15.5:0/1621792739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c8108740 0x7f25c819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25c8107df0 0x7f25c819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 -- 172.21.15.5:0/1621792739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25c8107df0 msgr2=0x7f25c819c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25c8107df0 0x7f25c819c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 -- 172.21.15.5:0/1621792739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 msgr2=0x7f25c819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 0x7f25c819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 -- 172.21.15.5:0/1621792739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c81a6f80 con 0x7f25c8108740 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c659c700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 0x7f25c819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c8108740 0x7f25c819c6d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f25c000efc0 tx=0x7f25c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25b77fe700 1 -- 172.21.15.5:0/1621792739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00092e0 con 0x7f25c8108740 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25b77fe700 1 -- 172.21.15.5:0/1621792739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c0004bb0 con 0x7f25c8108740 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.735+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c81a7210 con 0x7f25c8108740 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.736+0000 7f25b77fe700 1 -- 172.21.15.5:0/1621792739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0004020 con 0x7f25c8108740 2024-05-14T15:30:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.736+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81a7700 con 0x7f25c8108740 2024-05-14T15:30:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.739+0000 7f25b77fe700 1 -- 172.21.15.5:0/1621792739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f25c001b030 con 0x7f25c8108740 2024-05-14T15:30:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.740+0000 7f25b77fe700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25b00b29f0 0x7f25b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.740+0000 7f25c659c700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25b00b29f0 0x7f25b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.740+0000 7f25b77fe700 1 -- 172.21.15.5:0/1621792739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f25c00e2470 con 0x7f25c8108740 2024-05-14T15:30:58.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.740+0000 7f25db6c7b80 1 Processor -- start 2024-05-14T15:30:58.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.740+0000 7f25c659c700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25b00b29f0 0x7f25b00b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009990 tx=0x7f25b801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:58.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.740+0000 7f25db6c7b80 1 -- start start 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 0x55d46db54720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 0x55d46db58f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d46da76860 con 0x55d46db54c60 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d46da157c0 con 0x55d46db59470 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d46da72820 con 0x55d46d9d6e40 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 0x55d46db58f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25c659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 0x55d46db54720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 0x55d46db58f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54790/0 (socket says 172.21.15.5:54790) 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25c659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 0x55d46db54720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39922/0 (socket says 172.21.15.5:39922) 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25c5d9b700 1 -- 172.21.15.5:0/1357853256 learned_addr learned my addr 172.21.15.5:0/1357853256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.741+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c6d9d700 1 -- 172.21.15.5:0/1357853256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 msgr2=0x55d46db54720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 0x55d46db54720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c6d9d700 1 -- 172.21.15.5:0/1357853256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 msgr2=0x55d46db58f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 0x55d46db58f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c6d9d700 1 -- 172.21.15.5:0/1357853256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46d9c38d0 con 0x55d46db59470 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c659c700 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 0x55d46db54720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 0x55d46db58f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db59880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25c00df800 tx=0x7f25c00a8f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce36c71fa3f014 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25b4ff9700 1 -- 172.21.15.5:0/1357853256 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00ad7d0 con 0x55d46db59470 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25b4ff9700 1 -- 172.21.15.5:0/1357853256 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c00b21f0 con 0x55d46db59470 2024-05-14T15:30:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.742+0000 7f25b4ff9700 1 -- 172.21.15.5:0/1357853256 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00b1f60 con 0x55d46db59470 2024-05-14T15:30:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 -- 172.21.15.5:0/1357853256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 msgr2=0x55d46db59880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db59880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25c00df800 tx=0x7f25c00a8f30 comp rx=0 tx=0).stop 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 -- 172.21.15.5:0/1357853256 shutdown_connections 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db59880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46d9d6e40 0x55d46db54720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/1357853256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db54c60 0x55d46db58f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 -- 172.21.15.5:0/1357853256 >> 172.21.15.5:0/1357853256 conn(0x55d46da1e600 msgr2=0x55d46db4f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 -- 172.21.15.5:0/1357853256 shutdown_connections 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.743+0000 7f25db6c7b80 1 -- 172.21.15.5:0/1357853256 wait complete. 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 -- start start 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 Processor -- start 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db5e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 0x55d46db5e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db59470 0x55d46db5ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d46db613c0 con 0x55d46d9d6e40 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d46d9dcc30 con 0x55d46db54c60 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d46db61540 con 0x55d46db59470 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.744+0000 7f25db6c7b80 10 client.0 _ll_register_callbacks cb 0x55d46da0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db5e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 0x55d46db5e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 0x55d46db5e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53778/0 (socket says 172.21.15.5:53778) 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 -- 172.21.15.5:0/3038647323 learned_addr learned my addr 172.21.15.5:0/3038647323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db5e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54794/0 (socket says 172.21.15.5:54794) 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db59470 0x55d46db5ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 -- 172.21.15.5:0/3038647323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db59470 msgr2=0x55d46db5ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db59470 0x55d46db5ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 -- 172.21.15.5:0/3038647323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 msgr2=0x55d46db5e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db5e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 -- 172.21.15.5:0/3038647323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46da12690 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 0x55d46db5e6f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25bc012980 tx=0x7f25bc012c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f82bcbc62921c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25adffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25adffb700 1 -- 172.21.15.5:0/3038647323 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc004050 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25adffb700 1 -- 172.21.15.5:0/3038647323 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f25bc0041f0 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.745+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c00b21f0 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25adffb700 1 -- 172.21.15.5:0/3038647323 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f25bc0043f0 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 10 client.26480 fetch_fsmap learned FSMap version 32 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d46db6c050 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25adffb700 1 -- 172.21.15.5:0/3038647323 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c00b21f0 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25adffb700 1 -- 172.21.15.5:0/3038647323 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f25bc00f8d0 con 0x55d46db54c60 2024-05-14T15:30:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 10 client.26480 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 10 client.26480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 20 client.26480 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46db6e690 0x55d46db70ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 4 client.26480 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d46da84240 con 0x55d46db6e690 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.746+0000 7f25c659c700 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46db6e690 0x55d46db70ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.747+0000 7f25c659c700 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46db6e690 0x55d46db70ab0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=408880a8814f853 server_cookie=d17af7f9fa7403a1 in_seq=0 out_seq=0 2024-05-14T15:30:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.747+0000 7f25adffb700 10 client.26480 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25adffb700 1 -- 172.21.15.5:0/3038647323 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d46da84240 con 0x55d46db6e690 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25adffb700 10 client.26480 handle_command_reply: tid=0 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 client.26480 shutdown 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 msgr2=0x55d46db5e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 0x55d46db5e6f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25bc012980 tx=0x7f25bc012c90 comp rx=0 tx=0).stop 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 shutdown_connections 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46db6e690 0x55d46db70ab0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db59470 0x55d46db5ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db5e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.749+0000 7f25adffb700 0 client.26480 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.750+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/3038647323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db54c60 0x55d46db5e6f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.750+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 >> 172.21.15.5:0/3038647323 conn(0x55d46da1e600 msgr2=0x55d46db53450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.750+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 shutdown_connections 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.750+0000 7f25db6c7b80 1 -- 172.21.15.5:0/3038647323 wait complete. 2024-05-14T15:30:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.750+0000 7f25db6c7b80 20 client.26480 trim_cache size 0 max 16384 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 Processor -- start 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 -- start start 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 0x55d46db5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db6a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db59470 0x55d46db76220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b50d0 con 0x55d46db59470 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b2960 con 0x55d46d9d6e40 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b07b0 con 0x55d46db54c60 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 0x55d46db5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db6a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.771+0000 7f25c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db59470 0x55d46db76220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db6a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39954/0 (socket says 172.21.15.5:39954) 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 0x55d46db5eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53780/0 (socket says 172.21.15.5:53780) 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c5d9b700 1 -- 172.21.15.5:0/45697499 learned_addr learned my addr 172.21.15.5:0/45697499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c6d9d700 1 -- 172.21.15.5:0/45697499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 msgr2=0x55d46db6a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db6a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c6d9d700 1 -- 172.21.15.5:0/45697499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 msgr2=0x55d46db5eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 0x55d46db5eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c6d9d700 1 -- 172.21.15.5:0/45697499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46db5df80 con 0x55d46db59470 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db6a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c659c700 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 0x55d46db5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db59470 0x55d46db76220 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f25c00df830 tx=0x7f25c000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fb3e65638f64f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25affff700 1 -- 172.21.15.5:0/45697499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00ad7d0 con 0x55d46db59470 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25affff700 1 -- 172.21.15.5:0/45697499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c00e2140 con 0x55d46db59470 2024-05-14T15:30:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.772+0000 7f25affff700 1 -- 172.21.15.5:0/45697499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00b1fc0 con 0x55d46db59470 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 -- 172.21.15.5:0/45697499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db59470 msgr2=0x55d46db76220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db59470 0x55d46db76220 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f25c00df830 tx=0x7f25c000c9f0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 -- 172.21.15.5:0/45697499 shutdown_connections 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46db59470 0x55d46db76220 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46d9d6e40 0x55d46db5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/45697499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db6a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 -- 172.21.15.5:0/45697499 >> 172.21.15.5:0/45697499 conn(0x55d46da1e600 msgr2=0x55d46db4f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 -- 172.21.15.5:0/45697499 shutdown_connections 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.773+0000 7f25db6c7b80 1 -- 172.21.15.5:0/45697499 wait complete. 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 -- start start 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 Processor -- start 2024-05-14T15:30:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db62f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b0e20 con 0x55d46d9d6e40 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b2290 con 0x55d46db59470 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54812/0 (socket says 172.21.15.5:54812) 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25c659c700 1 -- 172.21.15.5:0/4175119598 learned_addr learned my addr 172.21.15.5:0/4175119598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b14f0 con 0x55d46db54c60 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db62f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25db6c7b80 10 client.0 _ll_register_callbacks cb 0x55d46da0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.774+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c659c700 1 -- 172.21.15.5:0/4175119598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 msgr2=0x55d46db629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c659c700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c659c700 1 -- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 msgr2=0x55d46db62f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:30:58.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c659c700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db62f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c659c700 1 -- 172.21.15.5:0/4175119598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46dbd7e80 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c6d9d700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db62f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25c659c700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db624b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f25b80096a0 tx=0x7f25b8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835548de1af2fc6e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25adffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25adffb700 1 -- 172.21.15.5:0/4175119598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8038060 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25adffb700 1 -- 172.21.15.5:0/4175119598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f25b8005580 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c00e2140 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25adffb700 1 -- 172.21.15.5:0/4175119598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f25b803f050 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25db6c7b80 10 client.16860 fetch_fsmap learned FSMap version 32 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d46dbd90f0 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.775+0000 7f25adffb700 1 -- 172.21.15.5:0/4175119598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c00e2140 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25adffb700 1 -- 172.21.15.5:0/4175119598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f25b8026550 con 0x55d46d9d6e40 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25db6c7b80 10 client.16860 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25db6c7b80 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25db6c7b80 20 client.16860 populate_metadata read hostname 'smithi005' 2024-05-14T15:30:58.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46dbd97b0 0x55d46dbdbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25db6c7b80 4 client.16860 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d46da7e2e0 con 0x55d46dbd97b0 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46dbd97b0 0x55d46dbdbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25adffb700 10 client.16860 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.776+0000 7f25c5d9b700 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46dbd97b0 0x55d46dbdbbd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5fd9ade61926e47 server_cookie=f56d5dcee8a1d8b8 in_seq=0 out_seq=0 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25adffb700 1 -- 172.21.15.5:0/4175119598 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d46da7e2e0 con 0x55d46dbd97b0 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25adffb700 10 client.16860 handle_command_reply: tid=0 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 client.16860 shutdown 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 msgr2=0x55d46db624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db624b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f25b80096a0 tx=0x7f25b8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 shutdown_connections 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d46dbd97b0 0x55d46dbdbbd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d46db59470 0x55d46db62f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d9d6e40 0x55d46db624b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 --2- 172.21.15.5:0/4175119598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d46db54c60 0x55d46db629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 >> 172.21.15.5:0/4175119598 conn(0x55d46da1e600 msgr2=0x55d46db5c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25adffb700 0 client.16860 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 shutdown_connections 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 1 -- 172.21.15.5:0/4175119598 wait complete. 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.777+0000 7f25db6c7b80 20 client.16860 trim_cache size 0 max 16384 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:30:58.783 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:30:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25b00b29f0 msgr2=0x7f25b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25b00b29f0 0x7f25b00b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009990 tx=0x7f25b801a040 comp rx=0 tx=0).stop 2024-05-14T15:30:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c8108740 msgr2=0x7f25c819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:30:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c8108740 0x7f25c819c6d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f25c000efc0 tx=0x7f25c000c3a0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 shutdown_connections 2024-05-14T15:30:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25b00b29f0 0x7f25b00b4f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c8108740 0x7f25c819c6d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25c81034c0 0x7f25c819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 --2- 172.21.15.5:0/1621792739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25c8107df0 0x7f25c819c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:30:58.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 >> 172.21.15.5:0/1621792739 conn(0x7f25c80fedc0 msgr2=0x7f25c8101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:30:58.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 shutdown_connections 2024-05-14T15:30:58.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:30:58.778+0000 7f25cccb3700 1 -- 172.21.15.5:0/1621792739 wait complete. 2024-05-14T15:30:59.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 554274 10.88 MB/sec execute 140 sec latency 16.731 ms 2024-05-14T15:30:59.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:30:59.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:30:59.918 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:00.042 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:30:59 smithi005 ceph-mon[31559]: pgmap v190: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:31:00.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555769 10.85 MB/sec execute 141 sec latency 33.674 ms 2024-05-14T15:31:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:30:59 smithi144 ceph-mon[35650]: pgmap v190: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:31:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:30:59 smithi193 ceph-mon[36001]: pgmap v190: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:31:01.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 556562 10.79 MB/sec execute 142 sec latency 57.207 ms 2024-05-14T15:31:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 -- 172.21.15.5:0/3317184186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a501045d0 msgr2=0x7f5a501049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 --2- 172.21.15.5:0/3317184186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a501045d0 0x7f5a501049e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00b0d0 tx=0x7f5a4c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:31:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 -- 172.21.15.5:0/3317184186 shutdown_connections 2024-05-14T15:31:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 --2- 172.21.15.5:0/3317184186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a50109850 0x7f5a501111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 --2- 172.21.15.5:0/3317184186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a501045d0 0x7f5a501049e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 --2- 172.21.15.5:0/3317184186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 0x7f5a50109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 -- 172.21.15.5:0/3317184186 >> 172.21.15.5:0/3317184186 conn(0x7f5a500ffea0 msgr2=0x7f5a501022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 -- 172.21.15.5:0/3317184186 shutdown_connections 2024-05-14T15:31:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.706+0000 7f5a578fc700 1 -- 172.21.15.5:0/3317184186 wait complete. 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.707+0000 7f5a578fc700 1 Processor -- start 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.707+0000 7f5a578fc700 1 -- start start 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a578fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 0x7f5a5019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a578fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 0x7f5a5019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a578fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a50109850 0x7f5a5019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a578fc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a501144a0 con 0x7f5a50109850 2024-05-14T15:31:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a578fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a50114320 con 0x7f5a50108f00 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a578fc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a50114620 con 0x7f5a501045d0 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a55698700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 0x7f5a5019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a54e97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 0x7f5a5019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a55698700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 0x7f5a5019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39974/0 (socket says 172.21.15.5:39974) 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a55698700 1 -- 172.21.15.5:0/2810453075 learned_addr learned my addr 172.21.15.5:0/2810453075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a55e99700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a50109850 0x7f5a5019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a54e97700 1 -- 172.21.15.5:0/2810453075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 msgr2=0x7f5a5019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a54e97700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 0x7f5a5019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a54e97700 1 -- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a50109850 msgr2=0x7f5a5019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a54e97700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a50109850 0x7f5a5019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a54e97700 1 -- 172.21.15.5:0/2810453075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5019fce0 con 0x7f5a50108f00 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.708+0000 7f5a55698700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 0x7f5a5019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.709+0000 7f5a54e97700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 0x7f5a5019b3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e9c0 tx=0x7f5a4000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.709+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000c9c0 con 0x7f5a50108f00 2024-05-14T15:31:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.709+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a4000cb60 con 0x7f5a50108f00 2024-05-14T15:31:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.709+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5019ffd0 con 0x7f5a50108f00 2024-05-14T15:31:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.709+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40018680 con 0x7f5a50108f00 2024-05-14T15:31:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.709+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a501a78f0 con 0x7f5a50108f00 2024-05-14T15:31:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.710+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a500512c0 con 0x7f5a50108f00 2024-05-14T15:31:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.712+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5a40010430 con 0x7f5a50108f00 2024-05-14T15:31:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.713+0000 7f5a467fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5a3c0b29f0 0x7f5a3c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.713+0000 7f5a55698700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5a3c0b29f0 0x7f5a3c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.713+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5a400e2eb0 con 0x7f5a50108f00 2024-05-14T15:31:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.713+0000 7f5a55698700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5a3c0b29f0 0x7f5a3c0b4f00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00bfd0 tx=0x7f5a4c00b040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:01.716+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a400aa4d0 con 0x7f5a50108f00 2024-05-14T15:31:02.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.019+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a5002e5f0 con 0x7f5a50108f00 2024-05-14T15:31:02.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:01 smithi005 ceph-mon[31559]: pgmap v191: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:31:02.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.020+0000 7f5a467fc700 1 -- 172.21.15.5:0/2810453075 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7f5a400a9dd0 con 0x7f5a50108f00 2024-05-14T15:31:02.022 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:31:02.022 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:31:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.026+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5a3c0b29f0 msgr2=0x7f5a3c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.026+0000 7f5a578fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5a3c0b29f0 0x7f5a3c0b4f00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00bfd0 tx=0x7f5a4c00b040 comp rx=0 tx=0).stop 2024-05-14T15:31:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.026+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 msgr2=0x7f5a5019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.026+0000 7f5a578fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 0x7f5a5019b3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e9c0 tx=0x7f5a4000ecd0 comp rx=0 tx=0).stop 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 shutdown_connections 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5a3c0b29f0 0x7f5a3c0b4f00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a50109850 0x7f5a5019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a501045d0 0x7f5a5019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 --2- 172.21.15.5:0/2810453075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50108f00 0x7f5a5019b3b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 >> 172.21.15.5:0/2810453075 conn(0x7f5a500ffea0 msgr2=0x7f5a5010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 shutdown_connections 2024-05-14T15:31:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:02.027+0000 7f5a578fc700 1 -- 172.21.15.5:0/2810453075 wait complete. 2024-05-14T15:31:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:31:02.100 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:01 smithi144 ceph-mon[35650]: pgmap v191: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:31:02.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:01 smithi193 ceph-mon[36001]: pgmap v191: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 406 op/s 2024-05-14T15:31:02.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 558260 10.80 MB/sec execute 143 sec latency 23.021 ms 2024-05-14T15:31:02.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:31:03.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:02 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2810453075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:31:03.111 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:03.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 559672 10.76 MB/sec execute 144 sec latency 47.108 ms 2024-05-14T15:31:03.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:02 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2810453075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:31:03.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:02 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2810453075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:31:04.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 561817 10.78 MB/sec execute 145 sec latency 20.328 ms 2024-05-14T15:31:04.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:03 smithi005 ceph-mon[31559]: pgmap v192: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-05-14T15:31:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:03 smithi144 ceph-mon[35650]: pgmap v192: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-05-14T15:31:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:03 smithi193 ceph-mon[36001]: pgmap v192: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 -- 172.21.15.5:0/3718024579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690108770 msgr2=0x7f36901100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3718024579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690108770 0x7f36901100d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f368800b0d0 tx=0x7f368800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 -- 172.21.15.5:0/3718024579 shutdown_connections 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3718024579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690108770 0x7f36901100d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3718024579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690103540 0x7f3690103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3718024579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 0x7f3690108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 -- 172.21.15.5:0/3718024579 >> 172.21.15.5:0/3718024579 conn(0x7f3690078e50 msgr2=0x7f3690079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 -- 172.21.15.5:0/3718024579 shutdown_connections 2024-05-14T15:31:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 -- 172.21.15.5:0/3718024579 wait complete. 2024-05-14T15:31:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 Processor -- start 2024-05-14T15:31:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.986+0000 7f3696fb7700 1 -- start start 2024-05-14T15:31:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3696fb7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 0x7f369019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3696fb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 0x7f369019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3696fb7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690108770 0x7f369019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3696fb7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36901133a0 con 0x7f3690103540 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3696fb7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3690113220 con 0x7f3690107e20 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3696fb7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3690113520 con 0x7f3690108770 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 0x7f369019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 0x7f369019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54856/0 (socket says 172.21.15.5:54856) 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 -- 172.21.15.5:0/3506173426 learned_addr learned my addr 172.21.15.5:0/3506173426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f368ffff700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 0x7f369019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3695554700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690108770 0x7f369019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 -- 172.21.15.5:0/3506173426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690108770 msgr2=0x7f369019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690108770 0x7f369019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 -- 172.21.15.5:0/3506173426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 msgr2=0x7f369019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 0x7f369019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3694d53700 1 -- 172.21.15.5:0/3506173426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369019fcf0 con 0x7f3690103540 2024-05-14T15:31:04.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f3695554700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690108770 0x7f369019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.987+0000 7f368ffff700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 0x7f369019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.988+0000 7f3694d53700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 0x7f369019ae20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f368400e9a0 tx=0x7f368400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.988+0000 7f368dffb700 1 -- 172.21.15.5:0/3506173426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368400cbd0 con 0x7f3690103540 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.988+0000 7f368dffb700 1 -- 172.21.15.5:0/3506173426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368400cd70 con 0x7f3690103540 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.988+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369019ffe0 con 0x7f3690103540 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.988+0000 7f368dffb700 1 -- 172.21.15.5:0/3506173426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684018680 con 0x7f3690103540 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.988+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36901a7900 con 0x7f3690103540 2024-05-14T15:31:04.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.991+0000 7f368dffb700 1 -- 172.21.15.5:0/3506173426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3684010430 con 0x7f3690103540 2024-05-14T15:31:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.992+0000 7f368dffb700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36800b29f0 0x7f36800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.992+0000 7f368ffff700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36800b29f0 0x7f36800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.992+0000 7f368dffb700 1 -- 172.21.15.5:0/3506173426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f36840e2ce0 con 0x7f3690103540 2024-05-14T15:31:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.992+0000 7f36a59cbb80 1 Processor -- start 2024-05-14T15:31:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.992+0000 7f368ffff700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36800b29f0 0x7f36800b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f369006b510 tx=0x7f367c009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.992+0000 7f36a59cbb80 1 -- start start 2024-05-14T15:31:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55beab5d6940 con 0x55beab6bad80 2024-05-14T15:31:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55beab53bda0 con 0x55beab6b4e10 2024-05-14T15:31:04.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55beab5d2900 con 0x55beab536e40 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f368ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f368ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54862/0 (socket says 172.21.15.5:54862) 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f368ffff700 1 -- 172.21.15.5:0/3504059601 learned_addr learned my addr 172.21.15.5:0/3504059601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3695554700 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 -- 172.21.15.5:0/3504059601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 msgr2=0x55beab6b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 -- 172.21.15.5:0/3504059601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 msgr2=0x55beab6b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 -- 172.21.15.5:0/3504059601 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beab5238d0 con 0x55beab536e40 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.993+0000 7f3694d53700 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6ba840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36840e0010 tx=0x7f36840a9390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240630a494aee54d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.994+0000 7f367affd700 1 -- 172.21.15.5:0/3504059601 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36840b2350 con 0x55beab536e40 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.994+0000 7f367affd700 1 -- 172.21.15.5:0/3504059601 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840cae50 con 0x55beab536e40 2024-05-14T15:31:04.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.994+0000 7f367affd700 1 -- 172.21.15.5:0/3504059601 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36840b27d0 con 0x55beab536e40 2024-05-14T15:31:04.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.994+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3504059601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 msgr2=0x55beab6ba840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:04.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.994+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6ba840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36840e0010 tx=0x7f36840a9390 comp rx=0 tx=0).stop 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3504059601 shutdown_connections 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6ba840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3504059601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3504059601 >> 172.21.15.5:0/3504059601 conn(0x55beab57e630 msgr2=0x55beab6afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3504059601 shutdown_connections 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3504059601 wait complete. 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 -- start start 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.995+0000 7f36a59cbb80 1 Processor -- start 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55beab6c17c0 con 0x55beab6bad80 2024-05-14T15:31:05.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55beab58fed0 con 0x55beab6b4e10 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55beab6c1940 con 0x55beab536e40 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f36a59cbb80 10 client.0 _ll_register_callbacks cb 0x55beab56e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6be510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:40004/0 (socket says 172.21.15.5:40004) 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f368ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 -- 172.21.15.5:0/3205218194 learned_addr learned my addr 172.21.15.5:0/3205218194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3695554700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 -- 172.21.15.5:0/3205218194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 msgr2=0x55beab6bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 -- 172.21.15.5:0/3205218194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 msgr2=0x55beab6bef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6bef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 -- 172.21.15.5:0/3205218194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beab58d760 con 0x55beab536e40 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f368ffff700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f3694d53700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6be510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36840e0070 tx=0x7f3684010bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700738ca1427838c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:05.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.996+0000 7f366ffff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f366ffff700 1 -- 172.21.15.5:0/3205218194 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368406d020 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36840cae50 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f366ffff700 1 -- 172.21.15.5:0/3205218194 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3684010720 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f366ffff700 1 -- 172.21.15.5:0/3205218194 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36840a9ba0 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 10 client.26479 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55beab6c17c0 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f366ffff700 1 -- 172.21.15.5:0/3205218194 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840cae50 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f366ffff700 1 -- 172.21.15.5:0/3205218194 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f368406f330 con 0x55beab536e40 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 10 client.26479 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 10 client.26479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 20 client.26479 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab6ce710 0x55beab6d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 4 client.26479 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:05.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beab5e4320 con 0x55beab6ce710 2024-05-14T15:31:05.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.997+0000 7f368ffff700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab6ce710 0x55beab6d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.998+0000 7f368ffff700 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab6ce710 0x55beab6d0b30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fcb441d549e83f2 server_cookie=7475043b8aff6b66 in_seq=0 out_seq=0 2024-05-14T15:31:05.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.998+0000 7f366ffff700 10 client.26479 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f366ffff700 1 -- 172.21.15.5:0/3205218194 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55beab5e4320 con 0x55beab6ce710 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f366ffff700 10 client.26479 handle_command_reply: tid=0 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 client.26479 shutdown 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 msgr2=0x55beab6be510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6be510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36840e0070 tx=0x7f3684010bb0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 shutdown_connections 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab6bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab536e40 0x55beab6be510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab6ce710 0x55beab6d0b30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/3205218194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f366ffff700 0 client.26479 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:05.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 >> 172.21.15.5:0/3205218194 conn(0x55beab57e630 msgr2=0x55beab6b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:05.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 shutdown_connections 2024-05-14T15:31:05.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 1 -- 172.21.15.5:0/3205218194 wait complete. 2024-05-14T15:31:05.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:04.999+0000 7f36a59cbb80 20 client.26479 trim_cache size 0 max 16384 2024-05-14T15:31:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 Processor -- start 2024-05-14T15:31:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 -- start start 2024-05-14T15:31:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 0x55beab6b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6bad80 0x55beab6bc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36901b2f70 con 0x55beab536e40 2024-05-14T15:31:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36901b0800 con 0x55beab6b4e10 2024-05-14T15:31:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f3695554700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6bad80 0x55beab6bc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f3694d53700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 0x55beab6b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f3694d53700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 0x55beab6b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54878/0 (socket says 172.21.15.5:54878) 2024-05-14T15:31:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.016+0000 7f3694d53700 1 -- 172.21.15.5:0/1442125194 learned_addr learned my addr 172.21.15.5:0/1442125194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f368ffff700 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f36a59cbb80 1 -- 172.21.15.5:0/1442125194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36901ae650 con 0x55beab6bad80 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f3694d53700 1 -- 172.21.15.5:0/1442125194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6bad80 msgr2=0x55beab6bc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f3694d53700 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6bad80 0x55beab6bc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f3694d53700 1 -- 172.21.15.5:0/1442125194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 msgr2=0x55beab6bd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f3694d53700 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f3694d53700 1 -- 172.21.15.5:0/1442125194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beab6aff40 con 0x55beab536e40 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.017+0000 7f368ffff700 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.018+0000 7f3694d53700 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 0x55beab6b48d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f368400e970 tx=0x7f36840a5b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a478b52d4fb329b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.018+0000 7f367a7fc700 1 -- 172.21.15.5:0/1442125194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368406d070 con 0x55beab536e40 2024-05-14T15:31:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.018+0000 7f367a7fc700 1 -- 172.21.15.5:0/1442125194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840cae50 con 0x55beab536e40 2024-05-14T15:31:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.018+0000 7f367a7fc700 1 -- 172.21.15.5:0/1442125194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368406f500 con 0x55beab536e40 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 -- 172.21.15.5:0/1442125194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 msgr2=0x55beab6b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 0x55beab6b48d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f368400e970 tx=0x7f36840a5b90 comp rx=0 tx=0).stop 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 -- 172.21.15.5:0/1442125194 shutdown_connections 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6bad80 0x55beab6bc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab536e40 0x55beab6b48d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/1442125194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab6b4e10 0x55beab6bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 -- 172.21.15.5:0/1442125194 >> 172.21.15.5:0/1442125194 conn(0x55beab57e630 msgr2=0x55beab6a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 -- 172.21.15.5:0/1442125194 shutdown_connections 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.019+0000 7f36a59cbb80 1 -- 172.21.15.5:0/1442125194 wait complete. 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 -- start start 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 Processor -- start 2024-05-14T15:31:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 0x55beab6d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6b4e10 0x55beab6d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab73b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36901aecc0 con 0x55beab6bad80 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36901b0130 con 0x55beab536e40 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36901af390 con 0x55beab6b4e10 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f36a59cbb80 10 client.0 _ll_register_callbacks cb 0x55beab56e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f3695554700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab73b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.020+0000 7f3694d53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 0x55beab6d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3695554700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab73b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54888/0 (socket says 172.21.15.5:54888) 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3695554700 1 -- 172.21.15.5:0/5806063 learned_addr learned my addr 172.21.15.5:0/5806063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 0x55beab6d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53872/0 (socket says 172.21.15.5:53872) 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f368ffff700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6b4e10 0x55beab6d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 -- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6b4e10 msgr2=0x55beab6d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6b4e10 0x55beab6d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 -- 172.21.15.5:0/5806063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 msgr2=0x55beab73b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab73b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 -- 172.21.15.5:0/5806063 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beab73b5a0 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f3694d53700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 0x55beab6d0b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55beab6b1090 tx=0x7f368406f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bce93e23f650bbe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f366ffff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f366ffff700 1 -- 172.21.15.5:0/5806063 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368406d070 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f366ffff700 1 -- 172.21.15.5:0/5806063 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f36840a9010 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36840cae50 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f366ffff700 1 -- 172.21.15.5:0/5806063 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36840a9310 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f36a59cbb80 10 client.26510 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.021+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55beab73b5a0 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f366ffff700 1 -- 172.21.15.5:0/5806063 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36840cae50 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f366ffff700 1 -- 172.21.15.5:0/5806063 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f36840af0e0 con 0x55beab536e40 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f36a59cbb80 10 client.26510 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f36a59cbb80 10 client.26510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:31:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f36a59cbb80 20 client.26510 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab73c870 0x55beab73ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f36a59cbb80 4 client.26510 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:31:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55beab5de3c0 con 0x55beab73c870 2024-05-14T15:31:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.022+0000 7f368ffff700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab73c870 0x55beab73ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f366ffff700 10 client.26510 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f368ffff700 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab73c870 0x55beab73ec90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a05ab4e4c4dc112 server_cookie=b65b204657de301f in_seq=0 out_seq=0 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f366ffff700 1 -- 172.21.15.5:0/5806063 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55beab5de3c0 con 0x55beab73c870 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f366ffff700 10 client.26510 handle_command_reply: tid=0 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 client.26510 shutdown 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 msgr2=0x55beab6d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 0x55beab6d0b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55beab6b1090 tx=0x7f368406f670 comp rx=0 tx=0).stop 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 shutdown_connections 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55beab6bad80 0x55beab73b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55beab73c870 0x55beab73ec90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55beab536e40 0x55beab6d0b90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.023+0000 7f36a59cbb80 1 --2- 172.21.15.5:0/5806063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55beab6b4e10 0x55beab6d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.024+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 >> 172.21.15.5:0/5806063 conn(0x55beab57e630 msgr2=0x55beab6a9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.024+0000 7f366ffff700 0 client.26510 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.024+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 shutdown_connections 2024-05-14T15:31:05.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.024+0000 7f36a59cbb80 1 -- 172.21.15.5:0/5806063 wait complete. 2024-05-14T15:31:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.024+0000 7f36a59cbb80 20 client.26510 trim_cache size 0 max 16384 2024-05-14T15:31:05.030 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:31:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36800b29f0 msgr2=0x7f36800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36800b29f0 0x7f36800b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f369006b510 tx=0x7f367c009260 comp rx=0 tx=0).stop 2024-05-14T15:31:05.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 msgr2=0x7f369019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 0x7f369019ae20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f368400e9a0 tx=0x7f368400ecb0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 shutdown_connections 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36800b29f0 0x7f36800b4f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3690108770 0x7f369019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3690103540 0x7f369019ae20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 --2- 172.21.15.5:0/3506173426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690107e20 0x7f369019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 >> 172.21.15.5:0/3506173426 conn(0x7f3690078e50 msgr2=0x7f3690075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 shutdown_connections 2024-05-14T15:31:05.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:05.025+0000 7f3696fb7700 1 -- 172.21.15.5:0/3506173426 wait complete. 2024-05-14T15:31:05.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563850 10.76 MB/sec execute 146 sec latency 31.262 ms 2024-05-14T15:31:05.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:31:06.142 INFO:tasks.workunit.client.0.smithi005.stdout: 1 565773 10.77 MB/sec execute 147 sec latency 16.942 ms 2024-05-14T15:31:06.194 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:05 smithi144 ceph-mon[35650]: pgmap v193: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-05-14T15:31:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:05 smithi193 ceph-mon[36001]: pgmap v193: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-05-14T15:31:06.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:05 smithi005 ceph-mon[31559]: pgmap v193: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 445 op/s 2024-05-14T15:31:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 -- 172.21.15.5:0/1257924831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750108bc0 msgr2=0x7f2750102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 --2- 172.21.15.5:0/1257924831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750108bc0 0x7f2750102760 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2740009900 tx=0x7f274002a410 comp rx=0 tx=0).stop 2024-05-14T15:31:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 -- 172.21.15.5:0/1257924831 shutdown_connections 2024-05-14T15:31:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 --2- 172.21.15.5:0/1257924831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750103310 0x7f275010f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 --2- 172.21.15.5:0/1257924831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 0x7f2750102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 --2- 172.21.15.5:0/1257924831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750108bc0 0x7f2750102760 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 -- 172.21.15.5:0/1257924831 >> 172.21.15.5:0/1257924831 conn(0x7f27500781b0 msgr2=0x7f2750078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 -- 172.21.15.5:0/1257924831 shutdown_connections 2024-05-14T15:31:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.953+0000 7f2755da8640 1 -- 172.21.15.5:0/1257924831 wait complete. 2024-05-14T15:31:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 Processor -- start 2024-05-14T15:31:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 -- start start 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750103310 0x7f27501a2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 0x7f27501a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 0x7f275019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2750112db0 con 0x7f2750103310 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2750112c30 con 0x7f2750109060 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2755da8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2750112f30 con 0x7f2750108bc0 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2747fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 0x7f27501a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750103310 0x7f27501a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f274ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 0x7f275019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2747fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 0x7f27501a2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35502/0 (socket says 172.21.15.5:35502) 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f274ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 0x7f275019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42370/0 (socket says 172.21.15.5:42370) 2024-05-14T15:31:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.954+0000 7f2747fff640 1 -- 172.21.15.5:0/3680093506 learned_addr learned my addr 172.21.15.5:0/3680093506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2747fff640 1 -- 172.21.15.5:0/3680093506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 msgr2=0x7f275019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2747fff640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 0x7f275019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2747fff640 1 -- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750103310 msgr2=0x7f27501a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2747fff640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750103310 0x7f27501a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2747fff640 1 -- 172.21.15.5:0/3680093506 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275019d010 con 0x7f2750108bc0 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2747fff640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 0x7f27501a2ca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f273800e9f0 tx=0x7f273800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f274d7fa640 1 -- 172.21.15.5:0/3680093506 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273800ccc0 con 0x7f2750108bc0 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275019d300 con 0x7f2750108bc0 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f274d7fa640 1 -- 172.21.15.5:0/3680093506 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27380035f0 con 0x7f2750108bc0 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f274d7fa640 1 -- 172.21.15.5:0/3680093506 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273800ce60 con 0x7f2750108bc0 2024-05-14T15:31:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.955+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501a95e0 con 0x7f2750108bc0 2024-05-14T15:31:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.958+0000 7f274d7fa640 1 -- 172.21.15.5:0/3680093506 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2738010690 con 0x7f2750108bc0 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.958+0000 7f275cc58740 1 Processor -- start 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 -- start start 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e19b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd94d0 0x56138e19b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56138e1005d0 con 0x56138ddd12f0 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56138e0b9890 con 0x56138ddd94d0 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f275cc58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56138e0fc590 con 0x56138ddfa1b0 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f274ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f274ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35514/0 (socket says 172.21.15.5:35514) 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f274ffff640 1 -- 172.21.15.5:0/856436498 learned_addr learned my addr 172.21.15.5:0/856436498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f2747fff640 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd94d0 0x56138e19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.959+0000 7f274d7fa640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f273c0b2950 0x7f273c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 -- 172.21.15.5:0/856436498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd94d0 msgr2=0x56138e19b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd94d0 0x56138e19b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 -- 172.21.15.5:0/856436498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 msgr2=0x56138e19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 -- 172.21.15.5:0/856436498 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138de12210 con 0x56138ddfa1b0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f273c0b2950 0x7f273c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274d7fa640 1 -- 172.21.15.5:0/3680093506 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f27380e2cd0 con 0x7f2750108bc0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f274ffff640 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e19abe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2748011590 tx=0x7f2748011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cec315539856289 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f2745ffb640 1 -- 172.21.15.5:0/856436498 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748017050 con 0x56138ddfa1b0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f2745ffb640 1 -- 172.21.15.5:0/856436498 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274800fcf0 con 0x56138ddfa1b0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.960+0000 7f2745ffb640 1 -- 172.21.15.5:0/856436498 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274801b9c0 con 0x56138ddfa1b0 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 -- 172.21.15.5:0/856436498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 msgr2=0x56138e19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e19abe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2748011590 tx=0x7f2748011a50 comp rx=0 tx=0).stop 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f274ffff640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f273c0b2950 0x7f273c0b4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2748022000 tx=0x7f2748023000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 -- 172.21.15.5:0/856436498 shutdown_connections 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd94d0 0x56138e19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 --2- 172.21.15.5:0/856436498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e19abe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 -- 172.21.15.5:0/856436498 >> 172.21.15.5:0/856436498 conn(0x56138ddee300 msgr2=0x56138e197340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 -- 172.21.15.5:0/856436498 shutdown_connections 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 -- 172.21.15.5:0/856436498 wait complete. 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.961+0000 7f275cc58740 1 -- start start 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 Processor -- start 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd12f0 0x56138e1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e1ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56138e1a7d10 con 0x56138ddd94d0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56138e0bd370 con 0x56138ddd12f0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56138e1a7e90 con 0x56138ddfa1b0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f275cc58740 10 client.0 _ll_register_callbacks cb 0x56138de11a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56426/0 (socket says 172.21.15.5:56426) 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 -- 172.21.15.5:0/928574991 learned_addr learned my addr 172.21.15.5:0/928574991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd12f0 0x56138e1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f274ffff640 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e1ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 -- 172.21.15.5:0/928574991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 msgr2=0x56138e1ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e1ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 -- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd12f0 msgr2=0x56138e1a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.962+0000 7f2747fff640 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd12f0 0x56138e1a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f2747fff640 1 -- 172.21.15.5:0/928574991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138dede620 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f2747fff640 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1a5750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f27380a9b20 tx=0x7f27380e62b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a31cbdfadba0247 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f272effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f272effd640 1 -- 172.21.15.5:0/928574991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27380a94e0 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f272effd640 1 -- 172.21.15.5:0/928574991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f27380b2560 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274800fcf0 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f272effd640 1 -- 172.21.15.5:0/928574991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f27380c3050 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f275cc58740 10 client.16896 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56138e1b4090 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.963+0000 7f272effd640 1 -- 172.21.15.5:0/928574991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274800fcf0 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f272effd640 1 -- 172.21.15.5:0/928574991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2738003fb0 con 0x56138ddd94d0 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f275cc58740 10 client.16896 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f275cc58740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f275cc58740 20 client.16896 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f275cc58740 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddb8d80 0x56138e1b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f275cc58740 4 client.16896 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56138e10e1e0 con 0x56138ddb8d80 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f274f7fe640 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddb8d80 0x56138e1b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f274f7fe640 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddb8d80 0x56138e1b7510 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd247d1cda0f2 server_cookie=c11903421de90da3 in_seq=0 out_seq=0 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.964+0000 7f272effd640 10 client.16896 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f272effd640 1 -- 172.21.15.5:0/928574991 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56138e10e1e0 con 0x56138ddb8d80 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f272effd640 10 client.16896 handle_command_reply: tid=0 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 client.16896 shutdown 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 msgr2=0x56138e1a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1a5750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f27380a9b20 tx=0x7f27380e62b0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 shutdown_connections 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddb8d80 0x56138e1b7510 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddfa1b0 0x56138e1ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1a5750 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f272effd640 0 client.16896 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 --2- 172.21.15.5:0/928574991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddd12f0 0x56138e1a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.965+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 >> 172.21.15.5:0/928574991 conn(0x56138ddee300 msgr2=0x56138e19aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.966+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 shutdown_connections 2024-05-14T15:31:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.966+0000 7f275cc58740 1 -- 172.21.15.5:0/928574991 wait complete. 2024-05-14T15:31:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.966+0000 7f275cc58740 20 client.16896 trim_cache size 0 max 16384 2024-05-14T15:31:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.983+0000 7f275cc58740 1 Processor -- start 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.983+0000 7f275cc58740 1 -- start start 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f275cc58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e197410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f275cc58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd12f0 0x56138e1ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f275cc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1ab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f275cc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b5f20 con 0x56138ddd94d0 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f275cc58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b37f0 con 0x56138ddee300 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f275cc58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b1640 con 0x56138ddd12f0 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56442/0 (socket says 172.21.15.5:56442) 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 -- 172.21.15.5:0/1456703813 learned_addr learned my addr 172.21.15.5:0/1456703813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f2747fff640 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd12f0 0x56138e1ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 -- 172.21.15.5:0/1456703813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd12f0 msgr2=0x56138e1ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd12f0 0x56138e1ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 -- 172.21.15.5:0/1456703813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 msgr2=0x56138e197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 -- 172.21.15.5:0/1456703813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e19b990 con 0x56138ddd94d0 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274f7fe640 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f2747fff640 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd12f0 0x56138e1ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.984+0000 7f274ffff640 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1ab640 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2748011560 tx=0x7f2748009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2757b5bb4449c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.985+0000 7f27457fa640 1 -- 172.21.15.5:0/1456703813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748036070 con 0x56138ddd94d0 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.985+0000 7f27457fa640 1 -- 172.21.15.5:0/1456703813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274801b430 con 0x56138ddd94d0 2024-05-14T15:31:06.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.985+0000 7f27457fa640 1 -- 172.21.15.5:0/1456703813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748004e00 con 0x56138ddd94d0 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 -- 172.21.15.5:0/1456703813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 msgr2=0x56138e1ab640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1ab640 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2748011560 tx=0x7f2748009b50 comp rx=0 tx=0).stop 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 -- 172.21.15.5:0/1456703813 shutdown_connections 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd12f0 0x56138e1ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd94d0 0x56138e1ab640 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 --2- 172.21.15.5:0/1456703813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 -- 172.21.15.5:0/1456703813 >> 172.21.15.5:0/1456703813 conn(0x56138e1a58f0 msgr2=0x56138e196d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 -- 172.21.15.5:0/1456703813 shutdown_connections 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 -- 172.21.15.5:0/1456703813 wait complete. 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 -- start start 2024-05-14T15:31:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.986+0000 7f275cc58740 1 Processor -- start 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e1b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd94d0 0x56138e1acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e1ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b1cb0 con 0x56138ddd12f0 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b3120 con 0x56138ddee300 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b2380 con 0x56138ddd94d0 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 10 client.0 _ll_register_callbacks cb 0x7f27380e6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e1b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e1b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56458/0 (socket says 172.21.15.5:56458) 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 -- 172.21.15.5:0/877344171 learned_addr learned my addr 172.21.15.5:0/877344171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f2747fff640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd94d0 0x56138e1acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274ffff640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e1ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 -- 172.21.15.5:0/877344171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd94d0 msgr2=0x56138e1acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd94d0 0x56138e1acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 -- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 msgr2=0x56138e1ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e1ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 -- 172.21.15.5:0/877344171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138e1adb00 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f274f7fe640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e1b1c00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2740005940 tx=0x7f274002abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5cde9444bec188 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f272effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f272effd640 1 -- 172.21.15.5:0/877344171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740035040 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f272effd640 1 -- 172.21.15.5:0/877344171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2740005040 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.987+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274801b430 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f272effd640 1 -- 172.21.15.5:0/877344171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2740055050 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 10 client.16902 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56138e21f6a0 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f272effd640 1 -- 172.21.15.5:0/877344171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274801b430 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f272effd640 1 -- 172.21.15.5:0/877344171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2740046320 con 0x56138ddd12f0 2024-05-14T15:31:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 10 client.16902 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 20 client.16902 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddfa1b0 0x56138e221e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 4 client.16902 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56138e108050 con 0x56138ddfa1b0 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.988+0000 7f2747fff640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddfa1b0 0x56138e221e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.989+0000 7f272effd640 10 client.16902 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:06.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:06.989+0000 7f2747fff640 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddfa1b0 0x56138e221e60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2010d9dfe83055 server_cookie=9382b5310682d604 in_seq=0 out_seq=0 2024-05-14T15:31:07.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 568172 10.79 MB/sec execute 148 sec latency 60.445 ms 2024-05-14T15:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.263+0000 7f272effd640 1 -- 172.21.15.5:0/877344171 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56138e108050 con 0x56138ddfa1b0 2024-05-14T15:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.263+0000 7f272effd640 10 client.16902 handle_command_reply: tid=0 2024-05-14T15:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.263+0000 7f275cc58740 1 client.16902 shutdown 2024-05-14T15:31:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.263+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 msgr2=0x56138e1b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.263+0000 7f275cc58740 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e1b1c00 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2740005940 tx=0x7f274002abc0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 shutdown_connections 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56138ddfa1b0 0x56138e221e60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f272effd640 0 client.16902 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138ddd94d0 0x56138e1acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56138ddee300 0x56138e1ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 --2- 172.21.15.5:0/877344171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56138ddd12f0 0x56138e1b1c00 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 >> 172.21.15.5:0/877344171 conn(0x56138e1a58f0 msgr2=0x56138e1a38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 shutdown_connections 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.264+0000 7f275cc58740 1 -- 172.21.15.5:0/877344171 wait complete. 2024-05-14T15:31:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.265+0000 7f275cc58740 20 client.16902 trim_cache size 0 max 16384 2024-05-14T15:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.265+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f273c0b2950 msgr2=0x7f273c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.265+0000 7f2755da8640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f273c0b2950 0x7f273c0b4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2748022000 tx=0x7f2748023000 comp rx=0 tx=0).stop 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.265+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 msgr2=0x7f27501a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.265+0000 7f2755da8640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 0x7f27501a2ca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f273800e9f0 tx=0x7f273800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 shutdown_connections 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f273c0b2950 0x7f273c0b4e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2750108bc0 0x7f27501a2ca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2750109060 0x7f275019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 --2- 172.21.15.5:0/3680093506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2750103310 0x7f27501a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 >> 172.21.15.5:0/3680093506 conn(0x7f27500781b0 msgr2=0x7f275006b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 shutdown_connections 2024-05-14T15:31:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:07.266+0000 7f2755da8640 1 -- 172.21.15.5:0/3680093506 wait complete. 2024-05-14T15:31:07.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:07 smithi005 ceph-mon[31559]: pgmap v194: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 443 op/s 2024-05-14T15:31:08.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 569084 10.75 MB/sec execute 149 sec latency 115.019 ms 2024-05-14T15:31:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 -- 172.21.15.5:0/3185755802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 msgr2=0x7f68d0105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 --2- 172.21.15.5:0/3185755802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 0x7f68d0105630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009990 tx=0x7f68c0009ca0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 -- 172.21.15.5:0/3185755802 shutdown_connections 2024-05-14T15:31:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 --2- 172.21.15.5:0/3185755802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0103390 0x7f68d0113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 --2- 172.21.15.5:0/3185755802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 0x7f68d0105630 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 --2- 172.21.15.5:0/3185755802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0102910 0x7f68d0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 -- 172.21.15.5:0/3185755802 >> 172.21.15.5:0/3185755802 conn(0x7f68d00fe6f0 msgr2=0x7f68d0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 -- 172.21.15.5:0/3185755802 shutdown_connections 2024-05-14T15:31:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.266+0000 7f68d68dc700 1 -- 172.21.15.5:0/3185755802 wait complete. 2024-05-14T15:31:08.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.267+0000 7f68d68dc700 1 Processor -- start 2024-05-14T15:31:08.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.267+0000 7f68d68dc700 1 -- start start 2024-05-14T15:31:08.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d68dc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 0x7f68d019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d68dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0103390 0x7f68d019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d68dc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 0x7f68d01a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d68dc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0116700 con 0x7f68d0105220 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d68dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0116580 con 0x7f68d0103390 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d68dc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0116880 con 0x7f68d0102910 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 0x7f68d019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 0x7f68d019d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35560/0 (socket says 172.21.15.5:35560) 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 -- 172.21.15.5:0/2394919572 learned_addr learned my addr 172.21.15.5:0/2394919572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0103390 0x7f68d019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68d4e79700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 0x7f68d01a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 -- 172.21.15.5:0/2394919572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0103390 msgr2=0x7f68d019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0103390 0x7f68d019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 -- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 msgr2=0x7f68d01a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 0x7f68d01a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.268+0000 7f68cffff700 1 -- 172.21.15.5:0/2394919572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d01a1fc0 con 0x7f68d0102910 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.269+0000 7f68cffff700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 0x7f68d019d0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68c0000c00 tx=0x7f68c00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.269+0000 7f68cd7fa700 1 -- 172.21.15.5:0/2394919572 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0026070 con 0x7f68d0102910 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.269+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d01a2250 con 0x7f68d0102910 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.269+0000 7f68cd7fa700 1 -- 172.21.15.5:0/2394919572 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c0021410 con 0x7f68d0102910 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.269+0000 7f68cd7fa700 1 -- 172.21.15.5:0/2394919572 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0017650 con 0x7f68d0102910 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.269+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d01a2730 con 0x7f68d0102910 2024-05-14T15:31:08.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.272+0000 7f68cd7fa700 1 -- 172.21.15.5:0/2394919572 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f68c0029070 con 0x7f68d0102910 2024-05-14T15:31:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.272+0000 7f68cd7fa700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f68b80b29f0 0x7f68b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.272+0000 7f68e52f0b80 1 Processor -- start 2024-05-14T15:31:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 -- start start 2024-05-14T15:31:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68cd7fa700 1 -- 172.21.15.5:0/2394919572 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f68c00e38c0 con 0x7f68d0102910 2024-05-14T15:31:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f68b80b29f0 0x7f68b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a24840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a24d80 0x5556f0a1e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 0x5556f0a1f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556f0940940 con 0x5556f0a1ee10 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556f08a5da0 con 0x5556f0a24d80 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5556f093c900 con 0x5556f08a0e40 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68d4e79700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 0x5556f0a1f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68d4e79700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 0x5556f0a1f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56484/0 (socket says 172.21.15.5:56484) 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68d4e79700 1 -- 172.21.15.5:0/4046663394 learned_addr learned my addr 172.21.15.5:0/4046663394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68cffff700 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a24840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.273+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a24d80 0x5556f0a1e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f68b80b29f0 0x7f68b80b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68d01047e0 tx=0x7f68c4009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cffff700 1 -- 172.21.15.5:0/4046663394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a24d80 msgr2=0x5556f0a1e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cffff700 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a24d80 0x5556f0a1e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cffff700 1 -- 172.21.15.5:0/4046663394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 msgr2=0x5556f0a1f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cffff700 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 0x5556f0a1f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cffff700 1 -- 172.21.15.5:0/4046663394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f088d8d0 con 0x5556f08a0e40 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68d4e79700 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 0x5556f0a1f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68cffff700 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a24840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68c00e0c50 tx=0x7f68c00aa770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32d1d2738f58a750 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68b67fc700 1 -- 172.21.15.5:0/4046663394 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00aec20 con 0x5556f08a0e40 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68b67fc700 1 -- 172.21.15.5:0/4046663394 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c00b3640 con 0x5556f08a0e40 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.274+0000 7f68b67fc700 1 -- 172.21.15.5:0/4046663394 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00aaad0 con 0x5556f08a0e40 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 -- 172.21.15.5:0/4046663394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 msgr2=0x5556f0a24840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a24840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68c00e0c50 tx=0x7f68c00aa770 comp rx=0 tx=0).stop 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 -- 172.21.15.5:0/4046663394 shutdown_connections 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a1ee10 0x5556f0a1f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a24840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/4046663394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a24d80 0x5556f0a1e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 -- 172.21.15.5:0/4046663394 >> 172.21.15.5:0/4046663394 conn(0x5556f08e8630 msgr2=0x5556f0a19a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 -- 172.21.15.5:0/4046663394 shutdown_connections 2024-05-14T15:31:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 -- 172.21.15.5:0/4046663394 wait complete. 2024-05-14T15:31:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.275+0000 7f68e52f0b80 1 -- start start 2024-05-14T15:31:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68e52f0b80 1 Processor -- start 2024-05-14T15:31:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a28600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a28b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a28600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a28600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56496/0 (socket says 172.21.15.5:56496) 2024-05-14T15:31:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68cffff700 1 -- 172.21.15.5:0/1643094776 learned_addr learned my addr 172.21.15.5:0/1643094776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556f0a2b5e0 con 0x5556f08a0e40 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.276+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a28b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556f08f9ed0 con 0x5556f0a1ee10 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68d4e79700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5556f0a2b760 con 0x5556f0a24d80 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68e52f0b80 10 client.0 _ll_register_callbacks cb 0x5556f08d8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68cffff700 1 -- 172.21.15.5:0/1643094776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 msgr2=0x5556f0a29080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68cffff700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a29080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68cffff700 1 -- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 msgr2=0x5556f0a28b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68cffff700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a28b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68cffff700 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f08f7760 con 0x5556f08a0e40 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68cffff700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a28600 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f68c00e11d0 tx=0x7f68c0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ff3ebb56e47d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.277+0000 7f68af7fe700 1 -- 172.21.15.5:0/1643094776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00aec20 con 0x5556f08a0e40 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68af7fe700 1 -- 172.21.15.5:0/1643094776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f68c00218c0 con 0x5556f08a0e40 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c00b3640 con 0x5556f08a0e40 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68af7fe700 1 -- 172.21.15.5:0/1643094776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68c00c7020 con 0x5556f08a0e40 2024-05-14T15:31:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 10 client.16914 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5556f0a2b5e0 con 0x5556f08a0e40 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68af7fe700 1 -- 172.21.15.5:0/1643094776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c00b3640 con 0x5556f08a0e40 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68af7fe700 1 -- 172.21.15.5:0/1643094776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f68c00cb070 con 0x5556f08a0e40 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 10 client.16914 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 20 client.16914 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0a43290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 4 client.16914 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556f094e320 con 0x5556f0a40dc0 2024-05-14T15:31:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.278+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0a43290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.279+0000 7f68af7fe700 10 client.16914 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.279+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0a43290 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51a745d133157bc4 server_cookie=8d739dbd0d25ac45 in_seq=0 out_seq=0 2024-05-14T15:31:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68af7fe700 1 -- 172.21.15.5:0/1643094776 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556f094e320 con 0x5556f0a40dc0 2024-05-14T15:31:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68af7fe700 10 client.16914 handle_command_reply: tid=0 2024-05-14T15:31:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 client.16914 shutdown 2024-05-14T15:31:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 msgr2=0x5556f0a28600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a28600 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f68c00e11d0 tx=0x7f68c0009ca0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 shutdown_connections 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0a43290 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68af7fe700 0 client.16914 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a29080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a28600 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1643094776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a28b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.280+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 >> 172.21.15.5:0/1643094776 conn(0x5556f08e8630 msgr2=0x5556f0a1d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.281+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 shutdown_connections 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.281+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1643094776 wait complete. 2024-05-14T15:31:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.281+0000 7f68e52f0b80 20 client.16914 trim_cache size 0 max 16384 2024-05-14T15:31:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.296+0000 7f68e52f0b80 1 Processor -- start 2024-05-14T15:31:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.296+0000 7f68e52f0b80 1 -- start start 2024-05-14T15:31:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:07 smithi144 ceph-mon[35650]: pgmap v194: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 443 op/s 2024-05-14T15:31:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.296+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a1e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.296+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.296+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a24d80 0x5556f0a28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01b0c00 con 0x5556f0a24d80 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ae490 con 0x5556f0a1ee10 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ac2e0 con 0x5556f08a0e40 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a1e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a1e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35586/0 (socket says 172.21.15.5:35586) 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cffff700 1 -- 172.21.15.5:0/177863229 learned_addr learned my addr 172.21.15.5:0/177863229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a261a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42436/0 (socket says 172.21.15.5:42436) 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68d4e79700 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a24d80 0x5556f0a28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 -- 172.21.15.5:0/177863229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 msgr2=0x5556f0a1e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a1e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 -- 172.21.15.5:0/177863229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a24d80 msgr2=0x5556f0a28ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a24d80 0x5556f0a28ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 -- 172.21.15.5:0/177863229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f0a29400 con 0x5556f0a1ee10 2024-05-14T15:31:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cffff700 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a1e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.297+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a261a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5556f0a15b90 tx=0x7f68c4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f731816dcbe7de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.298+0000 7f68b5ffb700 1 -- 172.21.15.5:0/177863229 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4004020 con 0x5556f0a1ee10 2024-05-14T15:31:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.298+0000 7f68b5ffb700 1 -- 172.21.15.5:0/177863229 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c4003680 con 0x5556f0a1ee10 2024-05-14T15:31:08.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.298+0000 7f68b5ffb700 1 -- 172.21.15.5:0/177863229 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c401f640 con 0x5556f0a1ee10 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.298+0000 7f68e52f0b80 1 -- 172.21.15.5:0/177863229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 msgr2=0x5556f0a261a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.298+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a261a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5556f0a15b90 tx=0x7f68c4014000 comp rx=0 tx=0).stop 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 -- 172.21.15.5:0/177863229 shutdown_connections 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f0a24d80 0x5556f0a28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f08a0e40 0x5556f0a1e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/177863229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a261a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 -- 172.21.15.5:0/177863229 >> 172.21.15.5:0/177863229 conn(0x5556f08e8630 msgr2=0x5556f0a19270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 -- 172.21.15.5:0/177863229 shutdown_connections 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 -- 172.21.15.5:0/177863229 wait complete. 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 -- start start 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.299+0000 7f68e52f0b80 1 Processor -- start 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a47a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ac950 con 0x5556f08a0e40 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01addc0 con 0x5556f0a1ee10 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01ad020 con 0x5556f0a24d80 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68e52f0b80 10 client.0 _ll_register_callbacks cb 0x5556f08d8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a47a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56510/0 (socket says 172.21.15.5:56510) 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68cffff700 1 -- 172.21.15.5:0/1930890079 learned_addr learned my addr 172.21.15.5:0/1930890079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 -- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 msgr2=0x5556f0a39700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a39700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 -- 172.21.15.5:0/1930890079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 msgr2=0x5556f0a391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 -- 172.21.15.5:0/1930890079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f0a47f80 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68cf7fe700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a39700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68d4e79700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a47a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68c8011e80 tx=0x7f68c80077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e1dad39046d821 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.300+0000 7f68af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68af7fe700 1 -- 172.21.15.5:0/1930890079 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8004bb0 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68af7fe700 1 -- 172.21.15.5:0/1930890079 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f68c8004d50 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c4003680 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68af7fe700 1 -- 172.21.15.5:0/1930890079 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68c802b050 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 10 client.26521 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5556f0aaacf0 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68af7fe700 1 -- 172.21.15.5:0/1930890079 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c4003680 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68af7fe700 1 -- 172.21.15.5:0/1930890079 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f68c8016400 con 0x5556f0a24d80 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 10 client.26521 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 10 client.26521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:31:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 20 client.26521 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.301+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0aad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68e52f0b80 4 client.26521 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556f09483c0 con 0x5556f0a40dc0 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68cffff700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0aad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68af7fe700 10 client.26521 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68cffff700 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0aad4f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12e561b0574be38e server_cookie=747b3ec8bcc2c0d4 in_seq=0 out_seq=0 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68af7fe700 1 -- 172.21.15.5:0/1930890079 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556f09483c0 con 0x5556f0a40dc0 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68af7fe700 10 client.26521 handle_command_reply: tid=0 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.302+0000 7f68e52f0b80 1 client.26521 shutdown 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 msgr2=0x5556f0a47a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a47a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68c8011e80 tx=0x7f68c80077e0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 shutdown_connections 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5556f0a40dc0 0x5556f0aad4f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556f0a24d80 0x5556f0a47a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f08a0e40 0x5556f0a391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 --2- 172.21.15.5:0/1930890079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556f0a1ee10 0x5556f0a39700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68af7fe700 0 client.26521 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 >> 172.21.15.5:0/1930890079 conn(0x5556f08e8630 msgr2=0x5556f0a29bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:08.311 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:31:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 shutdown_connections 2024-05-14T15:31:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 1 -- 172.21.15.5:0/1930890079 wait complete. 2024-05-14T15:31:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68e52f0b80 20 client.26521 trim_cache size 0 max 16384 2024-05-14T15:31:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f68b80b29f0 msgr2=0x7f68b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68d68dc700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f68b80b29f0 0x7f68b80b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68d01047e0 tx=0x7f68c4009260 comp rx=0 tx=0).stop 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 msgr2=0x7f68d019d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.303+0000 7f68d68dc700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 0x7f68d019d0f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68c0000c00 tx=0x7f68c00047b0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 shutdown_connections 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f68b80b29f0 0x7f68b80b4f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68d0105220 0x7f68d01a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68d0102910 0x7f68d019d0f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 --2- 172.21.15.5:0/2394919572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d0103390 0x7f68d019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 >> 172.21.15.5:0/2394919572 conn(0x7f68d00fe6f0 msgr2=0x7f68d00fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 shutdown_connections 2024-05-14T15:31:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:08.304+0000 7f68d68dc700 1 -- 172.21.15.5:0/2394919572 wait complete. 2024-05-14T15:31:08.314 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:07 smithi193 ceph-mon[36001]: pgmap v194: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 443 op/s 2024-05-14T15:31:09.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:31:09.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 570170 10.71 MB/sec execute 150 sec latency 20.959 ms 2024-05-14T15:31:09.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:09 smithi005 ceph-mon[31559]: pgmap v195: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 366 op/s 2024-05-14T15:31:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:09.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:09 smithi144 ceph-mon[35650]: pgmap v195: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 366 op/s 2024-05-14T15:31:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:09 smithi193 ceph-mon[36001]: pgmap v195: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 366 op/s 2024-05-14T15:31:10.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 572362 10.73 MB/sec execute 151 sec latency 16.727 ms 2024-05-14T15:31:10.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:10 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:31:11.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:10 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:31:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:10 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:31:11.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 574295 10.71 MB/sec execute 152 sec latency 28.540 ms 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.158+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/3212462132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f28107df0 msgr2=0x7f5f28108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/3212462132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f28107df0 0x7f5f28108200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f180099c0 tx=0x7f5f18009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/3212462132 shutdown_connections 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/3212462132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f28108740 0x7f5f281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/3212462132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f281034c0 0x7f5f281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/3212462132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f28107df0 0x7f5f28108200 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/3212462132 >> 172.21.15.5:0/3212462132 conn(0x7f5f280fedc0 msgr2=0x7f5f281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:11.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/3212462132 shutdown_connections 2024-05-14T15:31:11.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.159+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/3212462132 wait complete. 2024-05-14T15:31:11.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 Processor -- start 2024-05-14T15:31:11.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 -- start start 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 0x7f5f2819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f28107df0 0x7f5f2819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f28108740 0x7f5f2819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28113350 con 0x7f5f281034c0 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281131d0 con 0x7f5f28107df0 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2f9e7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281134d0 con 0x7f5f28108740 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2d783700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 0x7f5f2819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2cf82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f28107df0 0x7f5f2819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f28108740 0x7f5f2819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2d783700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 0x7f5f2819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56536/0 (socket says 172.21.15.5:56536) 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.160+0000 7f5f2d783700 1 -- 172.21.15.5:0/2099261518 learned_addr learned my addr 172.21.15.5:0/2099261518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2d783700 1 -- 172.21.15.5:0/2099261518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f28108740 msgr2=0x7f5f2819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2d783700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f28108740 0x7f5f2819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2d783700 1 -- 172.21.15.5:0/2099261518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f28107df0 msgr2=0x7f5f2819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2d783700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f28107df0 0x7f5f2819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2d783700 1 -- 172.21.15.5:0/2099261518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f281a6fe0 con 0x7f5f281034c0 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f28107df0 0x7f5f2819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2d783700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 0x7f5f2819bc90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400e9a0 tx=0x7f5f2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f1e7fc700 1 -- 172.21.15.5:0/2099261518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400ca10 con 0x7f5f281034c0 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f281a72d0 con 0x7f5f281034c0 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.161+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281a7830 con 0x7f5f281034c0 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.162+0000 7f5f1e7fc700 1 -- 172.21.15.5:0/2099261518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2400cbb0 con 0x7f5f281034c0 2024-05-14T15:31:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.162+0000 7f5f1e7fc700 1 -- 172.21.15.5:0/2099261518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24018680 con 0x7f5f281034c0 2024-05-14T15:31:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.165+0000 7f5f1e7fc700 1 -- 172.21.15.5:0/2099261518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5f24018820 con 0x7f5f281034c0 2024-05-14T15:31:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 Processor -- start 2024-05-14T15:31:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 -- start start 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587211e40 0x55e58738f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e587393f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f1e7fc700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f140b2a70 0x7f5f140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587394880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e5872b1860 con 0x55e58738fc60 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e5872507c0 con 0x55e587394470 2024-05-14T15:31:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e5872ad820 con 0x55e587211e40 2024-05-14T15:31:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.166+0000 7f5f2d783700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f140b2a70 0x7f5f140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e587393f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f1e7fc700 1 -- 172.21.15.5:0/2099261518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5f240e2a30 con 0x7f5f281034c0 2024-05-14T15:31:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e587393f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56538/0 (socket says 172.21.15.5:56538) 2024-05-14T15:31:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 -- 172.21.15.5:0/540550960 learned_addr learned my addr 172.21.15.5:0/540550960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2d783700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f140b2a70 0x7f5f140b4f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f24019000 tx=0x7f5f240d8f60 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2df84700 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587394880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2df84700 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587211e40 0x55e58738f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 -- 172.21.15.5:0/540550960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587211e40 msgr2=0x55e58738f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587211e40 0x55e58738f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 -- 172.21.15.5:0/540550960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 msgr2=0x55e587394880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587394880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2cf82700 1 -- 172.21.15.5:0/540550960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5871fe8d0 con 0x55e58738fc60 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.167+0000 7f5f2df84700 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587211e40 0x55e58738f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:31:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.168+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e587393f30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5f180099c0 tx=0x7f5f18005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a18341b4de3e65 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.168+0000 7f5f137fe700 1 -- 172.21.15.5:0/540550960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18027070 con 0x55e58738fc60 2024-05-14T15:31:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.168+0000 7f5f137fe700 1 -- 172.21.15.5:0/540550960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18005550 con 0x55e58738fc60 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.168+0000 7f5f137fe700 1 -- 172.21.15.5:0/540550960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003db0 con 0x55e58738fc60 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.169+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/540550960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 msgr2=0x55e587393f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.169+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e587393f30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5f180099c0 tx=0x7f5f18005fb0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/540550960 shutdown_connections 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587394880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587211e40 0x55e58738f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/540550960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e587393f30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/540550960 >> 172.21.15.5:0/540550960 conn(0x55e587259600 msgr2=0x55e58738a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/540550960 shutdown_connections 2024-05-14T15:31:11.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/540550960 wait complete. 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 -- start start 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.170+0000 7f5f3e3fbb80 1 Processor -- start 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 0x55e587399130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e587399670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587399bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e58739c3c0 con 0x55e587211e40 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e587217c30 con 0x55e587394470 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e58739c540 con 0x55e58738fc60 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f3e3fbb80 10 client.0 _ll_register_callbacks cb 0x55e587249920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 0x55e587399130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 0x55e587399130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56554/0 (socket says 172.21.15.5:56554) 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f2df84700 1 -- 172.21.15.5:0/2817549259 learned_addr learned my addr 172.21.15.5:0/2817549259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:11.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e587399670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.171+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587399bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2df84700 1 -- 172.21.15.5:0/2817549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 msgr2=0x55e587399670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e587399670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2df84700 1 -- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 msgr2=0x55e587399bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587399bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2df84700 1 -- 172.21.15.5:0/2817549259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58724d690 con 0x55e587211e40 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e587399670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 0x55e587399130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001c820 tx=0x7f5f20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e4eafdc46cf204 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2817549259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004bb0 con 0x55e587211e40 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f18005550 con 0x55e587211e40 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2817549259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5f20004d50 con 0x55e587211e40 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2817549259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f20028050 con 0x55e587211e40 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f3e3fbb80 10 client.16932 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:11.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.172+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5873a6fc0 con 0x55e587211e40 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2817549259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18005550 con 0x55e587211e40 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2817549259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5f20018b60 con 0x55e587211e40 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f3e3fbb80 10 client.16932 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f3e3fbb80 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f3e3fbb80 20 client.16932 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e5873ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f3e3fbb80 4 client.16932 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5872bf240 con 0x55e5873a9550 2024-05-14T15:31:11.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.173+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e5873ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.174+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e5873ab9f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94533d08784d61a5 server_cookie=43881377274cd144 in_seq=0 out_seq=0 2024-05-14T15:31:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.174+0000 7f5f10ff9700 10 client.16932 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.174+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2817549259 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5872bf240 con 0x55e5873a9550 2024-05-14T15:31:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.174+0000 7f5f10ff9700 10 client.16932 handle_command_reply: tid=0 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.174+0000 7f5f3e3fbb80 1 client.16932 shutdown 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 msgr2=0x55e587399130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 0x55e587399130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001c820 tx=0x7f5f20004ad0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 shutdown_connections 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e5873ab9f0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587394470 0x55e587399bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f10ff9700 0 client.16932 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587211e40 0x55e587399130 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2817549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e587399670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 >> 172.21.15.5:0/2817549259 conn(0x55e587259600 msgr2=0x55e58738e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 shutdown_connections 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2817549259 wait complete. 2024-05-14T15:31:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.175+0000 7f5f3e3fbb80 20 client.16932 trim_cache size 0 max 16384 2024-05-14T15:31:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 Processor -- start 2024-05-14T15:31:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 -- start start 2024-05-14T15:31:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e58738a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587394470 0x55e5873a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c011660 con 0x55e587394470 2024-05-14T15:31:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c00eef0 con 0x55e587211e40 2024-05-14T15:31:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.192+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c00cd40 con 0x55e58738fc60 2024-05-14T15:31:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42476/0 (socket says 172.21.15.5:42476) 2024-05-14T15:31:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 -- 172.21.15.5:0/1341427460 learned_addr learned my addr 172.21.15.5:0/1341427460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587394470 0x55e5873a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e58738a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 -- 172.21.15.5:0/1341427460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 msgr2=0x55e58738a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e58738a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 -- 172.21.15.5:0/1341427460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587394470 msgr2=0x55e5873a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587394470 0x55e5873a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 -- 172.21.15.5:0/1341427460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e587398450 con 0x55e587211e40 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e58738a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.193+0000 7f5f2df84700 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873999f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001cfd0 tx=0x7f5f2000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef1d9ef547047c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.194+0000 7f5f12ffd700 1 -- 172.21.15.5:0/1341427460 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200098a0 con 0x55e587211e40 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.194+0000 7f5f12ffd700 1 -- 172.21.15.5:0/1341427460 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f20005c60 con 0x55e587211e40 2024-05-14T15:31:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.194+0000 7f5f12ffd700 1 -- 172.21.15.5:0/1341427460 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20003d80 con 0x55e587211e40 2024-05-14T15:31:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/1341427460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 msgr2=0x55e5873999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873999f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001cfd0 tx=0x7f5f2000b040 comp rx=0 tx=0).stop 2024-05-14T15:31:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/1341427460 shutdown_connections 2024-05-14T15:31:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e587394470 0x55e5873a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873999f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/1341427460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e58738fc60 0x55e58738a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/1341427460 >> 172.21.15.5:0/1341427460 conn(0x55e587259600 msgr2=0x55e587389be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/1341427460 shutdown_connections 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/1341427460 wait complete. 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.195+0000 7f5f3e3fbb80 1 -- start start 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 Processor -- start 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e5873a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587394470 0x55e587395820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c00d3b0 con 0x55e58738fc60 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c00e820 con 0x55e587211e40 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c00da80 con 0x55e587394470 2024-05-14T15:31:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f3e3fbb80 10 client.0 _ll_register_callbacks cb 0x55e587249920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f2cf82700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e5873a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f2df84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42480/0 (socket says 172.21.15.5:42480) 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f2df84700 1 -- 172.21.15.5:0/2701787828 learned_addr learned my addr 172.21.15.5:0/2701787828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.196+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587394470 0x55e587395820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2cf82700 1 -- 172.21.15.5:0/2701787828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587394470 msgr2=0x55e587395820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587394470 0x55e587395820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2cf82700 1 -- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 msgr2=0x55e5873a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2cf82700 1 -- 172.21.15.5:0/2701787828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e587395f10 con 0x55e58738fc60 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f2cf82700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e5873a1d70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005a80 tx=0x7f5f18009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c610261d84c2ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2701787828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18027040 con 0x55e58738fc60 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2701787828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5f18003a00 con 0x55e58738fc60 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f20005c60 con 0x55e58738fc60 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.197+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2701787828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5f18005540 con 0x55e58738fc60 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 10 client.16938 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e587413590 con 0x55e58738fc60 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2701787828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f20005c60 con 0x55e58738fc60 2024-05-14T15:31:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2701787828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f5f18021070 con 0x55e58738fc60 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 10 client.16938 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 20 client.16938 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e587415880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 4 client.16938 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5872b92e0 con 0x55e5873a9550 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.198+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e587415880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.199+0000 7f5f10ff9700 10 client.16938 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.199+0000 7f5f2df84700 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e587415880 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c4350245d8fad9 server_cookie=29f3c3e7ddb70765 in_seq=0 out_seq=0 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.200+0000 7f5f10ff9700 1 -- 172.21.15.5:0/2701787828 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5872b92e0 con 0x55e5873a9550 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.200+0000 7f5f10ff9700 10 client.16938 handle_command_reply: tid=0 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.200+0000 7f5f3e3fbb80 1 client.16938 shutdown 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.200+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 msgr2=0x55e5873a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.200+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e5873a1d70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005a80 tx=0x7f5f18009310 comp rx=0 tx=0).stop 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 shutdown_connections 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e5873a9550 0x55e587415880 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e587394470 0x55e587395820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e587211e40 0x55e5873a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 --2- 172.21.15.5:0/2701787828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e58738fc60 0x55e5873a1d70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f10ff9700 0 client.16938 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 >> 172.21.15.5:0/2701787828 conn(0x55e587259600 msgr2=0x55e58738e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 shutdown_connections 2024-05-14T15:31:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 1 -- 172.21.15.5:0/2701787828 wait complete. 2024-05-14T15:31:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.201+0000 7f5f3e3fbb80 20 client.16938 trim_cache size 0 max 16384 2024-05-14T15:31:11.205 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f140b2a70 msgr2=0x7f5f140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f140b2a70 0x7f5f140b4f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f24019000 tx=0x7f5f240d8f60 comp rx=0 tx=0).stop 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 msgr2=0x7f5f2819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 0x7f5f2819bc90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400e9a0 tx=0x7f5f2400ecb0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 shutdown_connections 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5f140b2a70 0x7f5f140b4f80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f28108740 0x7f5f2819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f281034c0 0x7f5f2819bc90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 --2- 172.21.15.5:0/2099261518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f28107df0 0x7f5f2819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 >> 172.21.15.5:0/2099261518 conn(0x7f5f280fedc0 msgr2=0x7f5f281096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 shutdown_connections 2024-05-14T15:31:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:11.202+0000 7f5f2f9e7700 1 -- 172.21.15.5:0/2099261518 wait complete. 2024-05-14T15:31:11.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:11 smithi005 ceph-mon[31559]: pgmap v196: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-05-14T15:31:11.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:11 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:31:11.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:11 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:31:11.816 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:11 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:31:12.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:11 smithi144 ceph-mon[35650]: pgmap v196: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-05-14T15:31:12.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:11 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:31:12.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:11 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:31:12.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:11 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:31:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:11 smithi193 ceph-mon[36001]: pgmap v196: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-05-14T15:31:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:11 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:31:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:11 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:31:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:11 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:31:12.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 576115 10.72 MB/sec execute 153 sec latency 30.189 ms 2024-05-14T15:31:13.132 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:31:13.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:31:13.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 577835 10.69 MB/sec execute 154 sec latency 122.394 ms 2024-05-14T15:31:13.400 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:14.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:13 smithi144 ceph-mon[35650]: pgmap v197: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-05-14T15:31:14.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:13 smithi193 ceph-mon[36001]: pgmap v197: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-05-14T15:31:14.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:13 smithi005 ceph-mon[31559]: pgmap v197: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 461 op/s 2024-05-14T15:31:14.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 579656 10.70 MB/sec execute 155 sec latency 21.856 ms 2024-05-14T15:31:15.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 581908 10.71 MB/sec execute 156 sec latency 16.754 ms 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 -- 172.21.15.5:0/691679043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9708109850 msgr2=0x7f97081111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 --2- 172.21.15.5:0/691679043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9708109850 0x7f97081111b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f970000b0d0 tx=0x7f970000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 -- 172.21.15.5:0/691679043 shutdown_connections 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 --2- 172.21.15.5:0/691679043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9708109850 0x7f97081111b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 --2- 172.21.15.5:0/691679043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97081045d0 0x7f97081049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 --2- 172.21.15.5:0/691679043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 0x7f9708109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 -- 172.21.15.5:0/691679043 >> 172.21.15.5:0/691679043 conn(0x7f97080ffea0 msgr2=0x7f97081022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 -- 172.21.15.5:0/691679043 shutdown_connections 2024-05-14T15:31:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.171+0000 7f970cc32700 1 -- 172.21.15.5:0/691679043 wait complete. 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 Processor -- start 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 -- start start 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97081045d0 0x7f970819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 0x7f970819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9708109850 0x7f970819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97081144d0 con 0x7f97081045d0 2024-05-14T15:31:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9708114350 con 0x7f9708108f00 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970cc32700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9708114650 con 0x7f9708109850 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97081045d0 0x7f970819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97081045d0 0x7f970819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56604/0 (socket says 172.21.15.5:56604) 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f970659c700 1 -- 172.21.15.5:0/3690235010 learned_addr learned my addr 172.21.15.5:0/3690235010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f9705d9b700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 0x7f970819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.172+0000 7f9706d9d700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9708109850 0x7f970819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f9705d9b700 1 -- 172.21.15.5:0/3690235010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9708109850 msgr2=0x7f970819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f9705d9b700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9708109850 0x7f970819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f9705d9b700 1 -- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97081045d0 msgr2=0x7f970819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f9705d9b700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97081045d0 0x7f970819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f9705d9b700 1 -- 172.21.15.5:0/3690235010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970819fc70 con 0x7f9708108f00 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f9705d9b700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 0x7f970819b340 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e9c0 tx=0x7f96fc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f96f77fe700 1 -- 172.21.15.5:0/3690235010 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00cb70 con 0x7f9708108f00 2024-05-14T15:31:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970819ff60 con 0x7f9708108f00 2024-05-14T15:31:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f96f77fe700 1 -- 172.21.15.5:0/3690235010 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96fc00ee60 con 0x7f9708108f00 2024-05-14T15:31:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f96f77fe700 1 -- 172.21.15.5:0/3690235010 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc003d60 con 0x7f9708108f00 2024-05-14T15:31:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.173+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97081a7880 con 0x7f9708108f00 2024-05-14T15:31:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.176+0000 7f96f77fe700 1 -- 172.21.15.5:0/3690235010 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f96fc018430 con 0x7f9708108f00 2024-05-14T15:31:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.177+0000 7f971b646b80 1 Processor -- start 2024-05-14T15:31:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.177+0000 7f971b646b80 1 -- start start 2024-05-14T15:31:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.177+0000 7f96f77fe700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f96f00b29f0 0x7f96f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f9706d9d700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f96f00b29f0 0x7f96f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f96f77fe700 1 -- 172.21.15.5:0/3690235010 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f96fc0e2af0 con 0x7f9708108f00 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f971b646b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f971b646b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f971b646b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f971b646b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ea22e98e0 con 0x559ea23d0910 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f971b646b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559ea2288840 con 0x559ea2249ec0 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f971b646b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559ea22e58a0 con 0x559ea23cc0f0 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42508/0 (socket says 172.21.15.5:42508) 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 -- 172.21.15.5:0/3542547410 learned_addr learned my addr 172.21.15.5:0/3542547410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f9705d9b700 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f9706d9d700 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 -- 172.21.15.5:0/3542547410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 msgr2=0x559ea23d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f9706d9d700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f96f00b29f0 0x7f96f00b4f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fd0 tx=0x7f9700009c60 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 -- 172.21.15.5:0/3542547410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 msgr2=0x559ea23d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f970659c700 1 -- 172.21.15.5:0/3542547410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea2236950 con 0x559ea2249ec0 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.178+0000 7f9705d9b700 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.179+0000 7f970659c700 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23cbbb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009ea0 tx=0x7f96f8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4efe1fa05fa59d85 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.179+0000 7f96f4ff9700 1 -- 172.21.15.5:0/3542547410 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f801e070 con 0x559ea2249ec0 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.179+0000 7f96f4ff9700 1 -- 172.21.15.5:0/3542547410 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f8019ae0 con 0x559ea2249ec0 2024-05-14T15:31:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.179+0000 7f96f4ff9700 1 -- 172.21.15.5:0/3542547410 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f800fa30 con 0x559ea2249ec0 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 -- 172.21.15.5:0/3542547410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 msgr2=0x559ea23cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23cbbb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009ea0 tx=0x7f96f8011720 comp rx=0 tx=0).stop 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 -- 172.21.15.5:0/3542547410 shutdown_connections 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23cbbb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 --2- 172.21.15.5:0/3542547410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 -- 172.21.15.5:0/3542547410 >> 172.21.15.5:0/3542547410 conn(0x559ea2291680 msgr2=0x559ea23c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 -- 172.21.15.5:0/3542547410 shutdown_connections 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 -- 172.21.15.5:0/3542547410 wait complete. 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.180+0000 7f971b646b80 1 -- start start 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 Processor -- start 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ea23d88c0 con 0x559ea23d0910 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559ea224fcb0 con 0x559ea2249ec0 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559ea23d8a40 con 0x559ea23cc0f0 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f971b646b80 10 client.0 _ll_register_callbacks cb 0x559ea22819a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f9705d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42516/0 (socket says 172.21.15.5:42516) 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 -- 172.21.15.5:0/1841283640 learned_addr learned my addr 172.21.15.5:0/1841283640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f9706d9d700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 -- 172.21.15.5:0/1841283640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 msgr2=0x559ea23d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 -- 172.21.15.5:0/1841283640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 msgr2=0x559ea23d6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.181+0000 7f970659c700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f970659c700 1 -- 172.21.15.5:0/1841283640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea2285710 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f9705d9b700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f970659c700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d55a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005e90 tx=0x7f96f8019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f293e9c5cf65b032 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f96edffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f96edffb700 1 -- 172.21.15.5:0/1841283640 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f801e050 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f96edffb700 1 -- 172.21.15.5:0/1841283640 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f96f8004250 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f8019ae0 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f96edffb700 1 -- 172.21.15.5:0/1841283640 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96f8032050 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f971b646b80 10 client.26576 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559ea23e3440 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.182+0000 7f96edffb700 1 -- 172.21.15.5:0/1841283640 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f8019ae0 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f96edffb700 1 -- 172.21.15.5:0/1841283640 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f96f8003680 con 0x559ea2249ec0 2024-05-14T15:31:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f971b646b80 10 client.26576 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f971b646b80 10 client.26576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f971b646b80 20 client.26576 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f971b646b80 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea23e59c0 0x559ea23e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f971b646b80 4 client.26576 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea22f72c0 con 0x559ea23e59c0 2024-05-14T15:31:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.183+0000 7f9705d9b700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea23e59c0 0x559ea23e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.184+0000 7f9705d9b700 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea23e59c0 0x559ea23e7de0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8c84323244d088 server_cookie=6283a295d6412d02 in_seq=0 out_seq=0 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.184+0000 7f96edffb700 10 client.26576 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.184+0000 7f96edffb700 1 -- 172.21.15.5:0/1841283640 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ea22f72c0 con 0x559ea23e59c0 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.184+0000 7f96edffb700 10 client.26576 handle_command_reply: tid=0 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.184+0000 7f971b646b80 1 client.26576 shutdown 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 msgr2=0x559ea23d55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d55a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96f8005e90 tx=0x7f96f8019660 comp rx=0 tx=0).stop 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 shutdown_connections 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea23e59c0 0x559ea23e7de0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23d0910 0x559ea23d6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d55a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f96edffb700 0 client.26576 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 --2- 172.21.15.5:0/1841283640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23cc0f0 0x559ea23d5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 >> 172.21.15.5:0/1841283640 conn(0x559ea2291680 msgr2=0x559ea23c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 shutdown_connections 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 1 -- 172.21.15.5:0/1841283640 wait complete. 2024-05-14T15:31:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.185+0000 7f971b646b80 20 client.26576 trim_cache size 0 max 16384 2024-05-14T15:31:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 Processor -- start 2024-05-14T15:31:15.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 -- start start 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 0x559ea23ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97081b0cf0 con 0x559ea23cc0f0 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97081ae580 con 0x559ea2249ec0 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.209+0000 7f971b646b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97081ac3d0 con 0x559ea23d0910 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9705d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 0x559ea23ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f970659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9705d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23e1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56612/0 (socket says 172.21.15.5:56612) 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 0x559ea23ed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f970659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:42522/0 (socket says 172.21.15.5:42522) 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 -- 172.21.15.5:0/1643697548 learned_addr learned my addr 172.21.15.5:0/1643697548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 -- 172.21.15.5:0/1643697548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 msgr2=0x559ea23d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 -- 172.21.15.5:0/1643697548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 msgr2=0x559ea23e1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23e1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 -- 172.21.15.5:0/1643697548 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea23d3b30 con 0x559ea23d0910 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f970659c700 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f9706d9d700 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 0x559ea23ed410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f970000bdf0 tx=0x7f9700020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481ca064fe5b5cc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f96effff700 1 -- 172.21.15.5:0/1643697548 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700024070 con 0x559ea23d0910 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f96effff700 1 -- 172.21.15.5:0/1643697548 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970000be20 con 0x559ea23d0910 2024-05-14T15:31:15.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.210+0000 7f96effff700 1 -- 172.21.15.5:0/1643697548 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700028420 con 0x559ea23d0910 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 -- 172.21.15.5:0/1643697548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 msgr2=0x559ea23ed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 0x559ea23ed410 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f970000bdf0 tx=0x7f9700020000 comp rx=0 tx=0).stop 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 -- 172.21.15.5:0/1643697548 shutdown_connections 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea23d0910 0x559ea23ed410 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea2249ec0 0x559ea23d47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 --2- 172.21.15.5:0/1643697548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 -- 172.21.15.5:0/1643697548 >> 172.21.15.5:0/1643697548 conn(0x559ea2291680 msgr2=0x559ea23c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 -- 172.21.15.5:0/1643697548 shutdown_connections 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.211+0000 7f971b646b80 1 -- 172.21.15.5:0/1643697548 wait complete. 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 -- start start 2024-05-14T15:31:15.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 Processor -- start 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 0x559ea23dc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23dce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea23d0910 0x559ea23dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97081aca40 con 0x559ea23cc0f0 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97081adeb0 con 0x559ea23d0910 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97081ad110 con 0x559ea2249ec0 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f971b646b80 10 client.0 _ll_register_callbacks cb 0x559ea22819a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f9705d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.212+0000 7f970659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 0x559ea23dc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f970659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 0x559ea23dc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:35674/0 (socket says 172.21.15.5:35674) 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f970659c700 1 -- 172.21.15.5:0/173753183 learned_addr learned my addr 172.21.15.5:0/173753183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9706d9d700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea23d0910 0x559ea23dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9705d9b700 1 -- 172.21.15.5:0/173753183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 msgr2=0x559ea23dc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9705d9b700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 0x559ea23dc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9705d9b700 1 -- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea23d0910 msgr2=0x559ea23dd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9705d9b700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea23d0910 0x559ea23dd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9705d9b700 1 -- 172.21.15.5:0/173753183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea244ffe0 con 0x559ea23cc0f0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f970659c700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 0x559ea23dc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9706d9d700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea23d0910 0x559ea23dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f9705d9b700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23dce80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0dfe80 tx=0x7f96fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4974bab3f74965fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f96edffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f96edffb700 1 -- 172.21.15.5:0/173753183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0c3060 con 0x559ea23cc0f0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f96edffb700 1 -- 172.21.15.5:0/173753183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f96fc0a9e80 con 0x559ea23cc0f0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970000be20 con 0x559ea23cc0f0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.213+0000 7f96edffb700 1 -- 172.21.15.5:0/173753183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96fc0bee60 con 0x559ea23cc0f0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.214+0000 7f971b646b80 10 client.16956 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.214+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559ea2451290 con 0x559ea23cc0f0 2024-05-14T15:31:15.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.214+0000 7f96edffb700 1 -- 172.21.15.5:0/173753183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970000be20 con 0x559ea23cc0f0 2024-05-14T15:31:15.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f96edffb700 1 -- 172.21.15.5:0/173753183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f96fc0b23e0 con 0x559ea23cc0f0 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f971b646b80 10 client.16956 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f971b646b80 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f971b646b80 20 client.16956 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f971b646b80 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea2451920 0x559ea2453d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f971b646b80 4 client.16956 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ea22f1360 con 0x559ea2451920 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.215+0000 7f970659c700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea2451920 0x559ea2453d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.216+0000 7f96edffb700 10 client.16956 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.216+0000 7f970659c700 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea2451920 0x559ea2453d40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a816f04853b5c0 server_cookie=37eb0a0370116c2b in_seq=0 out_seq=0 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f96edffb700 1 -- 172.21.15.5:0/173753183 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ea22f1360 con 0x559ea2451920 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f96edffb700 10 client.16956 handle_command_reply: tid=0 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 client.16956 shutdown 2024-05-14T15:31:15.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 msgr2=0x559ea23dce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23dce80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0dfe80 tx=0x7f96fc00cf40 comp rx=0 tx=0).stop 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 shutdown_connections 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559ea2451920 0x559ea2453d40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ea23d0910 0x559ea23dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ea2249ec0 0x559ea23dc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 --2- 172.21.15.5:0/173753183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ea23cc0f0 0x559ea23dce80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.217+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 >> 172.21.15.5:0/173753183 conn(0x559ea2291680 msgr2=0x559ea23d6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.218+0000 7f96edffb700 0 client.16956 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.218+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 shutdown_connections 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.218+0000 7f971b646b80 1 -- 172.21.15.5:0/173753183 wait complete. 2024-05-14T15:31:15.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.218+0000 7f971b646b80 20 client.16956 trim_cache size 0 max 16384 2024-05-14T15:31:15.223 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:31:15.223 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:31:15.223 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "cfcf6de0-53aa-4e19-a480-5359a1a23beb", 2024-05-14T15:31:15.223 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:31:15.223 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f96f00b29f0 msgr2=0x7f96f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f96f00b29f0 0x7f96f00b4f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fd0 tx=0x7f9700009c60 comp rx=0 tx=0).stop 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 msgr2=0x7f970819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 0x7f970819b340 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e9c0 tx=0x7f96fc00ecd0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 shutdown_connections 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f96f00b29f0 0x7f96f00b4f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9708109850 0x7f970819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97081045d0 0x7f970819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 --2- 172.21.15.5:0/3690235010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9708108f00 0x7f970819b340 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 >> 172.21.15.5:0/3690235010 conn(0x7f97080ffea0 msgr2=0x7f9708108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 shutdown_connections 2024-05-14T15:31:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:15.219+0000 7f970cc32700 1 -- 172.21.15.5:0/3690235010 wait complete. 2024-05-14T15:31:16.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:15 smithi144 ceph-mon[35650]: pgmap v198: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 335 op/s 2024-05-14T15:31:16.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:15 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:31:16.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:15 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:16.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:15 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:31:16.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:15 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:16.049 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:15 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:31:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:15 smithi193 ceph-mon[36001]: pgmap v198: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 335 op/s 2024-05-14T15:31:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:15 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:31:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:15 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:15 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:31:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:15 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:15 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:31:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:15 smithi005 ceph-mon[31559]: pgmap v198: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 335 op/s 2024-05-14T15:31:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:15 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:31:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:15 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:15 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:31:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:15 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:15 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:31:16.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 583586 10.70 MB/sec execute 157 sec latency 22.908 ms 2024-05-14T15:31:16.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:31:16.650 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:17.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 586070 10.73 MB/sec execute 158 sec latency 17.832 ms 2024-05-14T15:31:18.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:17 smithi144 ceph-mon[35650]: pgmap v199: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-05-14T15:31:18.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:17 smithi193 ceph-mon[36001]: pgmap v199: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-05-14T15:31:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:17 smithi005 ceph-mon[31559]: pgmap v199: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 405 op/s 2024-05-14T15:31:18.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 587222 10.68 MB/sec execute 159 sec latency 95.010 ms 2024-05-14T15:31:19.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 -- 172.21.15.5:0/3173657089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 msgr2=0x7f22e0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 --2- 172.21.15.5:0/3173657089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e0108200 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f22c80099c0 tx=0x7f22c8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 -- 172.21.15.5:0/3173657089 shutdown_connections 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 --2- 172.21.15.5:0/3173657089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e0108740 0x7f22e01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 --2- 172.21.15.5:0/3173657089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e01034c0 0x7f22e01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 --2- 172.21.15.5:0/3173657089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e0108200 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 -- 172.21.15.5:0/3173657089 >> 172.21.15.5:0/3173657089 conn(0x7f22e00fedf0 msgr2=0x7f22e0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 -- 172.21.15.5:0/3173657089 shutdown_connections 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.815+0000 7f22e52cd700 1 -- 172.21.15.5:0/3173657089 wait complete. 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 Processor -- start 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 -- start start 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e01034c0 0x7f22e0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e0108740 0x7f22e019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0113370 con 0x7f22e0107df0 2024-05-14T15:31:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01131f0 con 0x7f22e01034c0 2024-05-14T15:31:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22e52cd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01134f0 con 0x7f22e0108740 2024-05-14T15:31:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35572/0 (socket says 172.21.15.5:35572) 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.816+0000 7f22d67fc700 1 -- 172.21.15.5:0/4206357186 learned_addr learned my addr 172.21.15.5:0/4206357186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22deffd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e01034c0 0x7f22e0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22df7fe700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e0108740 0x7f22e019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22d67fc700 1 -- 172.21.15.5:0/4206357186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e0108740 msgr2=0x7f22e019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22d67fc700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e0108740 0x7f22e019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22d67fc700 1 -- 172.21.15.5:0/4206357186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e01034c0 msgr2=0x7f22e0199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22d67fc700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e01034c0 0x7f22e0199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22d67fc700 1 -- 172.21.15.5:0/4206357186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e019eb40 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22d67fc700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e019a2c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f22c80045e0 tx=0x7f22c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22dcff9700 1 -- 172.21.15.5:0/4206357186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8026070 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e019edd0 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22dcff9700 1 -- 172.21.15.5:0/4206357186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c800f460 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22dcff9700 1 -- 172.21.15.5:0/4206357186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8009e60 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.817+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e019f2b0 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.821+0000 7f22dcff9700 1 -- 172.21.15.5:0/4206357186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f22c8029070 con 0x7f22e0107df0 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.821+0000 7f22f3ce1b80 1 Processor -- start 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.821+0000 7f22f3ce1b80 1 -- start start 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 0x564b172bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b171db860 con 0x564b172b9c60 2024-05-14T15:31:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564b1717a7c0 con 0x564b1713be40 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b171d7820 con 0x564b172be470 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 0x564b172bdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 0x564b172bdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35576/0 (socket says 172.21.15.5:35576) 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 -- 172.21.15.5:0/3378893586 learned_addr learned my addr 172.21.15.5:0/3378893586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22d67fc700 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22dcff9700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22cc0b29f0 0x7f22cc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 -- 172.21.15.5:0/3378893586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 msgr2=0x564b172be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 -- 172.21.15.5:0/3378893586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 msgr2=0x564b172b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22deffd700 1 -- 172.21.15.5:0/3378893586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b171288d0 con 0x564b172b9c60 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22df7fe700 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.822+0000 7f22df7fe700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22cc0b29f0 0x7f22cc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.823+0000 7f22dcff9700 1 -- 172.21.15.5:0/4206357186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f22c80e32d0 con 0x7f22e0107df0 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.823+0000 7f22deffd700 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 0x564b172bdf30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f22d0008c60 tx=0x7f22d0008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1017d6f8da1a732 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.823+0000 7f22d57fa700 1 -- 172.21.15.5:0/3378893586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0014070 con 0x564b172b9c60 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.823+0000 7f22d57fa700 1 -- 172.21.15.5:0/3378893586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d000ed70 con 0x564b172b9c60 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.823+0000 7f22d57fa700 1 -- 172.21.15.5:0/3378893586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0010430 con 0x564b172b9c60 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/3378893586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 msgr2=0x564b172bdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22df7fe700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22cc0b29f0 0x7f22cc0b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x564b172ba620 tx=0x7f22d8004ad0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 0x564b172bdf30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f22d0008c60 tx=0x7f22d0008f70 comp rx=0 tx=0).stop 2024-05-14T15:31:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/3378893586 shutdown_connections 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/3378893586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172b9c60 0x564b172bdf30 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/3378893586 >> 172.21.15.5:0/3378893586 conn(0x564b17183600 msgr2=0x564b172b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/3378893586 shutdown_connections 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/3378893586 wait complete. 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 -- start start 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.824+0000 7f22f3ce1b80 1 Processor -- start 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 0x564b172c3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172be470 0x564b172c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b172c6420 con 0x564b172be470 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564b17141c30 con 0x564b1713be40 2024-05-14T15:31:19.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b172c65a0 con 0x564b172b9c60 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22f3ce1b80 10 client.0 _ll_register_callbacks cb 0x564b17173920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172be470 0x564b172c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22deffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 0x564b172c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 0x564b172c3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:46374/0 (socket says 172.21.15.5:46374) 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 -- 172.21.15.5:0/383603462 learned_addr learned my addr 172.21.15.5:0/383603462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22deffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:36452/0 (socket says 172.21.15.5:36452) 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 -- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 msgr2=0x564b172c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 -- 172.21.15.5:0/383603462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172be470 msgr2=0x564b172c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172be470 0x564b172c3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.825+0000 7f22d67fc700 1 -- 172.21.15.5:0/383603462 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b17177690 con 0x564b172b9c60 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22d67fc700 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 0x564b172c3780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c800bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c27bdc5a999359a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:19.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22c67fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22c67fc700 1 -- 172.21.15.5:0/383603462 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80aa980 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22c67fc700 1 -- 172.21.15.5:0/383603462 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f22c80bc0c0 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d000ed70 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22c67fc700 1 -- 172.21.15.5:0/383603462 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22c80ae630 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22f3ce1b80 10 client.26563 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564b172c6420 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.826+0000 7f22c67fc700 1 -- 172.21.15.5:0/383603462 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c80b3050 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22c67fc700 1 -- 172.21.15.5:0/383603462 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f22c80aa4e0 con 0x564b172b9c60 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22f3ce1b80 10 client.26563 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22f3ce1b80 10 client.26563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22f3ce1b80 20 client.26563 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b172d3580 0x564b172d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22f3ce1b80 4 client.26563 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22d000ed70 con 0x564b172d3580 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22deffd700 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b172d3580 0x564b172d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22c67fc700 10 client.26563 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.827+0000 7f22deffd700 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b172d3580 0x564b172d59d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6fce164ac73539f server_cookie=546e88dc77fdc271 in_seq=0 out_seq=0 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22c67fc700 1 -- 172.21.15.5:0/383603462 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f22d000ed70 con 0x564b172d3580 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22c67fc700 10 client.26563 handle_command_reply: tid=0 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 client.26563 shutdown 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 msgr2=0x564b172c3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 0x564b172c3780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c800bec0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 shutdown_connections 2024-05-14T15:31:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b172d3580 0x564b172d59d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b172be470 0x564b172c3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22c67fc700 0 client.26563 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b1713be40 0x564b172c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/383603462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172b9c60 0x564b172c3780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.830+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 >> 172.21.15.5:0/383603462 conn(0x564b17183600 msgr2=0x564b172b8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.831+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 shutdown_connections 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.831+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/383603462 wait complete. 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.831+0000 7f22f3ce1b80 20 client.26563 trim_cache size 0 max 16384 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 Processor -- start 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 -- start start 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01b1d10 con 0x564b1713be40 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01af5a0 con 0x564b172b9c60 2024-05-14T15:31:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.852+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01ad3f0 con 0x564b172be470 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172c3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35602/0 (socket says 172.21.15.5:35602) 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 -- 172.21.15.5:0/2074486926 learned_addr learned my addr 172.21.15.5:0/2074486926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22d67fc700 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22df7fe700 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 -- 172.21.15.5:0/2074486926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 msgr2=0x564b172db170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172db170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 -- 172.21.15.5:0/2074486926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 msgr2=0x564b172b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 -- 172.21.15.5:0/2074486926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b172ba150 con 0x564b1713be40 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22deffd700 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172c3b00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x564b172b5b70 tx=0x7f22d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0940ec4ae5d8434 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.853+0000 7f22d4ff9700 1 -- 172.21.15.5:0/2074486926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d002b070 con 0x564b1713be40 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.854+0000 7f22d4ff9700 1 -- 172.21.15.5:0/2074486926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d0025900 con 0x564b1713be40 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.854+0000 7f22d4ff9700 1 -- 172.21.15.5:0/2074486926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0014040 con 0x564b1713be40 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/2074486926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 msgr2=0x564b172c3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172c3b00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x564b172b5b70 tx=0x7f22d000ef20 comp rx=0 tx=0).stop 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/2074486926 shutdown_connections 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172db170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172c3b00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/2074486926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/2074486926 >> 172.21.15.5:0/2074486926 conn(0x564b17183600 msgr2=0x564b172b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/2074486926 shutdown_connections 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/2074486926 wait complete. 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 -- start start 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.855+0000 7f22f3ce1b80 1 Processor -- start 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01ada60 con 0x564b1713be40 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01aeed0 con 0x564b172b9c60 2024-05-14T15:31:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f22e01ae130 con 0x564b172be470 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 10 client.0 _ll_register_callbacks cb 0x564b17173920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35604/0 (socket says 172.21.15.5:35604) 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 -- 172.21.15.5:0/1285061535 learned_addr learned my addr 172.21.15.5:0/1285061535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22d67fc700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22df7fe700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 -- 172.21.15.5:0/1285061535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 msgr2=0x564b172ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 -- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 msgr2=0x564b172c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 -- 172.21.15.5:0/1285061535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b172cc260 con 0x564b1713be40 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22d67fc700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22deffd700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172cbb30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f22d000e7a0 tx=0x7f22d00083c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721e46447ab71a9a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22c67fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22c67fc700 1 -- 172.21.15.5:0/1285061535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0034060 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22c67fc700 1 -- 172.21.15.5:0/1285061535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f22d0025650 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.856+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d0025900 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22c67fc700 1 -- 172.21.15.5:0/1285061535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22d003b050 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 10 client.16974 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564b17342e00 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22c67fc700 1 -- 172.21.15.5:0/1285061535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d0025900 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22c67fc700 1 -- 172.21.15.5:0/1285061535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f22d00109c0 con 0x564b1713be40 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 20 client.16974 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b17343260 0x564b17345680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 4 client.16974 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:31:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b171e9240 con 0x564b17343260 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.857+0000 7f22d67fc700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b17343260 0x564b17345680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22c67fc700 10 client.16974 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22d67fc700 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b17343260 0x564b17345680 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31400008fc85536b server_cookie=9c81114849298489 in_seq=0 out_seq=0 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22c67fc700 1 -- 172.21.15.5:0/1285061535 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564b171e9240 con 0x564b17343260 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22c67fc700 10 client.16974 handle_command_reply: tid=0 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 client.16974 shutdown 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:31:19.100 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 msgr2=0x564b172cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172cbb30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f22d000e7a0 tx=0x7f22d00083c0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 shutdown_connections 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x564b17343260 0x564b17345680 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b172be470 0x564b172ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b1713be40 0x564b172cbb30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22c67fc700 0 client.16974 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 --2- 172.21.15.5:0/1285061535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564b172b9c60 0x564b172c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 >> 172.21.15.5:0/1285061535 conn(0x564b17183600 msgr2=0x564b172b8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 shutdown_connections 2024-05-14T15:31:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 1 -- 172.21.15.5:0/1285061535 wait complete. 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.858+0000 7f22f3ce1b80 20 client.16974 trim_cache size 0 max 16384 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22cc0b29f0 msgr2=0x7f22cc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22cc0b29f0 0x7f22cc0b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x564b172ba620 tx=0x7f22d8004ad0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 msgr2=0x7f22e019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e019a2c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f22c80045e0 tx=0x7f22c80046c0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 shutdown_connections 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f22cc0b29f0 0x7f22cc0b4f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22e0108740 0x7f22e019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e01034c0 0x7f22e0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 --2- 172.21.15.5:0/4206357186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e0107df0 0x7f22e019a2c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 >> 172.21.15.5:0/4206357186 conn(0x7f22e00fedf0 msgr2=0x7f22e01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 shutdown_connections 2024-05-14T15:31:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:18.859+0000 7f22e52cd700 1 -- 172.21.15.5:0/4206357186 wait complete. 2024-05-14T15:31:19.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 589592 10.71 MB/sec execute 160 sec latency 54.126 ms 2024-05-14T15:31:19.890 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:31:19.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:31:20.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:19 smithi193 ceph-mon[36001]: pgmap v200: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-14T15:31:20.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:19 smithi005 ceph-mon[31559]: pgmap v200: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-14T15:31:20.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 591092 10.69 MB/sec execute 161 sec latency 18.964 ms 2024-05-14T15:31:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:19 smithi144 ceph-mon[35650]: pgmap v200: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-14T15:31:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 -- 172.21.15.5:0/926218365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec1032e0 msgr2=0x7f21ec10f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 --2- 172.21.15.5:0/926218365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec1032e0 0x7f21ec10f6b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009ca0 tx=0x7f21e402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 -- 172.21.15.5:0/926218365 shutdown_connections 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 --2- 172.21.15.5:0/926218365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec1032e0 0x7f21ec10f6b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 --2- 172.21.15.5:0/926218365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ec109030 0x7f21ec102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 --2- 172.21.15.5:0/926218365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 0x7f21ec102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 -- 172.21.15.5:0/926218365 >> 172.21.15.5:0/926218365 conn(0x7f21ec0fe390 msgr2=0x7f21ec1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 -- 172.21.15.5:0/926218365 shutdown_connections 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.384+0000 7f21f22b8640 1 -- 172.21.15.5:0/926218365 wait complete. 2024-05-14T15:31:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 Processor -- start 2024-05-14T15:31:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 -- start start 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ec1032e0 0x7f21ec1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 0x7f21ec1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 0x7f21ec19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec112da0 con 0x7f21ec109030 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec112c20 con 0x7f21ec1032e0 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.385+0000 7f21f22b8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec112f20 con 0x7f21ec108b90 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21f0855640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 0x7f21ec19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21f0855640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 0x7f21ec19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35632/0 (socket says 172.21.15.5:35632) 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21f0855640 1 -- 172.21.15.5:0/3907442349 learned_addr learned my addr 172.21.15.5:0/3907442349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 0x7f21ec1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21ebfff640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ec1032e0 0x7f21ec1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 -- 172.21.15.5:0/3907442349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ec1032e0 msgr2=0x7f21ec1a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ec1032e0 0x7f21ec1a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 -- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 msgr2=0x7f21ec19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 0x7f21ec19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 -- 172.21.15.5:0/3907442349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec19adc0 con 0x7f21ec108b90 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21f0855640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 0x7f21ec19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 0x7f21ec1a0a50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21ec06c610 tx=0x7f21e000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21e97fa640 1 -- 172.21.15.5:0/3907442349 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000cc40 con 0x7f21ec108b90 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec19b050 con 0x7f21ec108b90 2024-05-14T15:31:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21e97fa640 1 -- 172.21.15.5:0/3907442349 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e000cde0 con 0x7f21ec108b90 2024-05-14T15:31:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.386+0000 7f21e97fa640 1 -- 172.21.15.5:0/3907442349 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0010430 con 0x7f21ec108b90 2024-05-14T15:31:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.387+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1a72b0 con 0x7f21ec108b90 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.389+0000 7f21e97fa640 1 -- 172.21.15.5:0/3907442349 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f21e00105d0 con 0x7f21ec108b90 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.389+0000 7f21f9168740 1 Processor -- start 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.389+0000 7f21f9168740 1 -- start start 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f9168740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e469be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f9168740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a02f0 0x56381e46a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f9168740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a84d0 0x56381e46a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f9168740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56381e3cf5d0 con 0x56381e0c91b0 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f9168740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56381e388890 con 0x56381e0a02f0 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f9168740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56381e3cb590 con 0x56381e0a84d0 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e469be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a02f0 0x56381e46a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a84d0 0x56381e46a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a02f0 0x56381e46a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:36490/0 (socket says 172.21.15.5:36490) 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21ebfff640 1 -- 172.21.15.5:0/1910161789 learned_addr learned my addr 172.21.15.5:0/1910161789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e469be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35644/0 (socket says 172.21.15.5:35644) 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21e97fa640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f21d40b2950 0x7f21d40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 -- 172.21.15.5:0/1910161789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a84d0 msgr2=0x56381e46a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a84d0 0x56381e46a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 -- 172.21.15.5:0/1910161789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a02f0 msgr2=0x56381e46a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a02f0 0x56381e46a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 -- 172.21.15.5:0/1910161789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381e0e1210 con 0x56381e0c91b0 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21e97fa640 1 -- 172.21.15.5:0/3907442349 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f21e00e2d80 con 0x7f21ec108b90 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.390+0000 7f21f0855640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f21d40b2950 0x7f21d40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.391+0000 7f21f0855640 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e469be0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f21e4033040 tx=0x7f21e402f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deee137644ce0e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.391+0000 7f21d27fc640 1 -- 172.21.15.5:0/1910161789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4043040 con 0x56381e0c91b0 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.391+0000 7f21d27fc640 1 -- 172.21.15.5:0/1910161789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e4036040 con 0x56381e0c91b0 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.391+0000 7f21d27fc640 1 -- 172.21.15.5:0/1910161789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e402f8a0 con 0x56381e0c91b0 2024-05-14T15:31:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.391+0000 7f21f0855640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f21d40b2950 0x7f21d40b4e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56381e46b690 tx=0x7f21e404f000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 -- 172.21.15.5:0/1910161789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 msgr2=0x56381e469be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e469be0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f21e4033040 tx=0x7f21e402f390 comp rx=0 tx=0).stop 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 -- 172.21.15.5:0/1910161789 shutdown_connections 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a84d0 0x56381e46a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a02f0 0x56381e46a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 --2- 172.21.15.5:0/1910161789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e469be0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 -- 172.21.15.5:0/1910161789 >> 172.21.15.5:0/1910161789 conn(0x56381e0bd300 msgr2=0x56381e466340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 -- 172.21.15.5:0/1910161789 shutdown_connections 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 -- 172.21.15.5:0/1910161789 wait complete. 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.392+0000 7f21f9168740 1 -- start start 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 Processor -- start 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e474210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e474750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56381e476d10 con 0x56381e0a84d0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56381e38c370 con 0x56381e0c91b0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56381e476e90 con 0x56381e0a02f0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f9168740 10 client.0 _ll_register_callbacks cb 0x56381e0e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e474750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e474210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e474750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35646/0 (socket says 172.21.15.5:35646) 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21eb7fe640 1 -- 172.21.15.5:0/1583801795 learned_addr learned my addr 172.21.15.5:0/1583801795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21f0855640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21eb7fe640 1 -- 172.21.15.5:0/1583801795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 msgr2=0x56381e474210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.393+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e474210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21eb7fe640 1 -- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 msgr2=0x56381e47a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21eb7fe640 1 -- 172.21.15.5:0/1583801795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381e1ad620 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21f0855640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e474750 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f21e00a9c30 tx=0x7f21e00e64c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f5417a1168ac1d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21cb7fe640 1 -- 172.21.15.5:0/1583801795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e00a9410 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4036040 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21cb7fe640 1 -- 172.21.15.5:0/1583801795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f21e00a5610 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21cb7fe640 1 -- 172.21.15.5:0/1583801795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f21e00b2440 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21f9168740 10 client.16986 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56381e483090 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.394+0000 7f21cb7fe640 1 -- 172.21.15.5:0/1583801795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e4036040 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21cb7fe640 1 -- 172.21.15.5:0/1583801795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f21e00a5810 con 0x56381e0a84d0 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21f9168740 10 client.16986 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21f9168740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21f9168740 20 client.16986 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21f9168740 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e087d80 0x56381e486510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21f9168740 4 client.16986 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56381e3dd1e0 con 0x56381e087d80 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21ebfff640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e087d80 0x56381e486510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21cb7fe640 10 client.16986 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.395+0000 7f21ebfff640 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e087d80 0x56381e486510 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc64cb27491b627a server_cookie=b49c1a3e1690165f in_seq=0 out_seq=0 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21cb7fe640 1 -- 172.21.15.5:0/1583801795 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56381e3dd1e0 con 0x56381e087d80 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21cb7fe640 10 client.16986 handle_command_reply: tid=0 2024-05-14T15:31:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 client.16986 shutdown 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 msgr2=0x56381e474750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e474750 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f21e00a9c30 tx=0x7f21e00e64c0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 shutdown_connections 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e087d80 0x56381e486510 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e474750 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 --2- 172.21.15.5:0/1583801795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e474210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 >> 172.21.15.5:0/1583801795 conn(0x56381e0bd300 msgr2=0x56381e469a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21cb7fe640 0 client.16986 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.396+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 shutdown_connections 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.397+0000 7f21f9168740 1 -- 172.21.15.5:0/1583801795 wait complete. 2024-05-14T15:31:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.397+0000 7f21f9168740 20 client.16986 trim_cache size 0 max 16384 2024-05-14T15:31:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.417+0000 7f21f9168740 1 Processor -- start 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.417+0000 7f21f9168740 1 -- start start 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f9168740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e476150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f9168740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a84d0 0x56381e46a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f9168740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e46a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f9168740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1b3bb0 con 0x56381e0c91b0 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f9168740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1b1480 con 0x56381e0a84d0 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f9168740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1af2d0 con 0x56381e0a02f0 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e476150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a84d0 0x56381e46a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e476150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:46450/0 (socket says 172.21.15.5:46450) 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a84d0 0x56381e46a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:36510/0 (socket says 172.21.15.5:36510) 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 -- 172.21.15.5:0/54916437 learned_addr learned my addr 172.21.15.5:0/54916437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f0855640 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e46a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 -- 172.21.15.5:0/54916437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a84d0 msgr2=0x56381e46a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a84d0 0x56381e46a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 -- 172.21.15.5:0/54916437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 msgr2=0x56381e46a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e46a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21ebfff640 1 -- 172.21.15.5:0/54916437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381e4768a0 con 0x56381e0a02f0 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.418+0000 7f21f0855640 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e46a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.419+0000 7f21ebfff640 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e476150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56381e4676d0 tx=0x7f21dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950f799f01b7e967 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.419+0000 7f21d1ffb640 1 -- 172.21.15.5:0/54916437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc005220 con 0x56381e0a02f0 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.419+0000 7f21d1ffb640 1 -- 172.21.15.5:0/54916437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc024030 con 0x56381e0a02f0 2024-05-14T15:31:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.419+0000 7f21d1ffb640 1 -- 172.21.15.5:0/54916437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc020330 con 0x56381e0a02f0 2024-05-14T15:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 -- 172.21.15.5:0/54916437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 msgr2=0x56381e476150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e476150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56381e4676d0 tx=0x7f21dc011810 comp rx=0 tx=0).stop 2024-05-14T15:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 -- 172.21.15.5:0/54916437 shutdown_connections 2024-05-14T15:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0c91b0 0x56381e46a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0a84d0 0x56381e46a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 --2- 172.21.15.5:0/54916437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e476150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 -- 172.21.15.5:0/54916437 >> 172.21.15.5:0/54916437 conn(0x56381e0bd300 msgr2=0x56381e469970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 -- 172.21.15.5:0/54916437 shutdown_connections 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.420+0000 7f21f9168740 1 -- 172.21.15.5:0/54916437 wait complete. 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 -- start start 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 Processor -- start 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e4e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e4e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1af940 con 0x56381e0a84d0 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1b0db0 con 0x56381e0c91b0 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1b0010 con 0x56381e0a02f0 2024-05-14T15:31:20.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f9168740 10 client.0 _ll_register_callbacks cb 0x7f21e00e65c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e4e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e4e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35670/0 (socket says 172.21.15.5:35670) 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21eb7fe640 1 -- 172.21.15.5:0/2733189079 learned_addr learned my addr 172.21.15.5:0/2733189079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21f0855640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.421+0000 7f21ebfff640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e4e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21eb7fe640 1 -- 172.21.15.5:0/2733189079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 msgr2=0x56381e4e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e4e5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21eb7fe640 1 -- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 msgr2=0x56381e47d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21eb7fe640 1 -- 172.21.15.5:0/2733189079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381e4619a0 con 0x56381e0a84d0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21eb7fe640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e4e6100 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f21e00a99a0 tx=0x7f21e00a9320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a13aa46fa097f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21cb7fe640 1 -- 172.21.15.5:0/2733189079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e00a9410 con 0x56381e0a84d0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21cb7fe640 1 -- 172.21.15.5:0/2733189079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f21e00a5820 con 0x56381e0a84d0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.422+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc024030 con 0x56381e0a84d0 2024-05-14T15:31:20.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.423+0000 7f21cb7fe640 1 -- 172.21.15.5:0/2733189079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f21e00a97b0 con 0x56381e0a84d0 2024-05-14T15:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.423+0000 7f21f9168740 10 client.16992 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.423+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56381e4ee060 con 0x56381e0a84d0 2024-05-14T15:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21cb7fe640 1 -- 172.21.15.5:0/2733189079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21dc024030 con 0x56381e0a84d0 2024-05-14T15:31:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21cb7fe640 1 -- 172.21.15.5:0/2733189079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f21e0076240 con 0x56381e0a84d0 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21f9168740 10 client.16992 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21f9168740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21f9168740 20 client.16992 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21f9168740 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e05d1c0 0x56381e4f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21f9168740 4 client.16992 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56381e3d7050 con 0x56381e05d1c0 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.424+0000 7f21ebfff640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e05d1c0 0x56381e4f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.425+0000 7f21ebfff640 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e05d1c0 0x56381e4f0650 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2f04c3ab097342 server_cookie=479cd8b3763bce71 in_seq=0 out_seq=0 2024-05-14T15:31:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.425+0000 7f21cb7fe640 10 client.16992 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21cb7fe640 1 -- 172.21.15.5:0/2733189079 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56381e3d7050 con 0x56381e05d1c0 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21cb7fe640 10 client.16992 handle_command_reply: tid=0 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21f9168740 1 client.16992 shutdown 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 msgr2=0x56381e4e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21f9168740 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e4e6100 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f21e00a99a0 tx=0x7f21e00a9320 comp rx=0 tx=0).stop 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 shutdown_connections 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21f9168740 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56381e05d1c0 0x56381e4f0650 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.757+0000 7f21cb7fe640 0 client.16992 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56381e0c91b0 0x56381e47d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56381e0a84d0 0x56381e4e6100 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 1 --2- 172.21.15.5:0/2733189079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56381e0a02f0 0x56381e4e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 >> 172.21.15.5:0/2733189079 conn(0x56381e0bd300 msgr2=0x56381e475eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 shutdown_connections 2024-05-14T15:31:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 1 -- 172.21.15.5:0/2733189079 wait complete. 2024-05-14T15:31:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.758+0000 7f21f9168740 20 client.16992 trim_cache size 0 max 16384 2024-05-14T15:31:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:31:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:31:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:31:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:31:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.759+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f21d40b2950 msgr2=0x7f21d40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.759+0000 7f21f22b8640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f21d40b2950 0x7f21d40b4e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56381e46b690 tx=0x7f21e404f000 comp rx=0 tx=0).stop 2024-05-14T15:31:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.759+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 msgr2=0x7f21ec1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 0x7f21ec1a0a50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21ec06c610 tx=0x7f21e000ee50 comp rx=0 tx=0).stop 2024-05-14T15:31:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 shutdown_connections 2024-05-14T15:31:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f21d40b2950 0x7f21d40b4e60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21ec108b90 0x7f21ec1a0a50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec109030 0x7f21ec19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 --2- 172.21.15.5:0/3907442349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21ec1032e0 0x7f21ec1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 >> 172.21.15.5:0/3907442349 conn(0x7f21ec0fe390 msgr2=0x7f21ec0752d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 shutdown_connections 2024-05-14T15:31:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:20.760+0000 7f21f22b8640 1 -- 172.21.15.5:0/3907442349 wait complete. 2024-05-14T15:31:21.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 592827 10.67 MB/sec execute 162 sec latency 233.460 ms 2024-05-14T15:31:22.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:21 smithi193 ceph-mon[36001]: pgmap v201: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-05-14T15:31:22.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:21 smithi005 ceph-mon[31559]: pgmap v201: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-05-14T15:31:22.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594288 10.66 MB/sec execute 163 sec latency 51.413 ms 2024-05-14T15:31:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:21 smithi144 ceph-mon[35650]: pgmap v201: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 -- 172.21.15.5:0/156398924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2501034c0 msgr2=0x7ff2501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 --2- 172.21.15.5:0/156398924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2501034c0 0x7ff2501038d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff2400099c0 tx=0x7ff240009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 -- 172.21.15.5:0/156398924 shutdown_connections 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 --2- 172.21.15.5:0/156398924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff250108740 0x7ff2501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 --2- 172.21.15.5:0/156398924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2501034c0 0x7ff2501038d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 --2- 172.21.15.5:0/156398924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250107df0 0x7ff250108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 -- 172.21.15.5:0/156398924 >> 172.21.15.5:0/156398924 conn(0x7ff2500fedc0 msgr2=0x7ff2501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 -- 172.21.15.5:0/156398924 shutdown_connections 2024-05-14T15:31:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.333+0000 7ff2551a4700 1 -- 172.21.15.5:0/156398924 wait complete. 2024-05-14T15:31:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.334+0000 7ff2551a4700 1 Processor -- start 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 -- start start 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 0x7ff25019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff250107df0 0x7ff25019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250108740 0x7ff250198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501122d0 con 0x7ff250107df0 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff250112150 con 0x7ff2501034c0 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff2551a4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff250112450 con 0x7ff250108740 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 0x7ff25019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 0x7ff25019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:36542/0 (socket says 172.21.15.5:36542) 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 -- 172.21.15.5:0/1306059479 learned_addr learned my addr 172.21.15.5:0/1306059479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24f59e700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250108740 0x7ff250198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24e59c700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff250107df0 0x7ff25019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 -- 172.21.15.5:0/1306059479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250108740 msgr2=0x7ff250198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250108740 0x7ff250198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 -- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff250107df0 msgr2=0x7ff25019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff250107df0 0x7ff25019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24ed9d700 1 -- 172.21.15.5:0/1306059479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2501990c0 con 0x7ff2501034c0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.335+0000 7ff24f59e700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250108740 0x7ff250198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.336+0000 7ff24ed9d700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 0x7ff25019e7f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff240006010 tx=0x7ff2400045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.336+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240026070 con 0x7ff2501034c0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.336+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24000f460 con 0x7ff2501034c0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.336+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240009e60 con 0x7ff2501034c0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.336+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff250199350 con 0x7ff2501034c0 2024-05-14T15:31:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.336+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff250199830 con 0x7ff2501034c0 2024-05-14T15:31:22.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.337+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2500512c0 con 0x7ff2501034c0 2024-05-14T15:31:22.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.339+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff240029070 con 0x7ff2501034c0 2024-05-14T15:31:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.340+0000 7ff237fff700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff2380b29f0 0x7ff2380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.340+0000 7ff24e59c700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff2380b29f0 0x7ff2380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.340+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff2400e3610 con 0x7ff2501034c0 2024-05-14T15:31:22.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.340+0000 7ff24e59c700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff2380b29f0 0x7ff2380b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2501063e0 tx=0x7ff24400d040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.343+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff240030080 con 0x7ff2501034c0 2024-05-14T15:31:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.641+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff25002e5f0 con 0x7ff2501034c0 2024-05-14T15:31:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.642+0000 7ff237fff700 1 -- 172.21.15.5:0/1306059479 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7ff240013070 con 0x7ff2501034c0 2024-05-14T15:31:22.644 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:31:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:31:22.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff2380b29f0 msgr2=0x7ff2380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:22.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff2380b29f0 0x7ff2380b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2501063e0 tx=0x7ff24400d040 comp rx=0 tx=0).stop 2024-05-14T15:31:22.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 msgr2=0x7ff25019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:22.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 0x7ff25019e7f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff240006010 tx=0x7ff2400045e0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 shutdown_connections 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff2380b29f0 0x7ff2380b4f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff250108740 0x7ff250198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2501034c0 0x7ff25019e7f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 --2- 172.21.15.5:0/1306059479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff250107df0 0x7ff25019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.644+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 >> 172.21.15.5:0/1306059479 conn(0x7ff2500fedc0 msgr2=0x7ff250101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.645+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 shutdown_connections 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:22.645+0000 7ff2551a4700 1 -- 172.21.15.5:0/1306059479 wait complete. 2024-05-14T15:31:22.647 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:31:23.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:22 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1306059479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:31:23.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 596523 10.67 MB/sec execute 164 sec latency 86.758 ms 2024-05-14T15:31:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:22 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1306059479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:31:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:22 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1306059479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:31:23.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:31:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:24.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:23 smithi005 ceph-mon[31559]: pgmap v202: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 549 op/s 2024-05-14T15:31:24.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 598146 10.66 MB/sec execute 165 sec latency 50.524 ms 2024-05-14T15:31:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:23 smithi144 ceph-mon[35650]: pgmap v202: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 549 op/s 2024-05-14T15:31:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:23 smithi193 ceph-mon[36001]: pgmap v202: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 549 op/s 2024-05-14T15:31:25.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 600354 10.67 MB/sec execute 166 sec latency 107.731 ms 2024-05-14T15:31:26.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:25 smithi005 ceph-mon[31559]: pgmap v203: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:31:26.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 601824 10.65 MB/sec execute 167 sec latency 30.832 ms 2024-05-14T15:31:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:25 smithi144 ceph-mon[35650]: pgmap v203: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:31:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:25 smithi193 ceph-mon[36001]: pgmap v203: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 -- 172.21.15.5:0/3132501978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a41045f0 msgr2=0x7f60a4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 --2- 172.21.15.5:0/3132501978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a41045f0 0x7f60a4104a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6094009990 tx=0x7f6094009ca0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 -- 172.21.15.5:0/3132501978 shutdown_connections 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 --2- 172.21.15.5:0/3132501978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 0x7f60a41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 --2- 172.21.15.5:0/3132501978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a41045f0 0x7f60a4104a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 --2- 172.21.15.5:0/3132501978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a4108f20 0x7f60a4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 -- 172.21.15.5:0/3132501978 >> 172.21.15.5:0/3132501978 conn(0x7f60a4100040 msgr2=0x7f60a4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 -- 172.21.15.5:0/3132501978 shutdown_connections 2024-05-14T15:31:26.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.357+0000 7f60aa05f700 1 -- 172.21.15.5:0/3132501978 wait complete. 2024-05-14T15:31:26.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 Processor -- start 2024-05-14T15:31:26.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 -- start start 2024-05-14T15:31:26.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 0x7f60a419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a4108f20 0x7f60a419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 0x7f60a419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41144f0 con 0x7f60a4109870 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4114370 con 0x7f60a4108f20 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60aa05f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4114670 con 0x7f60a41045f0 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 0x7f60a419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 0x7f60a419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 0x7f60a419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:60148/0 (socket says 172.21.15.5:60148) 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 0x7f60a419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37560/0 (socket says 172.21.15.5:37560) 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f60a37fe700 1 -- 172.21.15.5:0/587516598 learned_addr learned my addr 172.21.15.5:0/587516598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.358+0000 7f609bfff700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a4108f20 0x7f60a419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a37fe700 1 -- 172.21.15.5:0/587516598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a4108f20 msgr2=0x7f60a419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a37fe700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a4108f20 0x7f60a419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a37fe700 1 -- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 msgr2=0x7f60a419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a37fe700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 0x7f60a419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a37fe700 1 -- 172.21.15.5:0/587516598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a419fd20 con 0x7f60a41045f0 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f609bfff700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a4108f20 0x7f60a419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a37fe700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 0x7f60a419aea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6094005910 tx=0x7f609400bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a17fa700 1 -- 172.21.15.5:0/587516598 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094027070 con 0x7f60a41045f0 2024-05-14T15:31:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a419ffb0 con 0x7f60a41045f0 2024-05-14T15:31:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a17fa700 1 -- 172.21.15.5:0/587516598 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609400b610 con 0x7f60a41045f0 2024-05-14T15:31:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60a17fa700 1 -- 172.21.15.5:0/587516598 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940173f0 con 0x7f60a41045f0 2024-05-14T15:31:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.359+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a41a7930 con 0x7f60a41045f0 2024-05-14T15:31:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.362+0000 7f60a17fa700 1 -- 172.21.15.5:0/587516598 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f609402a070 con 0x7f60a41045f0 2024-05-14T15:31:26.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.363+0000 7f60a17fa700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f60900b2a70 0x7f60900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.363+0000 7f609bfff700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f60900b2a70 0x7f60900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.363+0000 7f60a17fa700 1 -- 172.21.15.5:0/587516598 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f60940e3980 con 0x7f60a41045f0 2024-05-14T15:31:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.363+0000 7f609bfff700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f60900b2a70 0x7f60900b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60a4108ea0 tx=0x7f608c009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.363+0000 7f60b8a73b80 1 Processor -- start 2024-05-14T15:31:26.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 -- start start 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742801b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742801ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x557428020880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557427f3d860 con 0x557428020470 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557427edc7c0 con 0x557427e9de40 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557427f39820 con 0x55742801bc60 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742801b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x557428020880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742801b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47428/0 (socket says 172.21.15.5:47428) 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 -- 172.21.15.5:0/3516195587 learned_addr learned my addr 172.21.15.5:0/3516195587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f609bfff700 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742801ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 -- 172.21.15.5:0/3516195587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 msgr2=0x55742801ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742801ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 -- 172.21.15.5:0/3516195587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 msgr2=0x557428020880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x557428020880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a37fe700 1 -- 172.21.15.5:0/3516195587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557427e8a8d0 con 0x557427e9de40 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f609bfff700 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742801ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.364+0000 7f60a3fff700 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x557428020880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.365+0000 7f60a37fe700 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742801b720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60940aaa20 tx=0x7f60940aa460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8ac5464112997c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.365+0000 7f6099ffb700 1 -- 172.21.15.5:0/3516195587 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940aa5a0 con 0x557427e9de40 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.365+0000 7f6099ffb700 1 -- 172.21.15.5:0/3516195587 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60940b36a0 con 0x557427e9de40 2024-05-14T15:31:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.365+0000 7f6099ffb700 1 -- 172.21.15.5:0/3516195587 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940cbaf0 con 0x557427e9de40 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 -- 172.21.15.5:0/3516195587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 msgr2=0x55742801b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742801b720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60940aaa20 tx=0x7f60940aa460 comp rx=0 tx=0).stop 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 -- 172.21.15.5:0/3516195587 shutdown_connections 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x557428020880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742801b720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/3516195587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742801ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 -- 172.21.15.5:0/3516195587 >> 172.21.15.5:0/3516195587 conn(0x557427ee5600 msgr2=0x5574280164b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 -- 172.21.15.5:0/3516195587 shutdown_connections 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 -- 172.21.15.5:0/3516195587 wait complete. 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.366+0000 7f60b8a73b80 1 -- start start 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 Processor -- start 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428025240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x557428025780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574280281e0 con 0x557427e9de40 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557427ea3c30 con 0x55742801bc60 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557428028360 con 0x557428020470 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60b8a73b80 10 client.0 _ll_register_callbacks cb 0x557427ed5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428025240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428025240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37586/0 (socket says 172.21.15.5:37586) 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 -- 172.21.15.5:0/2992449108 learned_addr learned my addr 172.21.15.5:0/2992449108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f609bfff700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x557428025780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a3fff700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 -- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 msgr2=0x557428025cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 -- 172.21.15.5:0/2992449108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 msgr2=0x557428025780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x557428025780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.367+0000 7f60a37fe700 1 -- 172.21.15.5:0/2992449108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557427ed9690 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f60a37fe700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428025240 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f60940cd270 tx=0x7f60940cd2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae96e77f3017a7fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f6082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f6082ffd700 1 -- 172.21.15.5:0/2992449108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940c4070 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f6082ffd700 1 -- 172.21.15.5:0/2992449108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f60940b3420 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60940b36a0 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f6082ffd700 1 -- 172.21.15.5:0/2992449108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f60940aa5a0 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f60b8a73b80 10 client.17010 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574280281e0 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.368+0000 7f6082ffd700 1 -- 172.21.15.5:0/2992449108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60940b36a0 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f6082ffd700 1 -- 172.21.15.5:0/2992449108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f60940abce0 con 0x557427e9de40 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f60b8a73b80 10 client.17010 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f60b8a73b80 10 client.17010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:31:26.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f60b8a73b80 20 client.17010 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:26.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5574280350f0 0x557428037510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f60b8a73b80 4 client.17010 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:26.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557427f4b240 con 0x5574280350f0 2024-05-14T15:31:26.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.369+0000 7f609bfff700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5574280350f0 0x557428037510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.370+0000 7f6082ffd700 10 client.17010 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:26.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.370+0000 7f609bfff700 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5574280350f0 0x557428037510 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31977223d5302093 server_cookie=5bfbf1e4839d3e80 in_seq=0 out_seq=0 2024-05-14T15:31:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f6082ffd700 1 -- 172.21.15.5:0/2992449108 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557427f4b240 con 0x5574280350f0 2024-05-14T15:31:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f6082ffd700 10 client.17010 handle_command_reply: tid=0 2024-05-14T15:31:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 client.17010 shutdown 2024-05-14T15:31:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 msgr2=0x557428025240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428025240 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f60940cd270 tx=0x7f60940cd2a0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 shutdown_connections 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428025240 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5574280350f0 0x557428037510 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2992449108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x557428025780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 >> 172.21.15.5:0/2992449108 conn(0x557427ee5600 msgr2=0x55742801a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f6082ffd700 0 client.17010 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 shutdown_connections 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2992449108 wait complete. 2024-05-14T15:31:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.371+0000 7f60b8a73b80 20 client.17010 trim_cache size 0 max 16384 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.386+0000 7f60b8a73b80 1 Processor -- start 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.386+0000 7f60b8a73b80 1 -- start start 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.386+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428024560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.386+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x55742801fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.386+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.386+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41b4fc0 con 0x557427e9de40 2024-05-14T15:31:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41b2850 con 0x55742801bc60 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41b06c0 con 0x557428020470 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428024560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428024560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37588/0 (socket says 172.21.15.5:37588) 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 -- 172.21.15.5:0/1947614531 learned_addr learned my addr 172.21.15.5:0/1947614531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f609bfff700 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x55742801fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a3fff700 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 -- 172.21.15.5:0/1947614531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 msgr2=0x557428025b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 -- 172.21.15.5:0/1947614531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 msgr2=0x55742801fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x55742801fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 -- 172.21.15.5:0/1947614531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574280237b0 con 0x557427e9de40 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60a37fe700 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428024560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f60940aa740 tx=0x7f60940229c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56c8790dac2e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60997fa700 1 -- 172.21.15.5:0/1947614531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940c4070 con 0x557427e9de40 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60997fa700 1 -- 172.21.15.5:0/1947614531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60940c7030 con 0x557427e9de40 2024-05-14T15:31:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.387+0000 7f60997fa700 1 -- 172.21.15.5:0/1947614531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940cbaf0 con 0x557427e9de40 2024-05-14T15:31:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.388+0000 7f60b8a73b80 1 -- 172.21.15.5:0/1947614531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 msgr2=0x557428024560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.388+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428024560 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f60940aa740 tx=0x7f60940229c0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 -- 172.21.15.5:0/1947614531 shutdown_connections 2024-05-14T15:31:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557428020470 0x557428025b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557427e9de40 0x557428024560 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/1947614531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55742801bc60 0x55742801fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 -- 172.21.15.5:0/1947614531 >> 172.21.15.5:0/1947614531 conn(0x557427ee5600 msgr2=0x557428015be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 -- 172.21.15.5:0/1947614531 shutdown_connections 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 -- 172.21.15.5:0/1947614531 wait complete. 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 -- start start 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.389+0000 7f60b8a73b80 1 Processor -- start 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742803d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742802d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x55742802dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41b0d10 con 0x557428020470 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41b2180 con 0x557427e9de40 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60a41b13e0 con 0x55742801bc60 2024-05-14T15:31:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60b8a73b80 10 client.0 _ll_register_callbacks cb 0x557427ed5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x55742802dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742803d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742802d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742803d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47464/0 (socket says 172.21.15.5:47464) 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x55742802dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37590/0 (socket says 172.21.15.5:37590) 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742802d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:60186/0 (socket says 172.21.15.5:60186) 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a37fe700 1 -- 172.21.15.5:0/2021141964 learned_addr learned my addr 172.21.15.5:0/2021141964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 -- 172.21.15.5:0/2021141964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 msgr2=0x55742803d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742803d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 -- 172.21.15.5:0/2021141964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 msgr2=0x55742802dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x55742802dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f609bfff700 1 -- 172.21.15.5:0/2021141964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742803be40 con 0x55742801bc60 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a3fff700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x55742802dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.390+0000 7f60a37fe700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742803d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f609bfff700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742802d5b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x557428037bb0 tx=0x7f608c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b77f7ff5c958a886 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f6082ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f6082ffd700 1 -- 172.21.15.5:0/2021141964 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c003dc0 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f6082ffd700 1 -- 172.21.15.5:0/2021141964 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f608c02b480 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60940c7030 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f6082ffd700 1 -- 172.21.15.5:0/2021141964 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f608c03b050 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f60b8a73b80 10 client.26611 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55742809ecb0 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.391+0000 7f6082ffd700 1 -- 172.21.15.5:0/2021141964 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60940c7030 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f6082ffd700 1 -- 172.21.15.5:0/2021141964 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f608c028070 con 0x55742801bc60 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60b8a73b80 10 client.26611 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60b8a73b80 10 client.26611 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60b8a73b80 20 client.26611 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55742809f370 0x5574280a1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60b8a73b80 4 client.26611 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:31:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557427f452e0 con 0x55742809f370 2024-05-14T15:31:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60a37fe700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55742809f370 0x5574280a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f6082ffd700 10 client.26611 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.392+0000 7f60a37fe700 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55742809f370 0x5574280a1790 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3692a51e50e31198 server_cookie=605d5e3722350bfd in_seq=0 out_seq=0 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f6082ffd700 1 -- 172.21.15.5:0/2021141964 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557427f452e0 con 0x55742809f370 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f6082ffd700 10 client.26611 handle_command_reply: tid=0 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 client.26611 shutdown 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 msgr2=0x55742802d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742802d5b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x557428037bb0 tx=0x7f608c0047c0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 shutdown_connections 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557428020470 0x55742802dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55742809f370 0x5574280a1790 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557427e9de40 0x55742803d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 --2- 172.21.15.5:0/2021141964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55742801bc60 0x55742802d5b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 >> 172.21.15.5:0/2021141964 conn(0x557427ee5600 msgr2=0x5574280242c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f6082ffd700 0 client.26611 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.393+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 shutdown_connections 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.394+0000 7f60b8a73b80 1 -- 172.21.15.5:0/2021141964 wait complete. 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.394+0000 7f60b8a73b80 20 client.26611 trim_cache size 0 max 16384 2024-05-14T15:31:26.400 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f60900b2a70 msgr2=0x7f60900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f60900b2a70 0x7f60900b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60a4108ea0 tx=0x7f608c009260 comp rx=0 tx=0).stop 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 msgr2=0x7f60a419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 0x7f60a419aea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6094005910 tx=0x7f609400bfc0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 shutdown_connections 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f60900b2a70 0x7f60900b4f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60a4109870 0x7f60a419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60a41045f0 0x7f60a419aea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 --2- 172.21.15.5:0/587516598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a4108f20 0x7f60a419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 >> 172.21.15.5:0/587516598 conn(0x7f60a4100040 msgr2=0x7f60a4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 shutdown_connections 2024-05-14T15:31:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:26.395+0000 7f60aa05f700 1 -- 172.21.15.5:0/587516598 wait complete. 2024-05-14T15:31:27.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 604109 10.67 MB/sec execute 168 sec latency 25.086 ms 2024-05-14T15:31:27.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:31:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:27.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:27 smithi005 ceph-mon[31559]: pgmap v204: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-05-14T15:31:28.143 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605580 10.64 MB/sec execute 169 sec latency 100.182 ms 2024-05-14T15:31:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:27 smithi144 ceph-mon[35650]: pgmap v204: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-05-14T15:31:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:27 smithi193 ceph-mon[36001]: pgmap v204: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-05-14T15:31:29.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 607747 10.66 MB/sec execute 170 sec latency 126.073 ms 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.701+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1667014906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f10 msgr2=0x7fe1cc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.701+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1667014906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f10 0x7fe1cc109320 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099c0 tx=0x7fe1b4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1667014906 shutdown_connections 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1667014906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc109860 0x7fe1cc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1667014906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1667014906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f10 0x7fe1cc109320 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1667014906 >> 172.21.15.5:0/1667014906 conn(0x7fe1cc0ffe70 msgr2=0x7fe1cc1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1667014906 shutdown_connections 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1667014906 wait complete. 2024-05-14T15:31:29.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.702+0000 7fe1d0cc0700 1 Processor -- start 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 -- start start 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc108f10 0x7fe1cc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc109860 0x7fe1cc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc114490 con 0x7fe1cc109860 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc114310 con 0x7fe1cc108f10 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1d0cc0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc114610 con 0x7fe1cc1045e0 2024-05-14T15:31:29.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1ca59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1c1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc108f10 0x7fe1cc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc109860 0x7fe1cc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1ca59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:60194/0 (socket says 172.21.15.5:60194) 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1ca59c700 1 -- 172.21.15.5:0/1279899854 learned_addr learned my addr 172.21.15.5:0/1279899854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.703+0000 7fe1c1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc108f10 0x7fe1cc19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47472/0 (socket says 172.21.15.5:47472) 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1ca59c700 1 -- 172.21.15.5:0/1279899854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc108f10 msgr2=0x7fe1cc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc108f10 0x7fe1cc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1ca59c700 1 -- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc109860 msgr2=0x7fe1cc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc109860 0x7fe1cc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1ca59c700 1 -- 172.21.15.5:0/1279899854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc19fd20 con 0x7fe1cc1045e0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc109860 0x7fe1cc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc19aea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9c0 tx=0x7fe1bc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1c3fff700 1 -- 172.21.15.5:0/1279899854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00cbd0 con 0x7fe1cc1045e0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1c3fff700 1 -- 172.21.15.5:0/1279899854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1bc00ee60 con 0x7fe1cc1045e0 2024-05-14T15:31:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1c3fff700 1 -- 172.21.15.5:0/1279899854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0055c0 con 0x7fe1cc1045e0 2024-05-14T15:31:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc1a0010 con 0x7fe1cc1045e0 2024-05-14T15:31:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.704+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc1a7930 con 0x7fe1cc1045e0 2024-05-14T15:31:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.707+0000 7fe1c3fff700 1 -- 172.21.15.5:0/1279899854 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe1bc004750 con 0x7fe1cc1045e0 2024-05-14T15:31:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.708+0000 7fe1c3fff700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe1b80b29f0 0x7fe1b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.708+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe1b80b29f0 0x7fe1b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.708+0000 7fe1c3fff700 1 -- 172.21.15.5:0/1279899854 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe1bc0e2cc0 con 0x7fe1cc1045e0 2024-05-14T15:31:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe1b80b29f0 0x7fe1b80b4f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 Processor -- start 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 -- start start 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa968e8e40 0x55fa96a70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 0x55fa96a6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a6b220 0x55fa96a6b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa96988940 con 0x55fa96a6b220 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa968edda0 con 0x55fa96a710e0 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.709+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fa96984900 con 0x55fa968e8e40 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 0x55fa96a6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a6b220 0x55fa96a6b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 0x55fa96a6ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47484/0 (socket says 172.21.15.5:47484) 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/683911582 learned_addr learned my addr 172.21.15.5:0/683911582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa968e8e40 0x55fa96a70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/683911582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa968e8e40 msgr2=0x55fa96a70ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa968e8e40 0x55fa96a70ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/683911582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a6b220 msgr2=0x55fa96a6b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a6b220 0x55fa96a6b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/683911582 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa968d58d0 con 0x55fa96a710e0 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.710+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 0x55fa96a6ace0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4005b20 tx=0x7fe1b4009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea300dda48fe931 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.711+0000 7fe1c0d99700 1 -- 172.21.15.5:0/683911582 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4003e20 con 0x55fa96a710e0 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.711+0000 7fe1c0d99700 1 -- 172.21.15.5:0/683911582 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b400bc40 con 0x55fa96a710e0 2024-05-14T15:31:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.711+0000 7fe1c0d99700 1 -- 172.21.15.5:0/683911582 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4025bd0 con 0x55fa96a710e0 2024-05-14T15:31:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.711+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/683911582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 msgr2=0x55fa96a6ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.711+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 0x55fa96a6ace0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4005b20 tx=0x7fe1b4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/683911582 shutdown_connections 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a6b220 0x55fa96a6b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa968e8e40 0x55fa96a70ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/683911582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a710e0 0x55fa96a6ace0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/683911582 >> 172.21.15.5:0/683911582 conn(0x55fa96930630 msgr2=0x55fa96a61a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/683911582 shutdown_connections 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/683911582 wait complete. 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 -- start start 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.712+0000 7fe1df6d4b80 1 Processor -- start 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 0x55fa96a749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a6b220 0x55fa96a74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a710e0 0x55fa96a75420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa96a77b80 con 0x55fa968e8e40 2024-05-14T15:31:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa96941ed0 con 0x55fa96a6b220 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fa96a77d00 con 0x55fa96a710e0 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 0x55fa96a749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 0x55fa96a749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37614/0 (socket says 172.21.15.5:37614) 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 -- 172.21.15.5:0/959829799 learned_addr learned my addr 172.21.15.5:0/959829799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1df6d4b80 10 client.0 _ll_register_callbacks cb 0x55fa96920950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a6b220 0x55fa96a74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a710e0 0x55fa96a75420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 -- 172.21.15.5:0/959829799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a710e0 msgr2=0x55fa96a75420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a710e0 0x55fa96a75420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 -- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a6b220 msgr2=0x55fa96a74ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a6b220 0x55fa96a74ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 -- 172.21.15.5:0/959829799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9693f760 con 0x55fa968e8e40 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a710e0 0x55fa96a75420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a6b220 0x55fa96a74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 0x55fa96a749a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0a6df0 tx=0x7fe1bc0aa0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb9059e2f5c2ce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.713+0000 7fe1b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/959829799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0a9690 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/959829799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe1bc0b2760 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b400bc40 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/959829799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe1bc0aa280 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 10 client.17028 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa96a82850 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/959829799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b400bc40 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/959829799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe1bc0a5580 con 0x55fa968e8e40 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 10 client.17028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 20 client.17028 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96a87200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 4 client.17028 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.714+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa96996320 con 0x55fa96a84de0 2024-05-14T15:31:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.715+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96a87200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.715+0000 7fe1b1ffb700 10 client.17028 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.715+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96a87200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e7e2a44c01ba301 server_cookie=96250912da5bbe0c in_seq=0 out_seq=0 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/959829799 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa96996320 con 0x55fa96a84de0 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1b1ffb700 10 client.17028 handle_command_reply: tid=0 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 client.17028 shutdown 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 msgr2=0x55fa96a749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 0x55fa96a749a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0a6df0 tx=0x7fe1bc0aa0e0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 shutdown_connections 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96a87200 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a710e0 0x55fa96a75420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1b1ffb700 0 client.17028 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa968e8e40 0x55fa96a749a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/959829799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa96a6b220 0x55fa96a74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 >> 172.21.15.5:0/959829799 conn(0x55fa96930630 msgr2=0x55fa96a658c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 shutdown_connections 2024-05-14T15:31:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/959829799 wait complete. 2024-05-14T15:31:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.716+0000 7fe1df6d4b80 20 client.17028 trim_cache size 0 max 16384 2024-05-14T15:31:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 Processor -- start 2024-05-14T15:31:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 -- start start 2024-05-14T15:31:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a75260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a73290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1b0bd0 con 0x55fa96a710e0 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1ae460 con 0x55fa968e8e40 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1ac2d0 con 0x55fa96a6b220 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a73290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1ca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a75260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a80bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:60212/0 (socket says 172.21.15.5:60212) 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/3705141770 learned_addr learned my addr 172.21.15.5:0/3705141770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/3705141770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 msgr2=0x55fa96a75260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a75260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/3705141770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 msgr2=0x55fa96a73290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a73290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.732+0000 7fe1c1d9b700 1 -- 172.21.15.5:0/3705141770 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa96a77240 con 0x55fa96a6b220 2024-05-14T15:31:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.733+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a73290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.733+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a80bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099c0 tx=0x7fe1b4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f7592f382312185 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.733+0000 7fe1b3fff700 1 -- 172.21.15.5:0/3705141770 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4048070 con 0x55fa96a6b220 2024-05-14T15:31:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.733+0000 7fe1b3fff700 1 -- 172.21.15.5:0/3705141770 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4040070 con 0x55fa96a6b220 2024-05-14T15:31:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.733+0000 7fe1b3fff700 1 -- 172.21.15.5:0/3705141770 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4035040 con 0x55fa96a6b220 2024-05-14T15:31:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/3705141770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 msgr2=0x55fa96a80bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a80bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099c0 tx=0x7fe1b4005e80 comp rx=0 tx=0).stop 2024-05-14T15:31:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/3705141770 shutdown_connections 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a73290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a75260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/3705141770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a80bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/3705141770 >> 172.21.15.5:0/3705141770 conn(0x55fa96930630 msgr2=0x55fa96a61410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/3705141770 shutdown_connections 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/3705141770 wait complete. 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 -- start start 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.734+0000 7fe1df6d4b80 1 Processor -- start 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a7d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a8b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1ac920 con 0x55fa96a710e0 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1add90 con 0x55fa968e8e40 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1acff0 con 0x55fa96a6b220 2024-05-14T15:31:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1df6d4b80 10 client.0 _ll_register_callbacks cb 0x55fa96920950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a8b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1ca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a8b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37624/0 (socket says 172.21.15.5:37624) 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 -- 172.21.15.5:0/1786409680 learned_addr learned my addr 172.21.15.5:0/1786409680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1c1d9b700 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a7d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 -- 172.21.15.5:0/1786409680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 msgr2=0x55fa96a7d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a7d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 -- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 msgr2=0x55fa96a7cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a7cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.735+0000 7fe1cad9d700 1 -- 172.21.15.5:0/1786409680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa96a8d400 con 0x55fa96a710e0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1cad9d700 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a8b980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40176a0 tx=0x7fe1c40176d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce2642372b1ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/1786409680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4004bb0 con 0x55fa96a710e0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/1786409680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe1c4004d50 con 0x55fa96a710e0 2024-05-14T15:31:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b4040070 con 0x55fa96a710e0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/1786409680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe1c4016610 con 0x55fa96a710e0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1df6d4b80 10 client.17034 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa96aeeb60 con 0x55fa96a710e0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/1786409680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4040070 con 0x55fa96a710e0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.736+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/1786409680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe1c4021970 con 0x55fa96a710e0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 10 client.17034 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 10 client.17034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 20 client.17034 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96af12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 4 client.17034 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa969903c0 con 0x55fa96a84de0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96af12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1b1ffb700 10 client.17034 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1ca59c700 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96af12e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d21ebdde780f25d6 server_cookie=48088c95326a328e in_seq=0 out_seq=0 2024-05-14T15:31:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1b1ffb700 1 -- 172.21.15.5:0/1786409680 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa969903c0 con 0x55fa96a84de0 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1b1ffb700 10 client.17034 handle_command_reply: tid=0 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.737+0000 7fe1df6d4b80 1 client.17034 shutdown 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 msgr2=0x55fa96a8b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a8b980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40176a0 tx=0x7fe1c40176d0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 shutdown_connections 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fa96a84de0 0x55fa96af12e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa96a710e0 0x55fa96a8b980 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa968e8e40 0x55fa96a7cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1b1ffb700 0 client.17034 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 --2- 172.21.15.5:0/1786409680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa96a6b220 0x55fa96a7d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 >> 172.21.15.5:0/1786409680 conn(0x55fa96930630 msgr2=0x55fa96a73290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 shutdown_connections 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 1 -- 172.21.15.5:0/1786409680 wait complete. 2024-05-14T15:31:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1df6d4b80 20 client.17034 trim_cache size 0 max 16384 2024-05-14T15:31:29.745 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:31:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe1b80b29f0 msgr2=0x7fe1b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe1b80b29f0 0x7fe1b80b4f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c00 tx=0x7fe1b401a040 comp rx=0 tx=0).stop 2024-05-14T15:31:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 msgr2=0x7fe1cc19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc19aea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00e9c0 tx=0x7fe1bc00ecd0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 shutdown_connections 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe1b80b29f0 0x7fe1b80b4f00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc109860 0x7fe1cc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1cc1045e0 0x7fe1cc19aea0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 --2- 172.21.15.5:0/1279899854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1cc108f10 0x7fe1cc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.738+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 >> 172.21.15.5:0/1279899854 conn(0x7fe1cc0ffe70 msgr2=0x7fe1cc107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.739+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 shutdown_connections 2024-05-14T15:31:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:29.739+0000 7fe1d0cc0700 1 -- 172.21.15.5:0/1279899854 wait complete. 2024-05-14T15:31:30.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:29 smithi005 ceph-mon[31559]: pgmap v205: 129 pgs: 129 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-14T15:31:30.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 609485 10.63 MB/sec execute 171 sec latency 33.420 ms 2024-05-14T15:31:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:29 smithi144 ceph-mon[35650]: pgmap v205: 129 pgs: 129 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-14T15:31:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:29 smithi193 ceph-mon[36001]: pgmap v205: 129 pgs: 129 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-14T15:31:30.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:31:30.849 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:31.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 611503 10.64 MB/sec execute 172 sec latency 21.549 ms 2024-05-14T15:31:32.100 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:31 smithi144 ceph-mon[35650]: pgmap v206: 129 pgs: 129 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:31:32.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:31 smithi193 ceph-mon[36001]: pgmap v206: 129 pgs: 129 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:31:32.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 613670 10.65 MB/sec execute 173 sec latency 19.655 ms 2024-05-14T15:31:32.194 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:31 smithi005 ceph-mon[31559]: pgmap v206: 129 pgs: 129 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2024-05-14T15:31:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.564+0000 7fc69eea0700 1 -- 172.21.15.5:0/3769909559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6981034c0 msgr2=0x7fc6981038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.564+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3769909559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6981034c0 0x7fc6981038d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fc6880099c0 tx=0x7fc688009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 -- 172.21.15.5:0/3769909559 shutdown_connections 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3769909559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 0x7fc6981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3769909559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6981034c0 0x7fc6981038d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3769909559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698107df0 0x7fc698108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 -- 172.21.15.5:0/3769909559 >> 172.21.15.5:0/3769909559 conn(0x7fc6980fedc0 msgr2=0x7fc6981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 -- 172.21.15.5:0/3769909559 shutdown_connections 2024-05-14T15:31:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 -- 172.21.15.5:0/3769909559 wait complete. 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 Processor -- start 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.565+0000 7fc69eea0700 1 -- start start 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69eea0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6981034c0 0x7fc69819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69eea0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 0x7fc69819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69eea0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 0x7fc69819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69eea0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981133c0 con 0x7fc698107df0 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69eea0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc698113240 con 0x7fc698108740 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69eea0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc698113540 con 0x7fc6981034c0 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 0x7fc69819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69cc3c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6981034c0 0x7fc69819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 0x7fc69819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37646/0 (socket says 172.21.15.5:37646) 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 -- 172.21.15.5:0/3093876889 learned_addr learned my addr 172.21.15.5:0/3093876889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69d43d700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 0x7fc69819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 -- 172.21.15.5:0/3093876889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6981034c0 msgr2=0x7fc69819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6981034c0 0x7fc69819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 -- 172.21.15.5:0/3093876889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 msgr2=0x7fc69819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 0x7fc69819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 -- 172.21.15.5:0/3093876889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69819fcd0 con 0x7fc698107df0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69d43d700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 0x7fc69819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6981034c0 0x7fc69819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.566+0000 7fc697fff700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 0x7fc69819b3a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00d950 tx=0x7fc68c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.567+0000 7fc695ffb700 1 -- 172.21.15.5:0/3093876889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c016070 con 0x7fc698107df0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.567+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69819ffc0 con 0x7fc698107df0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.567+0000 7fc695ffb700 1 -- 172.21.15.5:0/3093876889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c004830 con 0x7fc698107df0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.567+0000 7fc695ffb700 1 -- 172.21.15.5:0/3093876889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c005450 con 0x7fc698107df0 2024-05-14T15:31:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.567+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a78e0 con 0x7fc698107df0 2024-05-14T15:31:32.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.569+0000 7fc695ffb700 1 -- 172.21.15.5:0/3093876889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc68c00f690 con 0x7fc698107df0 2024-05-14T15:31:32.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.570+0000 7fc695ffb700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc6800b29f0 0x7fc6800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.570+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc6800b29f0 0x7fc6800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.570+0000 7fc695ffb700 1 -- 172.21.15.5:0/3093876889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc68c0e27b0 con 0x7fc698107df0 2024-05-14T15:31:32.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.570+0000 7fc6ad8b4b80 1 Processor -- start 2024-05-14T15:31:32.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 -- start start 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572999e40 0x562572b17720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b1bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc6800b29f0 0x7fc6800b4f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc688000c00 tx=0x7fc68801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562572a39860 con 0x562572999e40 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625729d87c0 con 0x562572b17c60 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562572a35820 con 0x562572b1c470 2024-05-14T15:31:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc69cc3c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572999e40 0x562572b17720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b1bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b1bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47534/0 (socket says 172.21.15.5:47534) 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc69cc3c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572999e40 0x562572b17720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37662/0 (socket says 172.21.15.5:37662) 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 -- 172.21.15.5:0/4237857952 learned_addr learned my addr 172.21.15.5:0/4237857952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc69d43d700 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 -- 172.21.15.5:0/4237857952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 msgr2=0x562572b1c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b1c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 -- 172.21.15.5:0/4237857952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572999e40 msgr2=0x562572b17720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572999e40 0x562572b17720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc697fff700 1 -- 172.21.15.5:0/4237857952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625729868d0 con 0x562572b17c60 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.571+0000 7fc69d43d700 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b1c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.572+0000 7fc697fff700 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b1bf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc68c005cd0 tx=0x7fc68c0a9dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3464facc5187dda8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.572+0000 7fc67effd700 1 -- 172.21.15.5:0/4237857952 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0b2530 con 0x562572b17c60 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.572+0000 7fc67effd700 1 -- 172.21.15.5:0/4237857952 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c0a5cc0 con 0x562572b17c60 2024-05-14T15:31:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.572+0000 7fc67effd700 1 -- 172.21.15.5:0/4237857952 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c004c50 con 0x562572b17c60 2024-05-14T15:31:32.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/4237857952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 msgr2=0x562572b1bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b1bf30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc68c005cd0 tx=0x7fc68c0a9dd0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/4237857952 shutdown_connections 2024-05-14T15:31:32.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b1c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572999e40 0x562572b17720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/4237857952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b1bf30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/4237857952 >> 172.21.15.5:0/4237857952 conn(0x5625729e1600 msgr2=0x562572b124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/4237857952 shutdown_connections 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/4237857952 wait complete. 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.573+0000 7fc6ad8b4b80 1 -- start start 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 Processor -- start 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572999e40 0x562572b21240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b21780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b1c470 0x562572b21cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562572b24420 con 0x562572b1c470 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56257299fc30 con 0x562572b17c60 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562572b245a0 con 0x562572999e40 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc6ad8b4b80 10 client.0 _ll_register_callbacks cb 0x5625729d1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc697fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b21780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc697fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b21780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47546/0 (socket says 172.21.15.5:47546) 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc697fff700 1 -- 172.21.15.5:0/2609884921 learned_addr learned my addr 172.21.15.5:0/2609884921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc69d43d700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b1c470 0x562572b21cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.574+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572999e40 0x562572b21240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc697fff700 1 -- 172.21.15.5:0/2609884921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572999e40 msgr2=0x562572b21240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc697fff700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572999e40 0x562572b21240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc697fff700 1 -- 172.21.15.5:0/2609884921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b1c470 msgr2=0x562572b21cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc697fff700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b1c470 0x562572b21cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc697fff700 1 -- 172.21.15.5:0/2609884921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625729d5690 con 0x562572b17c60 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc697fff700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b21780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0a5e90 tx=0x7fc68c0dfff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf672aa0bea5d920 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc677fff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc677fff700 1 -- 172.21.15.5:0/2609884921 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0b2530 con 0x562572b17c60 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc677fff700 1 -- 172.21.15.5:0/2609884921 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc68c0910d0 con 0x562572b17c60 2024-05-14T15:31:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c0a5cc0 con 0x562572b17c60 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc677fff700 1 -- 172.21.15.5:0/2609884921 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc68c004c50 con 0x562572b17c60 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc6ad8b4b80 10 client.26669 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562572b2f0a0 con 0x562572b17c60 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.575+0000 7fc677fff700 1 -- 172.21.15.5:0/2609884921 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c0a5cc0 con 0x562572b17c60 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc677fff700 1 -- 172.21.15.5:0/2609884921 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc68c0a99c0 con 0x562572b17c60 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc6ad8b4b80 10 client.26669 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc6ad8b4b80 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc6ad8b4b80 20 client.26669 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b31570 0x562572b33990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc6ad8b4b80 4 client.26669 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562572a47240 con 0x562572b31570 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b31570 0x562572b33990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b31570 0x562572b33990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad6717e3e145d80 server_cookie=53746e7eb43449ea in_seq=0 out_seq=0 2024-05-14T15:31:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.576+0000 7fc677fff700 10 client.26669 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc677fff700 1 -- 172.21.15.5:0/2609884921 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562572a47240 con 0x562572b31570 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc677fff700 10 client.26669 handle_command_reply: tid=0 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 client.26669 shutdown 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 msgr2=0x562572b21780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b21780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0a5e90 tx=0x7fc68c0dfff0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 shutdown_connections 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b31570 0x562572b33990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b1c470 0x562572b21cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572999e40 0x562572b21240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc677fff700 0 client.26669 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/2609884921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572b17c60 0x562572b21780 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 >> 172.21.15.5:0/2609884921 conn(0x5625729e1600 msgr2=0x562572b16450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 shutdown_connections 2024-05-14T15:31:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/2609884921 wait complete. 2024-05-14T15:31:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.579+0000 7fc6ad8b4b80 20 client.26669 trim_cache size 0 max 16384 2024-05-14T15:31:32.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 Processor -- start 2024-05-14T15:31:32.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 -- start start 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b0c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b2f90 con 0x562572b17c60 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b0820 con 0x562572999e40 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981ae670 con 0x562572b1c470 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc69cc3c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b0c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b21b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37676/0 (socket says 172.21.15.5:37676) 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 -- 172.21.15.5:0/146266453 learned_addr learned my addr 172.21.15.5:0/146266453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc69d43d700 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 -- 172.21.15.5:0/146266453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 msgr2=0x562572b20240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b20240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 -- 172.21.15.5:0/146266453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 msgr2=0x562572b0c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b0c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc697fff700 1 -- 172.21.15.5:0/146266453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562572b2e390 con 0x562572b17c60 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.595+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b0c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.596+0000 7fc697fff700 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b21b00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0c3040 tx=0x7fc68c0700d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81141691175d08dc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.596+0000 7fc67e7fc700 1 -- 172.21.15.5:0/146266453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a5590 con 0x562572b17c60 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.596+0000 7fc67e7fc700 1 -- 172.21.15.5:0/146266453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c0a5cc0 con 0x562572b17c60 2024-05-14T15:31:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.596+0000 7fc67e7fc700 1 -- 172.21.15.5:0/146266453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a94a0 con 0x562572b17c60 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/146266453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 msgr2=0x562572b21b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b21b00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0c3040 tx=0x7fc68c0700d0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/146266453 shutdown_connections 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b20240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b0c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/146266453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b21b00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/146266453 >> 172.21.15.5:0/146266453 conn(0x5625729e1600 msgr2=0x562572b1fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/146266453 shutdown_connections 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/146266453 wait complete. 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 -- start start 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.597+0000 7fc6ad8b4b80 1 Processor -- start 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b35120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981aece0 con 0x562572b17c60 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b0150 con 0x562572999e40 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981af3b0 con 0x562572b1c470 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b35120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc6ad8b4b80 10 client.0 _ll_register_callbacks cb 0x5625729d1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b35120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37678/0 (socket says 172.21.15.5:37678) 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 -- 172.21.15.5:0/1122104222 learned_addr learned my addr 172.21.15.5:0/1122104222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:32.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc69d43d700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 -- 172.21.15.5:0/1122104222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 msgr2=0x562572b35660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b35660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 -- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 msgr2=0x562572b36d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b36d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc697fff700 1 -- 172.21.15.5:0/1122104222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562572b35ca0 con 0x562572b17c60 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.598+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b36d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc697fff700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b35120 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0b2160 tx=0x7fc68c0b2460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aa677f2c3b5257 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc677fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc677fff700 1 -- 172.21.15.5:0/1122104222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a94a0 con 0x562572b17c60 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc677fff700 1 -- 172.21.15.5:0/1122104222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc68c0a9930 con 0x562572b17c60 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c0a5cc0 con 0x562572b17c60 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc677fff700 1 -- 172.21.15.5:0/1122104222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc68c0a5590 con 0x562572b17c60 2024-05-14T15:31:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc6ad8b4b80 10 client.17052 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562572b35ca0 con 0x562572b17c60 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc677fff700 1 -- 172.21.15.5:0/1122104222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c0a5cc0 con 0x562572b17c60 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc677fff700 1 -- 172.21.15.5:0/1122104222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc68c0ca070 con 0x562572b17c60 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc6ad8b4b80 10 client.17052 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc6ad8b4b80 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.599+0000 7fc6ad8b4b80 20 client.17052 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.600+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b9ab50 0x562572b9cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.600+0000 7fc6ad8b4b80 4 client.17052 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.600+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562572a412e0 con 0x562572b9ab50 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.600+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b9ab50 0x562572b9cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.600+0000 7fc69cc3c700 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b9ab50 0x562572b9cf70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a5565666fafa73 server_cookie=d5366b5b7aae4624 in_seq=0 out_seq=0 2024-05-14T15:31:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.600+0000 7fc677fff700 10 client.17052 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.601+0000 7fc677fff700 1 -- 172.21.15.5:0/1122104222 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562572a412e0 con 0x562572b9ab50 2024-05-14T15:31:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.601+0000 7fc677fff700 10 client.17052 handle_command_reply: tid=0 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.601+0000 7fc6ad8b4b80 1 client.17052 shutdown 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 msgr2=0x562572b35120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b35120 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0b2160 tx=0x7fc68c0b2460 comp rx=0 tx=0).stop 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 shutdown_connections 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562572b9ab50 0x562572b9cf70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562572b1c470 0x562572b35660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562572999e40 0x562572b36d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 --2- 172.21.15.5:0/1122104222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562572b17c60 0x562572b35120 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 >> 172.21.15.5:0/1122104222 conn(0x5625729e1600 msgr2=0x562572b227e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc677fff700 0 client.17052 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 shutdown_connections 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 1 -- 172.21.15.5:0/1122104222 wait complete. 2024-05-14T15:31:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.602+0000 7fc6ad8b4b80 20 client.17052 trim_cache size 0 max 16384 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc6800b29f0 msgr2=0x7fc6800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc6800b29f0 0x7fc6800b4f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc688000c00 tx=0x7fc68801a040 comp rx=0 tx=0).stop 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 msgr2=0x7fc69819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 0x7fc69819b3a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00d950 tx=0x7fc68c00dc60 comp rx=0 tx=0).stop 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 shutdown_connections 2024-05-14T15:31:32.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc6800b29f0 0x7fc6800b4f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc698108740 0x7fc69819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6981034c0 0x7fc69819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 --2- 172.21.15.5:0/3093876889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc698107df0 0x7fc69819b3a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 >> 172.21.15.5:0/3093876889 conn(0x7fc6980fedc0 msgr2=0x7fc69806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 shutdown_connections 2024-05-14T15:31:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:32.603+0000 7fc69eea0700 1 -- 172.21.15.5:0/3093876889 wait complete. 2024-05-14T15:31:33.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 615426 10.64 MB/sec execute 174 sec latency 17.717 ms 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 -- 172.21.15.5:0/2419490137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8108bb0 msgr2=0x7f1ff8102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 --2- 172.21.15.5:0/2419490137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8108bb0 0x7f1ff8102750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009900 tx=0x7f1ff402a410 comp rx=0 tx=0).stop 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 -- 172.21.15.5:0/2419490137 shutdown_connections 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 --2- 172.21.15.5:0/2419490137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff8103300 0x7f1ff810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 --2- 172.21.15.5:0/2419490137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8109050 0x7f1ff8102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 --2- 172.21.15.5:0/2419490137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8108bb0 0x7f1ff8102750 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 -- 172.21.15.5:0/2419490137 >> 172.21.15.5:0/2419490137 conn(0x7f1ff80782e0 msgr2=0x7f1ff8078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 -- 172.21.15.5:0/2419490137 shutdown_connections 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.867+0000 7f2000315640 1 -- 172.21.15.5:0/2419490137 wait complete. 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 Processor -- start 2024-05-14T15:31:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 -- start start 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff8103300 0x7f1ff819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8108bb0 0x7f1ff819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 0x7f1ff819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8112da0 con 0x7f1ff8109050 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8112c20 con 0x7f1ff8108bb0 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f2000315640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8112f20 con 0x7f1ff8103300 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f1ffd8b0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8108bb0 0x7f1ff819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f1ffe8b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 0x7f1ff819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff8103300 0x7f1ff819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f1ffe8b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 0x7f1ff819c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37700/0 (socket says 172.21.15.5:37700) 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f1ffd8b0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8108bb0 0x7f1ff819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47588/0 (socket says 172.21.15.5:47588) 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.868+0000 7f1ffe8b2640 1 -- 172.21.15.5:0/389790309 learned_addr learned my addr 172.21.15.5:0/389790309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1ffe8b2640 1 -- 172.21.15.5:0/389790309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff8103300 msgr2=0x7f1ff819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1ffe8b2640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff8103300 0x7f1ff819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1ffe8b2640 1 -- 172.21.15.5:0/389790309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8108bb0 msgr2=0x7f1ff819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1ffe8b2640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8108bb0 0x7f1ff819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1ffe8b2640 1 -- 172.21.15.5:0/389790309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff81a6a60 con 0x7f1ff8109050 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1ffe8b2640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 0x7f1ff819c1c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000efd0 tx=0x7f1ff000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1fef7fe640 1 -- 172.21.15.5:0/389790309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0003b70 con 0x7f1ff8109050 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1fef7fe640 1 -- 172.21.15.5:0/389790309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff0003d10 con 0x7f1ff8109050 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff81a6d50 con 0x7f1ff8109050 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f1fef7fe640 1 -- 172.21.15.5:0/389790309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00108a0 con 0x7f1ff8109050 2024-05-14T15:31:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.869+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff81a7190 con 0x7f1ff8109050 2024-05-14T15:31:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.872+0000 7f20071c5740 1 Processor -- start 2024-05-14T15:31:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.872+0000 7f1fef7fe640 1 -- 172.21.15.5:0/389790309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1ff0004020 con 0x7f1ff8109050 2024-05-14T15:31:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.872+0000 7f20071c5740 1 -- start start 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f20071c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b18be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f20071c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b19120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f20071c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b19660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f20071c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560954a7e5d0 con 0x56095474f2f0 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f20071c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560954a37890 con 0x5609547781b0 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f20071c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560954a7a590 con 0x5609547574d0 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b19120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b19120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37706/0 (socket says 172.21.15.5:37706) 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/2094948254 learned_addr learned my addr 172.21.15.5:0/2094948254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b19660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b18be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1fef7fe640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1fe40b2950 0x7f1fe40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/2094948254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 msgr2=0x560954b19660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b19660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/2094948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 msgr2=0x560954b18be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b18be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/2094948254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560954790210 con 0x56095474f2f0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b19660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe8b2640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1fe40b2950 0x7f1fe40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1fef7fe640 1 -- 172.21.15.5:0/389790309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ff00e2e30 con 0x7f1ff8109050 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.873+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b19120 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009900 tx=0x7f1ff4032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664ac72dd58b8ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.874+0000 7f1fecbfe640 1 -- 172.21.15.5:0/2094948254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4040070 con 0x56095474f2f0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.874+0000 7f1ffe8b2640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1fe40b2950 0x7f1fe40b4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0015020 tx=0x7f1ff001b2c0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.874+0000 7f1fecbfe640 1 -- 172.21.15.5:0/2094948254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff4004030 con 0x56095474f2f0 2024-05-14T15:31:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.874+0000 7f1fecbfe640 1 -- 172.21.15.5:0/2094948254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4035040 con 0x56095474f2f0 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 -- 172.21.15.5:0/2094948254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 msgr2=0x560954b19120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b19120 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009900 tx=0x7f1ff4032b20 comp rx=0 tx=0).stop 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 -- 172.21.15.5:0/2094948254 shutdown_connections 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b19660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b19120 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 --2- 172.21.15.5:0/2094948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b18be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 -- 172.21.15.5:0/2094948254 >> 172.21.15.5:0/2094948254 conn(0x56095476c300 msgr2=0x560954b15340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 -- 172.21.15.5:0/2094948254 shutdown_connections 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 -- 172.21.15.5:0/2094948254 wait complete. 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 -- start start 2024-05-14T15:31:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.875+0000 7f20071c5740 1 Processor -- start 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b236d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560954b25cb0 con 0x5609547574d0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560954a3b370 con 0x5609547781b0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560954b25e30 con 0x56095474f2f0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 10 client.0 _ll_register_callbacks cb 0x56095478fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffd8b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffd8b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b23190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:60316/0 (socket says 172.21.15.5:60316) 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b236d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37710/0 (socket says 172.21.15.5:37710) 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffd8b0640 1 -- 172.21.15.5:0/335769926 learned_addr learned my addr 172.21.15.5:0/335769926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/335769926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 msgr2=0x560954b23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b23190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 msgr2=0x560954b2f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/335769926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56095485c620 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b236d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x560954b19f90 tx=0x7f1ff402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19990535b3e67f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/335769926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4040070 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/335769926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ff40316a0 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.876+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff4004030 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/335769926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ff4056050 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 10 client.17064 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560954b31fd0 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/335769926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff4004030 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/335769926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1ff4004c00 con 0x5609547574d0 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 10 client.17064 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 20 client.17064 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560954736d80 0x560954b35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 4 client.17064 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560954a8c1e0 con 0x560954736d80 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.877+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560954736d80 0x560954b35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.878+0000 7f1fe1ffb640 10 client.17064 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.878+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560954736d80 0x560954b35330 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3cc0a2cc2337229 server_cookie=cd74bc081ff9d58c in_seq=0 out_seq=0 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/335769926 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560954a8c1e0 con 0x560954736d80 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f1fe1ffb640 10 client.17064 handle_command_reply: tid=0 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 client.17064 shutdown 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 msgr2=0x560954b236d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b236d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x560954b19f90 tx=0x7f1ff402a410 comp rx=0 tx=0).stop 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 shutdown_connections 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x560954736d80 0x560954b35330 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b236d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 --2- 172.21.15.5:0/335769926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f1fe1ffb640 0 client.17064 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 >> 172.21.15.5:0/335769926 conn(0x56095476c300 msgr2=0x560954b1a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 shutdown_connections 2024-05-14T15:31:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 1 -- 172.21.15.5:0/335769926 wait complete. 2024-05-14T15:31:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.879+0000 7f20071c5740 20 client.17064 trim_cache size 0 max 16384 2024-05-14T15:31:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 Processor -- start 2024-05-14T15:31:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 -- start start 2024-05-14T15:31:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b21f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81b3bc0 con 0x56095474f2f0 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81b1490 con 0x5609547781b0 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f20071c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81af2e0 con 0x5609547574d0 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffd8b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b21f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffd8b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b21f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37716/0 (socket says 172.21.15.5:37716) 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffd8b0640 1 -- 172.21.15.5:0/1208129729 learned_addr learned my addr 172.21.15.5:0/1208129729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b2f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/1208129729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 msgr2=0x560954b2f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/1208129729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 msgr2=0x560954b21f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.895+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b21f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.896+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/1208129729 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560954b19fc0 con 0x5609547574d0 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.896+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b21f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.896+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b2f100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4031440 tx=0x7f1ff4009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8de539d95118507 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.896+0000 7f1fe3fff640 1 -- 172.21.15.5:0/1208129729 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4040070 con 0x5609547574d0 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.896+0000 7f1fe3fff640 1 -- 172.21.15.5:0/1208129729 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff404d070 con 0x5609547574d0 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.896+0000 7f1fe3fff640 1 -- 172.21.15.5:0/1208129729 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4035040 con 0x5609547574d0 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 -- 172.21.15.5:0/1208129729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 msgr2=0x560954b2f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b2f100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4031440 tx=0x7f1ff4009520 comp rx=0 tx=0).stop 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 -- 172.21.15.5:0/1208129729 shutdown_connections 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609547574d0 0x560954b2f100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b2f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 --2- 172.21.15.5:0/1208129729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56095474f2f0 0x560954b21f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 -- 172.21.15.5:0/1208129729 >> 172.21.15.5:0/1208129729 conn(0x56095476c300 msgr2=0x560954b21850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 -- 172.21.15.5:0/1208129729 shutdown_connections 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 -- 172.21.15.5:0/1208129729 wait complete. 2024-05-14T15:31:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 -- start start 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.897+0000 7f20071c5740 1 Processor -- start 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b95490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81af950 con 0x5609547574d0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81b0dc0 con 0x5609547781b0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81b0020 con 0x56095474f2f0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 10 client.0 _ll_register_callbacks cb 0x7f1fe8009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b35d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37722/0 (socket says 172.21.15.5:37722) 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/3437715977 learned_addr learned my addr 172.21.15.5:0/3437715977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b95490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/3437715977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 msgr2=0x560954b357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 msgr2=0x560954b95490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b95490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 -- 172.21.15.5:0/3437715977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560954b95a00 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1ffe0b1640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b35d30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4000c00 tx=0x7f1ff4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6cc6d955341e6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/3437715977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4040070 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/3437715977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ff4031790 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.898+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff404d070 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/3437715977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ff4070050 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 10 client.17070 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560954b9ccf0 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/3437715977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff404d070 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/3437715977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1ff406b030 con 0x5609547574d0 2024-05-14T15:31:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 10 client.17070 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 10 client.17070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 20 client.17070 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56095470c1c0 0x560954b9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 4 client.17070 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560954a86050 con 0x56095470c1c0 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.899+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56095470c1c0 0x560954b9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.900+0000 7f1ffd8b0640 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56095470c1c0 0x560954b9f3e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26505f513402ab03 server_cookie=deb94599302f1fd2 in_seq=0 out_seq=0 2024-05-14T15:31:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:33.900+0000 7f1fe1ffb640 10 client.17070 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:34.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 617155 10.63 MB/sec execute 175 sec latency 146.641 ms 2024-05-14T15:31:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f1fe1ffb640 1 -- 172.21.15.5:0/3437715977 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560954a86050 con 0x56095470c1c0 2024-05-14T15:31:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f1fe1ffb640 10 client.17070 handle_command_reply: tid=0 2024-05-14T15:31:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 client.17070 shutdown 2024-05-14T15:31:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 msgr2=0x560954b35d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b35d30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4000c00 tx=0x7f1ff4003820 comp rx=0 tx=0).stop 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 shutdown_connections 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56095470c1c0 0x560954b9f3e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609547781b0 0x560954b95490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609547574d0 0x560954b35d30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 --2- 172.21.15.5:0/3437715977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56095474f2f0 0x560954b357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 >> 172.21.15.5:0/3437715977 conn(0x56095476c300 msgr2=0x560954b21fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f1fe1ffb640 0 client.17070 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 shutdown_connections 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.187+0000 7f20071c5740 1 -- 172.21.15.5:0/3437715977 wait complete. 2024-05-14T15:31:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.188+0000 7f20071c5740 20 client.17070 trim_cache size 0 max 16384 2024-05-14T15:31:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:31:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:31:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:31:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:31:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.188+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1fe40b2950 msgr2=0x7f1fe40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.188+0000 7f2000315640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1fe40b2950 0x7f1fe40b4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0015020 tx=0x7f1ff001b2c0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.189+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 msgr2=0x7f1ff819c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.189+0000 7f2000315640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 0x7f1ff819c1c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000efd0 tx=0x7f1ff000c6a0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 shutdown_connections 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1fe40b2950 0x7f1fe40b4e60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff8109050 0x7f1ff819c1c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8108bb0 0x7f1ff819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 --2- 172.21.15.5:0/389790309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff8103300 0x7f1ff819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 >> 172.21.15.5:0/389790309 conn(0x7f1ff80782e0 msgr2=0x7f1ff8104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 shutdown_connections 2024-05-14T15:31:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:34.190+0000 7f2000315640 1 -- 172.21.15.5:0/389790309 wait complete. 2024-05-14T15:31:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:33 smithi144 ceph-mon[35650]: pgmap v207: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-05-14T15:31:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:33 smithi193 ceph-mon[36001]: pgmap v207: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-05-14T15:31:34.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:33 smithi005 ceph-mon[31559]: pgmap v207: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-05-14T15:31:34.325 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:31:34.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:31:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:35.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 618786 10.62 MB/sec execute 176 sec latency 20.820 ms 2024-05-14T15:31:36.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:35 smithi005 ceph-mon[31559]: pgmap v208: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-05-14T15:31:36.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 621180 10.63 MB/sec execute 177 sec latency 41.052 ms 2024-05-14T15:31:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:35 smithi144 ceph-mon[35650]: pgmap v208: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 -- 172.21.15.5:0/1127850331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c1034c0 msgr2=0x7f2f8c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 --2- 172.21.15.5:0/1127850331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c1038d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0099c0 tx=0x7f2f7c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 -- 172.21.15.5:0/1127850331 shutdown_connections 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 --2- 172.21.15.5:0/1127850331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c108740 0x7f2f8c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 --2- 172.21.15.5:0/1127850331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c1038d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 --2- 172.21.15.5:0/1127850331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c107df0 0x7f2f8c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 -- 172.21.15.5:0/1127850331 >> 172.21.15.5:0/1127850331 conn(0x7f2f8c0fedc0 msgr2=0x7f2f8c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 -- 172.21.15.5:0/1127850331 shutdown_connections 2024-05-14T15:31:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.300+0000 7f2f90e29700 1 -- 172.21.15.5:0/1127850331 wait complete. 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.301+0000 7f2f90e29700 1 Processor -- start 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.301+0000 7f2f90e29700 1 -- start start 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f90e29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f90e29700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 0x7f2f8c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f90e29700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c108740 0x7f2f8c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f90e29700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c113370 con 0x7f2f8c107df0 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f90e29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1131f0 con 0x7f2f8c1034c0 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f90e29700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1134f0 con 0x7f2f8c108740 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 0x7f2f8c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 0x7f2f8c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39556/0 (socket says 172.21.15.5:39556) 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 -- 172.21.15.5:0/449678658 learned_addr learned my addr 172.21.15.5:0/449678658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c108740 0x7f2f8c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 -- 172.21.15.5:0/449678658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c108740 msgr2=0x7f2f8c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c108740 0x7f2f8c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 -- 172.21.15.5:0/449678658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c1034c0 msgr2=0x7f2f8c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 -- 172.21.15.5:0/449678658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c19fc80 con 0x7f2f8c107df0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.302+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 0x7f2f8c19b350 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400d980 tx=0x7f2f7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.303+0000 7f2f83fff700 1 -- 172.21.15.5:0/449678658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74017070 con 0x7f2f8c107df0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.303+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c19ff70 con 0x7f2f8c107df0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.303+0000 7f2f83fff700 1 -- 172.21.15.5:0/449678658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f7400fd40 con 0x7f2f8c107df0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.303+0000 7f2f83fff700 1 -- 172.21.15.5:0/449678658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400e3b0 con 0x7f2f8c107df0 2024-05-14T15:31:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.303+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c1a7890 con 0x7f2f8c107df0 2024-05-14T15:31:36.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:35 smithi193 ceph-mon[36001]: pgmap v208: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-05-14T15:31:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.306+0000 7f2f83fff700 1 -- 172.21.15.5:0/449678658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2f74010610 con 0x7f2f8c107df0 2024-05-14T15:31:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.306+0000 7f2f83fff700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f780b29f0 0x7f2f780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 Processor -- start 2024-05-14T15:31:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f780b29f0 0x7f2f780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f83fff700 1 -- 172.21.15.5:0/449678658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2f740e2450 con 0x7f2f8c107df0 2024-05-14T15:31:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 -- start start 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad90950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d7acb29c0 con 0x555d7ad96e00 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d7ac17e20 con 0x555d7ad90e90 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.307+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555d7acae980 con 0x555d7ac12ec0 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad90950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad90950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39568/0 (socket says 172.21.15.5:39568) 2024-05-14T15:31:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 -- 172.21.15.5:0/1386004064 learned_addr learned my addr 172.21.15.5:0/1386004064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f780b29f0 0x7f2f780b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c000c00 tx=0x7f2f7c01a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 -- 172.21.15.5:0/1386004064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 msgr2=0x555d7ad968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 -- 172.21.15.5:0/1386004064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 msgr2=0x555d7ad912a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad912a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 -- 172.21.15.5:0/1386004064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7abff950 con 0x555d7ad96e00 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.308+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad90950 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f740a9070 tx=0x7f2f740a8e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ae5ac0d4d75294 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.309+0000 7f2f80d99700 1 -- 172.21.15.5:0/1386004064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74020070 con 0x555d7ad96e00 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.309+0000 7f2f80d99700 1 -- 172.21.15.5:0/1386004064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f740ad720 con 0x555d7ad96e00 2024-05-14T15:31:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.309+0000 7f2f80d99700 1 -- 172.21.15.5:0/1386004064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f740a85d0 con 0x555d7ad96e00 2024-05-14T15:31:36.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1386004064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 msgr2=0x555d7ad90950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad90950 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f740a9070 tx=0x7f2f740a8e80 comp rx=0 tx=0).stop 2024-05-14T15:31:36.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1386004064 shutdown_connections 2024-05-14T15:31:36.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad912a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1386004064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad90950 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1386004064 >> 172.21.15.5:0/1386004064 conn(0x555d7ac5a6b0 msgr2=0x555d7ad8ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1386004064 shutdown_connections 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1386004064 wait complete. 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 -- start start 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.310+0000 7f2f9f83db80 1 Processor -- start 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad9a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d7ad9d8a0 con 0x555d7ad96e00 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d7ac6bf50 con 0x555d7ad90e90 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555d7ad9da20 con 0x555d7ac12ec0 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f9f83db80 10 client.0 _ll_register_callbacks cb 0x555d7ac4a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:36.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad9a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad9a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:42110/0 (socket says 172.21.15.5:42110) 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 -- 172.21.15.5:0/65899645 learned_addr learned my addr 172.21.15.5:0/65899645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 -- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 msgr2=0x555d7ad9aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 -- 172.21.15.5:0/65899645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 msgr2=0x555d7ad9b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8a59c700 1 -- 172.21.15.5:0/65899645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7ac697e0 con 0x555d7ac12ec0 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.311+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad9a5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009fd0 tx=0x7f2f7c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99dd28cca6083e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:36.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f71ffb700 1 -- 172.21.15.5:0/65899645 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c034060 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f71ffb700 1 -- 172.21.15.5:0/65899645 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2f7c003db0 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f740ad720 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f71ffb700 1 -- 172.21.15.5:0/65899645 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c00be60 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 10 client.26677 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d7ada8500 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f71ffb700 1 -- 172.21.15.5:0/65899645 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f740ad720 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f71ffb700 1 -- 172.21.15.5:0/65899645 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2f7c0255e0 con 0x555d7ac12ec0 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 10 client.26677 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 20 client.26677 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.312+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7adaab40 0x555d7adacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.313+0000 7f2f9f83db80 4 client.26677 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:36.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.313+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d7acc03a0 con 0x555d7adaab40 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.313+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7adaab40 0x555d7adacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.313+0000 7f2f71ffb700 10 client.26677 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.313+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7adaab40 0x555d7adacf60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e7c4648281a081 server_cookie=a26a8f317a166c99 in_seq=0 out_seq=0 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f71ffb700 1 -- 172.21.15.5:0/65899645 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d7acc03a0 con 0x555d7adaab40 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f71ffb700 10 client.26677 handle_command_reply: tid=0 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 client.26677 shutdown 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 msgr2=0x555d7ad9a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad9a5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009fd0 tx=0x7f2f7c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 shutdown_connections 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7adaab40 0x555d7adacf60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad9a5a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f71ffb700 0 client.26677 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/65899645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 >> 172.21.15.5:0/65899645 conn(0x555d7ac5a6b0 msgr2=0x555d7ad8f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 shutdown_connections 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.314+0000 7f2f9f83db80 1 -- 172.21.15.5:0/65899645 wait complete. 2024-05-14T15:31:36.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.315+0000 7f2f9f83db80 20 client.26677 trim_cache size 0 max 16384 2024-05-14T15:31:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.338+0000 7f2f9f83db80 1 Processor -- start 2024-05-14T15:31:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.338+0000 7f2f9f83db80 1 -- start start 2024-05-14T15:31:36.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.338+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad90950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1b2ef0 con 0x555d7ad96e00 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1b0780 con 0x555d7ad90e90 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1ae5d0 con 0x555d7ac12ec0 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad90950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39580/0 (socket says 172.21.15.5:39580) 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 -- 172.21.15.5:0/1486736025 learned_addr learned my addr 172.21.15.5:0/1486736025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad90950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:42120/0 (socket says 172.21.15.5:42120) 2024-05-14T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48960/0 (socket says 172.21.15.5:48960) 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 -- 172.21.15.5:0/1486736025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 msgr2=0x555d7ad90950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad90950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 -- 172.21.15.5:0/1486736025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 msgr2=0x555d7ad9c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 -- 172.21.15.5:0/1486736025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7ad8bfc0 con 0x555d7ad96e00 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad90950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.339+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9ae60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f84011cf0 tx=0x7f2f84009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a92290f594526aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.340+0000 7f2f73fff700 1 -- 172.21.15.5:0/1486736025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84003a00 con 0x555d7ad96e00 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.340+0000 7f2f73fff700 1 -- 172.21.15.5:0/1486736025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f84017e50 con 0x555d7ad96e00 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.340+0000 7f2f73fff700 1 -- 172.21.15.5:0/1486736025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84004790 con 0x555d7ad96e00 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1486736025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 msgr2=0x555d7ad9ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9ae60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f84011cf0 tx=0x7f2f84009920 comp rx=0 tx=0).stop 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1486736025 shutdown_connections 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad96e00 0x555d7ad9ae60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ad90950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1486736025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad90e90 0x555d7ad9c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1486736025 >> 172.21.15.5:0/1486736025 conn(0x555d7ac5a6b0 msgr2=0x555d7ad99500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1486736025 shutdown_connections 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1486736025 wait complete. 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 -- start start 2024-05-14T15:31:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.341+0000 7f2f9f83db80 1 Processor -- start 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ada2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad90e90 0x555d7ada31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad96e00 0x555d7ad9fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1aec40 con 0x555d7ad90e90 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1b00b0 con 0x555d7ad96e00 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1af310 con 0x555d7ac12ec0 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f9f83db80 10 client.0 _ll_register_callbacks cb 0x555d7ac4a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad90e90 0x555d7ada31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad90e90 0x555d7ada31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39592/0 (socket says 172.21.15.5:39592) 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ada2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f81d9b700 1 -- 172.21.15.5:0/1455730496 learned_addr learned my addr 172.21.15.5:0/1455730496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8ad9d700 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad96e00 0x555d7ad9fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 -- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad96e00 msgr2=0x555d7ad9fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad96e00 0x555d7ad9fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 -- 172.21.15.5:0/1455730496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad90e90 msgr2=0x555d7ada31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad90e90 0x555d7ada31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 -- 172.21.15.5:0/1455730496 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7ada03c0 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f8a59c700 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ada2cb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c025b70 tx=0x7f2f7c010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d047fcb88b24dcf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.342+0000 7f2f71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f71ffb700 1 -- 172.21.15.5:0/1455730496 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c034040 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f71ffb700 1 -- 172.21.15.5:0/1455730496 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2f7c010b20 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f84017e50 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f71ffb700 1 -- 172.21.15.5:0/1455730496 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c010d60 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 10 client.26683 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555d7ae15ca0 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f71ffb700 1 -- 172.21.15.5:0/1455730496 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f84017e50 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f71ffb700 1 -- 172.21.15.5:0/1455730496 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2f7c018400 con 0x555d7ac12ec0 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 10 client.26683 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:36.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 20 client.26683 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7ae16280 0x555d7ae186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 4 client.26683 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:31:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.343+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d7acba440 con 0x555d7ae16280 2024-05-14T15:31:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.344+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7ae16280 0x555d7ae186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.344+0000 7f2f81d9b700 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7ae16280 0x555d7ae186a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42862fc9653ef5ef server_cookie=ee82d16662fbc1eb in_seq=0 out_seq=0 2024-05-14T15:31:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.344+0000 7f2f71ffb700 10 client.26683 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.345+0000 7f2f71ffb700 1 -- 172.21.15.5:0/1455730496 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d7acba440 con 0x555d7ae16280 2024-05-14T15:31:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.345+0000 7f2f71ffb700 10 client.26683 handle_command_reply: tid=0 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.345+0000 7f2f9f83db80 1 client.26683 shutdown 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.345+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 msgr2=0x555d7ada2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.345+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ada2cb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c025b70 tx=0x7f2f7c010720 comp rx=0 tx=0).stop 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 shutdown_connections 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555d7ae16280 0x555d7ae186a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d7ad96e00 0x555d7ad9fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555d7ac12ec0 0x555d7ada2cb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 --2- 172.21.15.5:0/1455730496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7ad90e90 0x555d7ada31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 >> 172.21.15.5:0/1455730496 conn(0x555d7ac5a6b0 msgr2=0x555d7ad9b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f71ffb700 0 client.26683 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 shutdown_connections 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 1 -- 172.21.15.5:0/1455730496 wait complete. 2024-05-14T15:31:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.346+0000 7f2f9f83db80 20 client.26683 trim_cache size 0 max 16384 2024-05-14T15:31:36.352 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:31:36.352 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:31:36.352 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "54ac9d6f-c9e2-42c8-a5c1-6526820402e3", 2024-05-14T15:31:36.352 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:31:36.352 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f780b29f0 msgr2=0x7f2f780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f780b29f0 0x7f2f780b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c000c00 tx=0x7f2f7c01a040 comp rx=0 tx=0).stop 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 msgr2=0x7f2f8c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 0x7f2f8c19b350 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400d980 tx=0x7f2f7400dc90 comp rx=0 tx=0).stop 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 shutdown_connections 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f780b29f0 0x7f2f780b4f00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f8c108740 0x7f2f8c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f8c1034c0 0x7f2f8c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 --2- 172.21.15.5:0/449678658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c107df0 0x7f2f8c19b350 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.347+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 >> 172.21.15.5:0/449678658 conn(0x7f2f8c0fedc0 msgr2=0x7f2f8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:36.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.348+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 shutdown_connections 2024-05-14T15:31:36.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:36.348+0000 7f2f90e29700 1 -- 172.21.15.5:0/449678658 wait complete. 2024-05-14T15:31:37.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:31:37.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 622714 10.62 MB/sec execute 178 sec latency 19.756 ms 2024-05-14T15:31:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:36 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:31:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:36 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:36 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:31:37.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:36 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:37.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:36 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:31:37.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:36 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:37.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:36 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:31:37.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:36 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:37.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:36 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:31:37.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:36 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:37.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:36 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:31:37.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:36 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:31:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:31:38.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:37 smithi005 ceph-mon[31559]: pgmap v209: 129 pgs: 129 active+clean; 77 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-05-14T15:31:38.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 625230 10.65 MB/sec execute 179 sec latency 16.037 ms 2024-05-14T15:31:38.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:37 smithi144 ceph-mon[35650]: pgmap v209: 129 pgs: 129 active+clean; 77 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-05-14T15:31:38.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:37 smithi193 ceph-mon[36001]: pgmap v209: 129 pgs: 129 active+clean; 77 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-05-14T15:31:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 -- 172.21.15.5:0/1803492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364108780 msgr2=0x7f43641100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 --2- 172.21.15.5:0/1803492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364108780 0x7f43641100e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b0d0 tx=0x7f435c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 -- 172.21.15.5:0/1803492997 shutdown_connections 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 --2- 172.21.15.5:0/1803492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364108780 0x7f43641100e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 --2- 172.21.15.5:0/1803492997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364103550 0x7f4364103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 --2- 172.21.15.5:0/1803492997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364107e30 0x7f4364108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 -- 172.21.15.5:0/1803492997 >> 172.21.15.5:0/1803492997 conn(0x7f4364078e00 msgr2=0x7f4364079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 -- 172.21.15.5:0/1803492997 shutdown_connections 2024-05-14T15:31:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.060+0000 7f436c399700 1 -- 172.21.15.5:0/1803492997 wait complete. 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.061+0000 7f436c399700 1 Processor -- start 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.061+0000 7f436c399700 1 -- start start 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436c399700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 0x7f436419d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436c399700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364107e30 0x7f436419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436c399700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364108780 0x7f43641a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436c399700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4364112310 con 0x7f4364103550 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436c399700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4364112190 con 0x7f4364107e30 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436c399700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4364112490 con 0x7f4364108780 2024-05-14T15:31:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 0x7f436419d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f4369934700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364107e30 0x7f436419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 0x7f436419d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39616/0 (socket says 172.21.15.5:39616) 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 -- 172.21.15.5:0/1210022115 learned_addr learned my addr 172.21.15.5:0/1210022115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a936700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364108780 0x7f43641a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 -- 172.21.15.5:0/1210022115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364108780 msgr2=0x7f43641a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364108780 0x7f43641a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 -- 172.21.15.5:0/1210022115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364107e30 msgr2=0x7f436419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364107e30 0x7f436419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 -- 172.21.15.5:0/1210022115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641a1f40 con 0x7f4364103550 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a936700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364108780 0x7f43641a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.062+0000 7f436a135700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 0x7f436419d070 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f436000e9a0 tx=0x7f436000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.063+0000 7f435b7fe700 1 -- 172.21.15.5:0/1210022115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436000ca10 con 0x7f4364103550 2024-05-14T15:31:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.063+0000 7f435b7fe700 1 -- 172.21.15.5:0/1210022115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f436000cbb0 con 0x7f4364103550 2024-05-14T15:31:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.063+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43641a2230 con 0x7f4364103550 2024-05-14T15:31:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.063+0000 7f435b7fe700 1 -- 172.21.15.5:0/1210022115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360018680 con 0x7f4364103550 2024-05-14T15:31:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.063+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43641a9b50 con 0x7f4364103550 2024-05-14T15:31:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.067+0000 7f435b7fe700 1 -- 172.21.15.5:0/1210022115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4360018820 con 0x7f4364103550 2024-05-14T15:31:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.068+0000 7f437adadb80 1 Processor -- start 2024-05-14T15:31:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.068+0000 7f437adadb80 1 -- start start 2024-05-14T15:31:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.068+0000 7f437adadb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.068+0000 7f437adadb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.068+0000 7f437adadb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f437adadb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dec92dd940 con 0x55dec93bbe10 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f437adadb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dec9242da0 con 0x55dec93c1d80 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f437adadb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dec92d9900 con 0x55dec923de40 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a135700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f4369934700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a936700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a936700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:42156/0 (socket says 172.21.15.5:42156) 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f4369934700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93bc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39628/0 (socket says 172.21.15.5:39628) 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a936700 1 -- 172.21.15.5:0/1284641694 learned_addr learned my addr 172.21.15.5:0/1284641694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f435b7fe700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43500b29e0 0x7f43500b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a936700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43500b29e0 0x7f43500b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a135700 1 -- 172.21.15.5:0/1284641694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 msgr2=0x55dec93c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a135700 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a135700 1 -- 172.21.15.5:0/1284641694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 msgr2=0x55dec93bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a135700 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f435b7fe700 1 -- 172.21.15.5:0/1210022115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f43600e2c60 con 0x7f4364103550 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.069+0000 7f436a135700 1 -- 172.21.15.5:0/1284641694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec922a8d0 con 0x55dec93c1d80 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.070+0000 7f436a135700 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93bb8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43600dff90 tx=0x7f4360010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499162f929ad5909 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.070+0000 7f436a936700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43500b29e0 0x7f43500b4ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f435c005b40 tx=0x7f435c00b3e0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.070+0000 7f4358d47700 1 -- 172.21.15.5:0/1284641694 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43600a9630 con 0x55dec93c1d80 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.070+0000 7f4358d47700 1 -- 172.21.15.5:0/1284641694 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43600aa220 con 0x55dec93c1d80 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.070+0000 7f4358d47700 1 -- 172.21.15.5:0/1284641694 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360018b10 con 0x55dec93c1d80 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 -- 172.21.15.5:0/1284641694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 msgr2=0x55dec93bb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93bb8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f43600dff90 tx=0x7f4360010c10 comp rx=0 tx=0).stop 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 -- 172.21.15.5:0/1284641694 shutdown_connections 2024-05-14T15:31:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 --2- 172.21.15.5:0/1284641694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93bb8d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 -- 172.21.15.5:0/1284641694 >> 172.21.15.5:0/1284641694 conn(0x55dec9285630 msgr2=0x55dec93b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 -- 172.21.15.5:0/1284641694 shutdown_connections 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 -- 172.21.15.5:0/1284641694 wait complete. 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 -- start start 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.071+0000 7f437adadb80 1 Processor -- start 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dec93c8760 con 0x55dec93bbe10 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dec9296ed0 con 0x55dec93c1d80 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dec93c88e0 con 0x55dec923de40 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f437adadb80 10 client.0 _ll_register_callbacks cb 0x55dec9275950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f4369934700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:42168/0 (socket says 172.21.15.5:42168) 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 -- 172.21.15.5:0/3960178914 learned_addr learned my addr 172.21.15.5:0/3960178914 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a936700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 -- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 msgr2=0x55dec93c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 -- 172.21.15.5:0/3960178914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 msgr2=0x55dec93c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 -- 172.21.15.5:0/3960178914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec9294760 con 0x55dec923de40 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f4369934700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f436a135700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c5410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c00 tx=0x7f43600a5770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ddcd04b206c3f34 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.072+0000 7f434dffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f434dffb700 1 -- 172.21.15.5:0/3960178914 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43600a9630 con 0x55dec923de40 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43600aa220 con 0x55dec923de40 2024-05-14T15:31:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f434dffb700 1 -- 172.21.15.5:0/3960178914 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f43600a9f90 con 0x55dec923de40 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f434dffb700 1 -- 172.21.15.5:0/3960178914 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f43600a9b20 con 0x55dec923de40 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 10 client.26695 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dec93c8760 con 0x55dec923de40 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f434dffb700 1 -- 172.21.15.5:0/3960178914 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43600aa220 con 0x55dec923de40 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f434dffb700 1 -- 172.21.15.5:0/3960178914 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f43600600c0 con 0x55dec923de40 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 10 client.26695 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 20 client.26695 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec93d55b0 0x55dec93d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 4 client.26695 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.073+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec92eb320 con 0x55dec93d55b0 2024-05-14T15:31:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.074+0000 7f4369934700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec93d55b0 0x55dec93d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.074+0000 7f4369934700 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec93d55b0 0x55dec93d79d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63abcdba39f1543f server_cookie=ba0472b48bfe51ac in_seq=0 out_seq=0 2024-05-14T15:31:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.074+0000 7f434dffb700 10 client.26695 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f434dffb700 1 -- 172.21.15.5:0/3960178914 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dec92eb320 con 0x55dec93d55b0 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f434dffb700 10 client.26695 handle_command_reply: tid=0 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 client.26695 shutdown 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 msgr2=0x55dec93c5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c5410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c00 tx=0x7f43600a5770 comp rx=0 tx=0).stop 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 shutdown_connections 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec93d55b0 0x55dec93d79d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f434dffb700 0 client.26695 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec93c1d80 0x55dec93c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec923de40 0x55dec93c5410 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 --2- 172.21.15.5:0/3960178914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93bbe10 0x55dec93c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.075+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 >> 172.21.15.5:0/3960178914 conn(0x55dec9285630 msgr2=0x55dec93ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.076+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 shutdown_connections 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.076+0000 7f437adadb80 1 -- 172.21.15.5:0/3960178914 wait complete. 2024-05-14T15:31:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.076+0000 7f437adadb80 20 client.26695 trim_cache size 0 max 16384 2024-05-14T15:31:39.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.100+0000 7f437adadb80 1 Processor -- start 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 -- start start 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43641b5130 con 0x55dec93c1d80 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43641b29c0 con 0x55dec923de40 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f437adadb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43641b0810 con 0x55dec93bbe10 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39652/0 (socket says 172.21.15.5:39652) 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 -- 172.21.15.5:0/3432926982 learned_addr learned my addr 172.21.15.5:0/3432926982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a135700 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f4369934700 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 -- 172.21.15.5:0/3432926982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 msgr2=0x55dec93bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 -- 172.21.15.5:0/3432926982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 msgr2=0x55dec93bb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93bb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f436a936700 1 -- 172.21.15.5:0/3432926982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec93b6f40 con 0x55dec93c1d80 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.101+0000 7f4369934700 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.102+0000 7f436a936700 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93c5cd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f435c007a10 tx=0x7f435c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df630adc41336ede server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.102+0000 7f434ffff700 1 -- 172.21.15.5:0/3432926982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c025040 con 0x55dec93c1d80 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.102+0000 7f434ffff700 1 -- 172.21.15.5:0/3432926982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435c004c10 con 0x55dec93c1d80 2024-05-14T15:31:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.102+0000 7f434ffff700 1 -- 172.21.15.5:0/3432926982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c01b690 con 0x55dec93c1d80 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 -- 172.21.15.5:0/3432926982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 msgr2=0x55dec93c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93c5cd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f435c007a10 tx=0x7f435c00be90 comp rx=0 tx=0).stop 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 -- 172.21.15.5:0/3432926982 shutdown_connections 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93c5cd0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93bb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 --2- 172.21.15.5:0/3432926982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 -- 172.21.15.5:0/3432926982 >> 172.21.15.5:0/3432926982 conn(0x55dec9285630 msgr2=0x55dec93b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 -- 172.21.15.5:0/3432926982 shutdown_connections 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.103+0000 7f437adadb80 1 -- 172.21.15.5:0/3432926982 wait complete. 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 -- start start 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 Processor -- start 2024-05-14T15:31:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43641b0e80 con 0x55dec93c1d80 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43641b22f0 con 0x55dec923de40 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43641b1550 con 0x55dec93bbe10 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f437adadb80 10 client.0 _ll_register_callbacks cb 0x55dec9275950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f436a936700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f436a135700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f436a936700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93dca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39660/0 (socket says 172.21.15.5:39660) 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f436a936700 1 -- 172.21.15.5:0/36283417 learned_addr learned my addr 172.21.15.5:0/36283417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.104+0000 7f4369934700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a936700 1 -- 172.21.15.5:0/36283417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 msgr2=0x55dec93dc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a936700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93dc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a936700 1 -- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 msgr2=0x55dec93cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a936700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a936700 1 -- 172.21.15.5:0/36283417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec93dcf40 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a135700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f436a936700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93dca00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f435c009900 tx=0x7f435c004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d727fc87ab95df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f434dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f434dffb700 1 -- 172.21.15.5:0/36283417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c025070 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f435c004c10 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f434dffb700 1 -- 172.21.15.5:0/36283417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f435c0051d0 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f434dffb700 1 -- 172.21.15.5:0/36283417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f435c042050 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f437adadb80 10 client.17106 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.105+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dec943f960 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f434dffb700 1 -- 172.21.15.5:0/36283417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f435c004c10 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f434dffb700 1 -- 172.21.15.5:0/36283417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f435c011070 con 0x55dec93c1d80 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f437adadb80 10 client.17106 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f437adadb80 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f437adadb80 20 client.17106 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f437adadb80 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec943ff60 0x55dec9442380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f437adadb80 4 client.17106 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dec92e53c0 con 0x55dec943ff60 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f436a135700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec943ff60 0x55dec9442380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f434dffb700 10 client.17106 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.106+0000 7f436a135700 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec943ff60 0x55dec9442380 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5bee9807ab1fb4 server_cookie=e94af7f64c92508b in_seq=0 out_seq=0 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f434dffb700 1 -- 172.21.15.5:0/36283417 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dec92e53c0 con 0x55dec943ff60 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f434dffb700 10 client.17106 handle_command_reply: tid=0 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 client.17106 shutdown 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 msgr2=0x55dec93dca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93dca00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f435c009900 tx=0x7f435c004a10 comp rx=0 tx=0).stop 2024-05-14T15:31:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 shutdown_connections 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dec943ff60 0x55dec9442380 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dec93c1d80 0x55dec93dca00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f434dffb700 0 client.17106 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dec923de40 0x55dec93cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.107+0000 7f437adadb80 1 --2- 172.21.15.5:0/36283417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dec93bbe10 0x55dec93dc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.108+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 >> 172.21.15.5:0/36283417 conn(0x55dec9285630 msgr2=0x55dec93b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.108+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 shutdown_connections 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.108+0000 7f437adadb80 1 -- 172.21.15.5:0/36283417 wait complete. 2024-05-14T15:31:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.108+0000 7f437adadb80 20 client.17106 trim_cache size 0 max 16384 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "54ac9d6f-c9e2-42c8-a5c1-6526820402e3": { 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "54ac9d6f-c9e2-42c8-a5c1-6526820402e3", 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:31:39.115 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43500b29e0 msgr2=0x7f43500b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43500b29e0 0x7f43500b4ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f435c005b40 tx=0x7f435c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 msgr2=0x7f436419d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 0x7f436419d070 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f436000e9a0 tx=0x7f436000ecb0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 shutdown_connections 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f43500b29e0 0x7f43500b4ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4364108780 0x7f43641a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4364103550 0x7f436419d070 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 --2- 172.21.15.5:0/1210022115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364107e30 0x7f436419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 >> 172.21.15.5:0/1210022115 conn(0x7f4364078e00 msgr2=0x7f436410e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 shutdown_connections 2024-05-14T15:31:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:31:39.109+0000 7f436c399700 1 -- 172.21.15.5:0/1210022115 wait complete. 2024-05-14T15:31:39.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 626486 10.62 MB/sec execute 180 sec latency 55.114 ms 2024-05-14T15:31:39.796 INFO:tasks.cephfs.filesystem:scrub status for tag:54ac9d6f-c9e2-42c8-a5c1-6526820402e3 - {'path': '/', 'tag': '54ac9d6f-c9e2-42c8-a5c1-6526820402e3', 'options': 'recursive,force'} 2024-05-14T15:31:40.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:39 smithi005 ceph-mon[31559]: pgmap v210: 129 pgs: 129 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-05-14T15:31:40.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 628697 10.63 MB/sec execute 181 sec latency 65.073 ms 2024-05-14T15:31:40.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:39 smithi144 ceph-mon[35650]: pgmap v210: 129 pgs: 129 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-05-14T15:31:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:39 smithi193 ceph-mon[36001]: pgmap v210: 129 pgs: 129 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-05-14T15:31:41.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 630073 10.61 MB/sec execute 182 sec latency 49.156 ms 2024-05-14T15:31:41.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:40 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:31:41.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:40 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:31:41.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:40 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:31:42.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 632387 10.63 MB/sec execute 183 sec latency 49.642 ms 2024-05-14T15:31:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:41 smithi144 ceph-mon[35650]: pgmap v211: 129 pgs: 129 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-05-14T15:31:42.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:41 smithi193 ceph-mon[36001]: pgmap v211: 129 pgs: 129 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-05-14T15:31:42.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:41 smithi005 ceph-mon[31559]: pgmap v211: 129 pgs: 129 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-05-14T15:31:43.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 633825 10.60 MB/sec execute 184 sec latency 21.052 ms 2024-05-14T15:31:44.144 INFO:tasks.workunit.client.0.smithi005.stdout: 1 635952 10.62 MB/sec execute 185 sec latency 74.327 ms 2024-05-14T15:31:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:43 smithi144 ceph-mon[35650]: pgmap v212: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 419 op/s 2024-05-14T15:31:44.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:43 smithi193 ceph-mon[36001]: pgmap v212: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 419 op/s 2024-05-14T15:31:44.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:43 smithi005 ceph-mon[31559]: pgmap v212: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 419 op/s 2024-05-14T15:31:45.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 637319 10.59 MB/sec execute 186 sec latency 23.723 ms 2024-05-14T15:31:46.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:45 smithi005 ceph-mon[31559]: pgmap v213: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-05-14T15:31:46.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 639637 10.61 MB/sec execute 187 sec latency 28.152 ms 2024-05-14T15:31:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:45 smithi144 ceph-mon[35650]: pgmap v213: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-05-14T15:31:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:45 smithi193 ceph-mon[36001]: pgmap v213: 129 pgs: 129 active+clean; 75 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-05-14T15:31:47.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 641654 10.59 MB/sec execute 188 sec latency 25.695 ms 2024-05-14T15:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 -- 172.21.15.5:0/1287782805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 msgr2=0x7f1b40102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 --2- 172.21.15.5:0/1287782805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b40102ee0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009900 tx=0x7f1b3002a410 comp rx=0 tx=0).stop 2024-05-14T15:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 -- 172.21.15.5:0/1287782805 shutdown_connections 2024-05-14T15:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 --2- 172.21.15.5:0/1287782805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 0x7f1b40103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 --2- 172.21.15.5:0/1287782805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 0x7f1b4010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 --2- 172.21.15.5:0/1287782805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b40102ee0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 -- 172.21.15.5:0/1287782805 >> 172.21.15.5:0/1287782805 conn(0x7f1b400779e0 msgr2=0x7f1b40077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 -- 172.21.15.5:0/1287782805 shutdown_connections 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.308+0000 7f1b46526640 1 -- 172.21.15.5:0/1287782805 wait complete. 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 Processor -- start 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 -- start start 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 0x7f1b401a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b401a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 0x7f1b4019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40112d80 con 0x7f1b40107ab0 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40112c00 con 0x7f1b4010d5a0 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b46526640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40112f00 con 0x7f1b40103420 2024-05-14T15:31:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b401a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b401a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34208/0 (socket says 172.21.15.5:34208) 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b37fff640 1 -- 172.21.15.5:0/3245324615 learned_addr learned my addr 172.21.15.5:0/3245324615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 0x7f1b4019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.309+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 0x7f1b401a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b37fff640 1 -- 172.21.15.5:0/3245324615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 msgr2=0x7f1b401a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b37fff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 0x7f1b401a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b37fff640 1 -- 172.21.15.5:0/3245324615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 msgr2=0x7f1b4019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b37fff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 0x7f1b4019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b37fff640 1 -- 172.21.15.5:0/3245324615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4019ad80 con 0x7f1b40107ab0 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 0x7f1b401a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 0x7f1b4019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b37fff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b401a0a10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d740 tx=0x7f1b2800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b3dffb640 1 -- 172.21.15.5:0/3245324615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28017070 con 0x7f1b40107ab0 2024-05-14T15:31:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b3dffb640 1 -- 172.21.15.5:0/3245324615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2800de50 con 0x7f1b40107ab0 2024-05-14T15:31:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4019b010 con 0x7f1b40107ab0 2024-05-14T15:31:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b3dffb640 1 -- 172.21.15.5:0/3245324615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28004ce0 con 0x7f1b40107ab0 2024-05-14T15:31:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.310+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b401a7270 con 0x7f1b40107ab0 2024-05-14T15:31:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.313+0000 7f1b3dffb640 1 -- 172.21.15.5:0/3245324615 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1b28004e80 con 0x7f1b40107ab0 2024-05-14T15:31:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.313+0000 7f1b4d3d8740 1 Processor -- start 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.313+0000 7f1b4d3d8740 1 -- start start 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a449a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a449fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a44a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55651a3af5d0 con 0x55651a0a91b0 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55651a368890 con 0x55651a0802f0 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55651a3ab590 con 0x55651a0884d0 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a449a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a449a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34218/0 (socket says 172.21.15.5:34218) 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a449fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1821183820 learned_addr learned my addr 172.21.15.5:0/1821183820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b37fff640 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a44a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1821183820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 msgr2=0x55651a44a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a44a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1821183820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 msgr2=0x55651a449fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a449fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1821183820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651a0c1210 con 0x55651a0a91b0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b3dffb640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1b2c0b2950 0x7f1b2c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a449a70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1b380117a0 tx=0x7f1b38011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27856c8db31e753d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1b2c0b2950 0x7f1b2c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b3dffb640 1 -- 172.21.15.5:0/3245324615 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1b280e23a0 con 0x7f1b40107ab0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.314+0000 7f1b367fc640 1 -- 172.21.15.5:0/1821183820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38011220 con 0x55651a0a91b0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.315+0000 7f1b367fc640 1 -- 172.21.15.5:0/1821183820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b3800f9a0 con 0x55651a0a91b0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.315+0000 7f1b367fc640 1 -- 172.21.15.5:0/1821183820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38004470 con 0x55651a0a91b0 2024-05-14T15:31:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.315+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1821183820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 msgr2=0x55651a449a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.315+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1b2c0b2950 0x7f1b2c0b4e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1b30031440 tx=0x7f1b300058b0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.315+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a449a70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1b380117a0 tx=0x7f1b38011c60 comp rx=0 tx=0).stop 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1821183820 shutdown_connections 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a44a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a449fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1821183820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a449a70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1821183820 >> 172.21.15.5:0/1821183820 conn(0x55651a09d300 msgr2=0x55651a446340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1821183820 shutdown_connections 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1821183820 wait complete. 2024-05-14T15:31:47.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 -- start start 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.316+0000 7f1b4d3d8740 1 Processor -- start 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a4585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a458b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a464630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55651a45b180 con 0x55651a0a91b0 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55651a36c370 con 0x55651a0802f0 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55651a45b300 con 0x55651a0884d0 2024-05-14T15:31:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b4d3d8740 10 client.0 _ll_register_callbacks cb 0x55651a0c0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a464630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a4585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a464630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34222/0 (socket says 172.21.15.5:34222) 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 -- 172.21.15.5:0/3765401195 learned_addr learned my addr 172.21.15.5:0/3765401195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b37fff640 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a458b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 -- 172.21.15.5:0/3765401195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 msgr2=0x55651a458b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a458b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 -- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 msgr2=0x55651a4585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a4585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 -- 172.21.15.5:0/3765401195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651a18d620 con 0x55651a0a91b0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a464630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1b380099b0 tx=0x7f1b380112f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ac75da022c5fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.317+0000 7f1b1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/3765401195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3801e040 con 0x55651a0a91b0 2024-05-14T15:31:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/3765401195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1b38004c40 con 0x55651a0a91b0 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3800f9a0 con 0x55651a0a91b0 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/3765401195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1b38031050 con 0x55651a0a91b0 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 10 client.17118 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55651a4674e0 con 0x55651a0a91b0 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/3765401195 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b3800f9a0 con 0x55651a0a91b0 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/3765401195 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1b38018070 con 0x55651a0a91b0 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 10 client.17118 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 20 client.17118 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a067d80 0x55651a46a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 4 client.17118 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.318+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55651a3bd1e0 con 0x55651a067d80 2024-05-14T15:31:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.319+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a067d80 0x55651a46a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.319+0000 7f1b1f7fe640 10 client.17118 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.319+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a067d80 0x55651a46a8f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fb2feb84241405 server_cookie=10c4a71dfa5cfb86 in_seq=0 out_seq=0 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/3765401195 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55651a3bd1e0 con 0x55651a067d80 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b1f7fe640 10 client.17118 handle_command_reply: tid=0 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 client.17118 shutdown 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 msgr2=0x55651a464630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a464630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1b380099b0 tx=0x7f1b380112f0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 shutdown_connections 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a067d80 0x55651a46a8f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a458b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a464630 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b1f7fe640 0 client.17118 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/3765401195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0802f0 0x55651a4585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 >> 172.21.15.5:0/3765401195 conn(0x55651a09d300 msgr2=0x55651a44b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 shutdown_connections 2024-05-14T15:31:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.320+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/3765401195 wait complete. 2024-05-14T15:31:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.321+0000 7f1b4d3d8740 20 client.17118 trim_cache size 0 max 16384 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 Processor -- start 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 -- start start 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 0x55651a458930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a45a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0a91b0 0x55651a46ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401b19d0 con 0x55651a0802f0 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401af2a0 con 0x55651a0a91b0 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ad0f0 con 0x55651a0884d0 2024-05-14T15:31:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 0x55651a458930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 0x55651a458930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34228/0 (socket says 172.21.15.5:34228) 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b3ffff640 1 -- 172.21.15.5:0/314590888 learned_addr learned my addr 172.21.15.5:0/314590888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0a91b0 0x55651a46ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.338+0000 7f1b37fff640 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a45a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b3ffff640 1 -- 172.21.15.5:0/314590888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 msgr2=0x55651a45a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a45a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b3ffff640 1 -- 172.21.15.5:0/314590888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0a91b0 msgr2=0x55651a46ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0a91b0 0x55651a46ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b3ffff640 1 -- 172.21.15.5:0/314590888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651a456930 con 0x55651a0802f0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 0x55651a458930 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1b3002a930 tx=0x7f1b3003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dbaac977e7d944 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b35ffb640 1 -- 172.21.15.5:0/314590888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30056070 con 0x55651a0802f0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b35ffb640 1 -- 172.21.15.5:0/314590888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b30004240 con 0x55651a0802f0 2024-05-14T15:31:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.339+0000 7f1b35ffb640 1 -- 172.21.15.5:0/314590888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30035040 con 0x55651a0802f0 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/314590888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 msgr2=0x55651a458930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 0x55651a458930 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1b3002a930 tx=0x7f1b3003e000 comp rx=0 tx=0).stop 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/314590888 shutdown_connections 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0884d0 0x55651a45a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0a91b0 0x55651a46ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/314590888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0802f0 0x55651a458930 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/314590888 >> 172.21.15.5:0/314590888 conn(0x55651a09d300 msgr2=0x55651a459b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/314590888 shutdown_connections 2024-05-14T15:31:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/314590888 wait complete. 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.340+0000 7f1b4d3d8740 1 -- start start 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 Processor -- start 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0802f0 0x55651a45b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0884d0 0x55651a45be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a4d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ad760 con 0x55651a0a91b0 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401aebd0 con 0x55651a0884d0 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ade30 con 0x55651a0802f0 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b4d3d8740 10 client.0 _ll_register_callbacks cb 0x7f1b380093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a4d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a4d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34238/0 (socket says 172.21.15.5:34238) 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0884d0 0x55651a45be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1521249719 learned_addr learned my addr 172.21.15.5:0/1521249719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0802f0 0x55651a45b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1521249719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0802f0 msgr2=0x55651a45b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0802f0 0x55651a45b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0884d0 msgr2=0x55651a45be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0884d0 0x55651a45be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.341+0000 7f1b44ac3640 1 -- 172.21.15.5:0/1521249719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651a4d1060 con 0x55651a0a91b0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b44ac3640 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a4d0b20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1b380099e0 tx=0x7f1b3800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce52c8d300b56a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/1521249719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800fe60 con 0x55651a0a91b0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b30004240 con 0x55651a0a91b0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/1521249719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1b38005540 con 0x55651a0a91b0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/1521249719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1b38043050 con 0x55651a0a91b0 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 10 client.17124 fetch_fsmap learned FSMap version 32 2024-05-14T15:31:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55651a4d2290 con 0x55651a0a91b0 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/1521249719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b30004240 con 0x55651a0a91b0 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/1521249719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1b38018040 con 0x55651a0a91b0 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 10 client.17124 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 20 client.17124 populate_metadata read hostname 'smithi005' 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a03d1c0 0x55651a4d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 4 client.17124 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.342+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55651a3b7050 con 0x55651a03d1c0 2024-05-14T15:31:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.343+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a03d1c0 0x55651a4d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:31:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.343+0000 7f1b1f7fe640 10 client.17124 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.343+0000 7f1b3ffff640 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a03d1c0 0x55651a4d49b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35083ad30a7bb3b0 server_cookie=77192713cc464a31 in_seq=0 out_seq=0 2024-05-14T15:31:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b1f7fe640 1 -- 172.21.15.5:0/1521249719 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55651a3b7050 con 0x55651a03d1c0 2024-05-14T15:31:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b1f7fe640 10 client.17124 handle_command_reply: tid=0 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 client.17124 shutdown 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 msgr2=0x55651a4d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a4d0b20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1b380099e0 tx=0x7f1b3800fa20 comp rx=0 tx=0).stop 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 shutdown_connections 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55651a03d1c0 0x55651a4d49b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55651a0a91b0 0x55651a4d0b20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b1f7fe640 0 client.17124 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55651a0884d0 0x55651a45be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 --2- 172.21.15.5:0/1521249719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a0802f0 0x55651a45b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.626+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 >> 172.21.15.5:0/1521249719 conn(0x55651a09d300 msgr2=0x55651a45a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.627+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 shutdown_connections 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.627+0000 7f1b4d3d8740 1 -- 172.21.15.5:0/1521249719 wait complete. 2024-05-14T15:31:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.627+0000 7f1b4d3d8740 20 client.17124 trim_cache size 0 max 16384 2024-05-14T15:31:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:31:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:31:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:31:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1b2c0b2950 msgr2=0x7f1b2c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1b2c0b2950 0x7f1b2c0b4e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1b30031440 tx=0x7f1b300058b0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 msgr2=0x7f1b401a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b401a0a10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800d740 tx=0x7f1b2800dc00 comp rx=0 tx=0).stop 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 shutdown_connections 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f1b2c0b2950 0x7f1b2c0b4e60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4010d5a0 0x7f1b4019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40107ab0 0x7f1b401a0a10 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 --2- 172.21.15.5:0/3245324615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b40103420 0x7f1b401a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:31:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 >> 172.21.15.5:0/3245324615 conn(0x7f1b400779e0 msgr2=0x7f1b40104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:31:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 shutdown_connections 2024-05-14T15:31:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:31:47.628+0000 7f1b46526640 1 -- 172.21.15.5:0/3245324615 wait complete. 2024-05-14T15:31:48.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643226 10.60 MB/sec execute 189 sec latency 182.180 ms 2024-05-14T15:31:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:47 smithi144 ceph-mon[35650]: pgmap v214: 129 pgs: 129 active+clean; 81 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-05-14T15:31:48.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:47 smithi193 ceph-mon[36001]: pgmap v214: 129 pgs: 129 active+clean; 81 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-05-14T15:31:48.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:47 smithi005 ceph-mon[31559]: pgmap v214: 129 pgs: 129 active+clean; 81 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-05-14T15:31:49.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644661 10.57 MB/sec execute 190 sec latency 95.936 ms 2024-05-14T15:31:50.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 646787 10.59 MB/sec execute 191 sec latency 45.702 ms 2024-05-14T15:31:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:49 smithi144 ceph-mon[35650]: pgmap v215: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 338 op/s 2024-05-14T15:31:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:49 smithi193 ceph-mon[36001]: pgmap v215: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 338 op/s 2024-05-14T15:31:50.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:49 smithi005 ceph-mon[31559]: pgmap v215: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 338 op/s 2024-05-14T15:31:51.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 648941 10.57 MB/sec execute 192 sec latency 15.627 ms 2024-05-14T15:31:52.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650890 10.58 MB/sec execute 193 sec latency 13.399 ms 2024-05-14T15:31:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:51 smithi144 ceph-mon[35650]: pgmap v216: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-05-14T15:31:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:51 smithi193 ceph-mon[36001]: pgmap v216: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-05-14T15:31:52.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:51 smithi005 ceph-mon[31559]: pgmap v216: 129 pgs: 129 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-05-14T15:31:53.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 653510 10.61 MB/sec execute 194 sec latency 16.824 ms 2024-05-14T15:31:54.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 654896 10.59 MB/sec execute 195 sec latency 51.275 ms 2024-05-14T15:31:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:53 smithi144 ceph-mon[35650]: pgmap v217: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-05-14T15:31:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:53 smithi193 ceph-mon[36001]: pgmap v217: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-05-14T15:31:54.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:53 smithi005 ceph-mon[31559]: pgmap v217: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-05-14T15:31:55.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 657133 10.60 MB/sec execute 196 sec latency 40.364 ms 2024-05-14T15:31:56.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 658623 10.58 MB/sec execute 197 sec latency 34.609 ms 2024-05-14T15:31:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:55 smithi144 ceph-mon[35650]: pgmap v218: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-05-14T15:31:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:55 smithi193 ceph-mon[36001]: pgmap v218: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-05-14T15:31:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:55 smithi005 ceph-mon[31559]: pgmap v218: 129 pgs: 129 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2024-05-14T15:31:57.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 660890 10.60 MB/sec execute 198 sec latency 23.480 ms 2024-05-14T15:31:58.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 662391 10.57 MB/sec execute 199 sec latency 16.141 ms 2024-05-14T15:31:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:57 smithi144 ceph-mon[35650]: pgmap v219: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-05-14T15:31:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:57 smithi193 ceph-mon[36001]: pgmap v219: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-05-14T15:31:58.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:57 smithi005 ceph-mon[31559]: pgmap v219: 129 pgs: 129 active+clean; 76 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.4 MiB/s wr, 394 op/s 2024-05-14T15:31:59.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 664417 10.59 MB/sec execute 200 sec latency 72.559 ms 2024-05-14T15:32:00.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 665796 10.56 MB/sec execute 201 sec latency 24.247 ms 2024-05-14T15:32:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:31:59 smithi144 ceph-mon[35650]: pgmap v220: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2024-05-14T15:32:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:31:59 smithi193 ceph-mon[36001]: pgmap v220: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2024-05-14T15:32:00.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:31:59 smithi005 ceph-mon[31559]: pgmap v220: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2024-05-14T15:32:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/1827906586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 msgr2=0x7fe09c102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/1827906586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c102740 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009ca0 tx=0x7fe08c02a8c0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/1827906586 shutdown_connections 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/1827906586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe09c1032f0 0x7fe09c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/1827906586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 0x7fe09c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/1827906586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c102740 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/1827906586 >> 172.21.15.5:0/1827906586 conn(0x7fe09c0782e0 msgr2=0x7fe09c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/1827906586 shutdown_connections 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.739+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/1827906586 wait complete. 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 Processor -- start 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 -- start start 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe09c1032f0 0x7fe09c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 0x7fe09c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c112d90 con 0x7fe09c108ba0 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c112c10 con 0x7fe09c109040 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe0a1ce0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c112f10 con 0x7fe09c1032f0 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.740+0000 7fe09affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c1a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43964/0 (socket says 172.21.15.5:43964) 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 -- 172.21.15.5:0/3743187150 learned_addr learned my addr 172.21.15.5:0/3743187150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe09c1032f0 0x7fe09c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 0x7fe09c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 -- 172.21.15.5:0/3743187150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe09c1032f0 msgr2=0x7fe09c1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe09c1032f0 0x7fe09c1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 -- 172.21.15.5:0/3743187150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 msgr2=0x7fe09c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 0x7fe09c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 -- 172.21.15.5:0/3743187150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c19ae30 con 0x7fe09c108ba0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 0x7fe09c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe09affd640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c1a0ac0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e9f0 tx=0x7fe09000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe098ff9640 1 -- 172.21.15.5:0/3743187150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09000ccc0 con 0x7fe09c108ba0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe098ff9640 1 -- 172.21.15.5:0/3743187150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0900035f0 con 0x7fe09c108ba0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c19b0c0 con 0x7fe09c108ba0 2024-05-14T15:32:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe098ff9640 1 -- 172.21.15.5:0/3743187150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09000ce60 con 0x7fe09c108ba0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.741+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a7320 con 0x7fe09c108ba0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.744+0000 7fe098ff9640 1 -- 172.21.15.5:0/3743187150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe090003aa0 con 0x7fe09c108ba0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.744+0000 7fe0a8b8a740 1 Processor -- start 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.744+0000 7fe0a8b8a740 1 -- start start 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861963be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861964120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861964660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f8618c95d0 con 0x55f86159a2f0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f861882890 con 0x55f8615a24d0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f8618c5590 con 0x55f8615c31b0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861963be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861964120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861963be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:48950/0 (socket says 172.21.15.5:48950) 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 -- 172.21.15.5:0/599914392 learned_addr learned my addr 172.21.15.5:0/599914392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09affd640 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861964660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 -- 172.21.15.5:0/599914392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 msgr2=0x55f861964660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861964660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 -- 172.21.15.5:0/599914392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 msgr2=0x55f861964120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861964120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe098ff9640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe0840b2950 0x7fe0840b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 -- 172.21.15.5:0/599914392 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8615db210 con 0x55f8615c31b0 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09affd640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe0840b2950 0x7fe0840b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.745+0000 7fe09bfff640 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861963be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe094011540 tx=0x7fe094011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219fecd28e6ad223 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.746+0000 7fe098ff9640 1 -- 172.21.15.5:0/3743187150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe0900e2c70 con 0x7fe09c108ba0 2024-05-14T15:32:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.746+0000 7fe081ffb640 1 -- 172.21.15.5:0/599914392 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09400f5c0 con 0x55f8615c31b0 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.746+0000 7fe081ffb640 1 -- 172.21.15.5:0/599914392 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09400fc10 con 0x55f8615c31b0 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.746+0000 7fe081ffb640 1 -- 172.21.15.5:0/599914392 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094019c50 con 0x55f8615c31b0 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/599914392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 msgr2=0x55f861963be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe09affd640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe0840b2950 0x7fe0840b4e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0900e00b0 tx=0x7fe0900e67c0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861963be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe094011540 tx=0x7fe094011a00 comp rx=0 tx=0).stop 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/599914392 shutdown_connections 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861964660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861964120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/599914392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861963be0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/599914392 >> 172.21.15.5:0/599914392 conn(0x55f8615b7300 msgr2=0x55f861960340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/599914392 shutdown_connections 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/599914392 wait complete. 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 -- start start 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.747+0000 7fe0a8b8a740 1 Processor -- start 2024-05-14T15:32:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f86196e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615a24d0 0x55f86196e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615c31b0 0x55f861974200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f861970d10 con 0x55f86159a2f0 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f861886370 con 0x55f8615c31b0 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f861970e90 con 0x55f8615a24d0 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe0a8b8a740 10 client.0 _ll_register_callbacks cb 0x55f8615daa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f86196e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f86196e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43990/0 (socket says 172.21.15.5:43990) 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 -- 172.21.15.5:0/2410588259 learned_addr learned my addr 172.21.15.5:0/2410588259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615a24d0 0x55f86196e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615c31b0 0x55f861974200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 -- 172.21.15.5:0/2410588259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615a24d0 msgr2=0x55f86196e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615a24d0 0x55f86196e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 -- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615c31b0 msgr2=0x55f861974200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615c31b0 0x55f861974200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09bfff640 1 -- 172.21.15.5:0/2410588259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8616a7620 con 0x55f86159a2f0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.748+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615a24d0 0x55f86196e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f86196e2b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe0940058c0 tx=0x7fe094005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2459cf307732fe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe07affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe07affd640 1 -- 172.21.15.5:0/2410588259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09403d060 con 0x55f86159a2f0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe07affd640 1 -- 172.21.15.5:0/2410588259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe09401f040 con 0x55f86159a2f0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe09400fc10 con 0x55f86159a2f0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe07affd640 1 -- 172.21.15.5:0/2410588259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe094019640 con 0x55f86159a2f0 2024-05-14T15:32:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe0a8b8a740 10 client.17136 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f86197d110 con 0x55f86159a2f0 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe07affd640 1 -- 172.21.15.5:0/2410588259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09400fc10 con 0x55f86159a2f0 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe07affd640 1 -- 172.21.15.5:0/2410588259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe0940035f0 con 0x55f86159a2f0 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.749+0000 7fe0a8b8a740 10 client.17136 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe0a8b8a740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe0a8b8a740 20 client.17136 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f861581d80 0x55f8619804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe0a8b8a740 4 client.17136 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8618d71e0 con 0x55f861581d80 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f861581d80 0x55f8619804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f861581d80 0x55f8619804d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f61a65eaa62896 server_cookie=ea790e0a3930ef89 in_seq=0 out_seq=0 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.750+0000 7fe07affd640 10 client.17136 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe07affd640 1 -- 172.21.15.5:0/2410588259 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8618d71e0 con 0x55f861581d80 2024-05-14T15:32:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe07affd640 10 client.17136 handle_command_reply: tid=0 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 client.17136 shutdown 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 msgr2=0x55f86196e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f86196e2b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe0940058c0 tx=0x7fe094005850 comp rx=0 tx=0).stop 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 shutdown_connections 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f861581d80 0x55f8619804d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615a24d0 0x55f86196e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615c31b0 0x55f861974200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.751+0000 7fe07affd640 0 client.17136 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.752+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2410588259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f86196e2b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.752+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 >> 172.21.15.5:0/2410588259 conn(0x55f8615b7300 msgr2=0x55f861963a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.752+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 shutdown_connections 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.752+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2410588259 wait complete. 2024-05-14T15:32:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.752+0000 7fe0a8b8a740 20 client.17136 trim_cache size 0 max 16384 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 Processor -- start 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 -- start start 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861967f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861960430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861974100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1ab4f0 con 0x55f86159a2f0 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1a8dc0 con 0x55f8615a24d0 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c06a240 con 0x55f8615c31b0 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861967f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861967f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44004/0 (socket says 172.21.15.5:44004) 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe09bfff640 1 -- 172.21.15.5:0/3982386546 learned_addr learned my addr 172.21.15.5:0/3982386546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861960430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.771+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861974100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09bfff640 1 -- 172.21.15.5:0/3982386546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 msgr2=0x55f861974100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861974100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09bfff640 1 -- 172.21.15.5:0/3982386546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 msgr2=0x55f861960430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861960430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09bfff640 1 -- 172.21.15.5:0/3982386546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f861970610 con 0x55f86159a2f0 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861960430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861967f10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe094009d90 tx=0x7fe094011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e92645c3faec670 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe0817fa640 1 -- 172.21.15.5:0/3982386546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09403d070 con 0x55f86159a2f0 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe0817fa640 1 -- 172.21.15.5:0/3982386546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0940037f0 con 0x55f86159a2f0 2024-05-14T15:32:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.772+0000 7fe0817fa640 1 -- 172.21.15.5:0/3982386546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0940052a0 con 0x55f86159a2f0 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/3982386546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 msgr2=0x55f861967f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861967f10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe094009d90 tx=0x7fe094011a00 comp rx=0 tx=0).stop 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/3982386546 shutdown_connections 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861974100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f861960430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/3982386546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861967f10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/3982386546 >> 172.21.15.5:0/3982386546 conn(0x55f8615b7300 msgr2=0x55f86195fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/3982386546 shutdown_connections 2024-05-14T15:32:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/3982386546 wait complete. 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.773+0000 7fe0a8b8a740 1 -- start start 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 Processor -- start 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861974cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f8619751f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861975730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c074cc0 con 0x55f86159a2f0 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1a86f0 con 0x55f8615a24d0 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c1a7950 con 0x55f8615c31b0 2024-05-14T15:32:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe0a8b8a740 10 client.0 _ll_register_callbacks cb 0x7fe094005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861974cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe09b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f8619751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861974cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44018/0 (socket says 172.21.15.5:44018) 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe09bfff640 1 -- 172.21.15.5:0/2374709661 learned_addr learned my addr 172.21.15.5:0/2374709661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.774+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861975730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe09b7fe640 1 -- 172.21.15.5:0/2374709661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 msgr2=0x55f861975730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861975730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe09b7fe640 1 -- 172.21.15.5:0/2374709661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 msgr2=0x55f861974cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861974cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe09b7fe640 1 -- 172.21.15.5:0/2374709661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8619e67f0 con 0x55f8615a24d0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe09b7fe640 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f8619751f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe08c006010 tx=0x7fe08c0316a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=706e9a4398eadd95 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe07affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:32:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe07affd640 1 -- 172.21.15.5:0/2374709661 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c003970 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0940037f0 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe07affd640 1 -- 172.21.15.5:0/2374709661 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe08c003b10 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe07affd640 1 -- 172.21.15.5:0/2374709661 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe08c059050 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe0a8b8a740 10 client.26765 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.775+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8619e7ad0 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe07affd640 1 -- 172.21.15.5:0/2374709661 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0940037f0 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe07affd640 1 -- 172.21.15.5:0/2374709661 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe08c02f660 con 0x55f8615a24d0 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe0a8b8a740 10 client.26765 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe0a8b8a740 10 client.26765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe0a8b8a740 20 client.26765 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8615571c0 0x55f8619ea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe0a8b8a740 4 client.26765 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:32:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8618d1050 con 0x55f8615571c0 2024-05-14T15:32:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.776+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8615571c0 0x55f8619ea1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.777+0000 7fe07affd640 10 client.26765 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:00.777+0000 7fe09bfff640 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8615571c0 0x55f8619ea1c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc22cefb723a3ce server_cookie=244f156176cdbb3a in_seq=0 out_seq=0 2024-05-14T15:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe07affd640 1 -- 172.21.15.5:0/2374709661 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8618d1050 con 0x55f8615571c0 2024-05-14T15:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe07affd640 10 client.26765 handle_command_reply: tid=0 2024-05-14T15:32:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 client.26765 shutdown 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 msgr2=0x55f8619751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f8619751f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe08c006010 tx=0x7fe08c0316a0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 shutdown_connections 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f8615571c0 0x55f8619ea1c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe07affd640 0 client.26765 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f8615c31b0 0x55f861975730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f8615a24d0 0x55f8619751f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 --2- 172.21.15.5:0/2374709661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f86159a2f0 0x55f861974cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 >> 172.21.15.5:0/2374709661 conn(0x55f8615b7300 msgr2=0x55f86196e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 shutdown_connections 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.028+0000 7fe0a8b8a740 1 -- 172.21.15.5:0/2374709661 wait complete. 2024-05-14T15:32:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a8b8a740 20 client.26765 trim_cache size 0 max 16384 2024-05-14T15:32:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:32:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:32:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:32:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe0840b2950 msgr2=0x7fe0840b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe0840b2950 0x7fe0840b4e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0900e00b0 tx=0x7fe0900e67c0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 msgr2=0x7fe09c1a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c1a0ac0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e9f0 tx=0x7fe09000eeb0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 shutdown_connections 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe0840b2950 0x7fe0840b4e60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe09c109040 0x7fe09c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c108ba0 0x7fe09c1a0ac0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 --2- 172.21.15.5:0/3743187150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe09c1032f0 0x7fe09c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.029+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 >> 172.21.15.5:0/3743187150 conn(0x7fe09c0782e0 msgr2=0x7fe09c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.030+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 shutdown_connections 2024-05-14T15:32:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:01.030+0000 7fe0a1ce0640 1 -- 172.21.15.5:0/3743187150 wait complete. 2024-05-14T15:32:01.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 667313 10.55 MB/sec execute 202 sec latency 266.794 ms 2024-05-14T15:32:02.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 668077 10.52 MB/sec execute 203 sec latency 56.658 ms 2024-05-14T15:32:02.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:01 smithi144 ceph-mon[35650]: pgmap v221: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:32:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:01 smithi193 ceph-mon[36001]: pgmap v221: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:32:02.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:01 smithi005 ceph-mon[31559]: pgmap v221: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:32:03.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 669958 10.51 MB/sec execute 204 sec latency 18.852 ms 2024-05-14T15:32:04.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:03 smithi005 ceph-mon[31559]: pgmap v222: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 581 op/s 2024-05-14T15:32:04.145 INFO:tasks.workunit.client.0.smithi005.stdout: 1 671635 10.52 MB/sec execute 205 sec latency 35.316 ms 2024-05-14T15:32:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:03 smithi144 ceph-mon[35650]: pgmap v222: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 581 op/s 2024-05-14T15:32:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:03 smithi193 ceph-mon[36001]: pgmap v222: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 581 op/s 2024-05-14T15:32:05.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 673185 10.49 MB/sec execute 206 sec latency 40.966 ms 2024-05-14T15:32:06.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 675139 10.51 MB/sec execute 207 sec latency 22.451 ms 2024-05-14T15:32:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:05 smithi144 ceph-mon[35650]: pgmap v223: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-05-14T15:32:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:05 smithi193 ceph-mon[36001]: pgmap v223: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-05-14T15:32:06.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:05 smithi005 ceph-mon[31559]: pgmap v223: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 346 op/s 2024-05-14T15:32:07.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:06 smithi005 ceph-mon[31559]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T15:32:07.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 675547 10.46 MB/sec execute 208 sec latency 73.029 ms 2024-05-14T15:32:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:06 smithi144 ceph-mon[35650]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T15:32:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:06 smithi193 ceph-mon[36001]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T15:32:08.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 675950 10.42 MB/sec execute 209 sec latency 87.739 ms 2024-05-14T15:32:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:07 smithi144 ceph-mon[35650]: pgmap v225: 129 pgs: 9 active+clean+snaptrim, 100 active+clean, 20 active+clean+snaptrim_wait; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 399 op/s 2024-05-14T15:32:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:07 smithi193 ceph-mon[36001]: pgmap v225: 129 pgs: 9 active+clean+snaptrim, 100 active+clean, 20 active+clean+snaptrim_wait; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 399 op/s 2024-05-14T15:32:08.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:07 smithi005 ceph-mon[31559]: pgmap v225: 129 pgs: 9 active+clean+snaptrim, 100 active+clean, 20 active+clean+snaptrim_wait; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 399 op/s 2024-05-14T15:32:09.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 676745 10.38 MB/sec execute 210 sec latency 48.095 ms 2024-05-14T15:32:09.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:32:10.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 678383 10.39 MB/sec execute 211 sec latency 87.354 ms 2024-05-14T15:32:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:10.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:09 smithi005 ceph-mon[31559]: pgmap v226: 129 pgs: 19 active+clean+snaptrim, 57 active+clean, 53 active+clean+snaptrim_wait; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-05-14T15:32:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:09 smithi144 ceph-mon[35650]: pgmap v226: 129 pgs: 19 active+clean+snaptrim, 57 active+clean, 53 active+clean+snaptrim_wait; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-05-14T15:32:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:09 smithi193 ceph-mon[36001]: pgmap v226: 129 pgs: 19 active+clean+snaptrim, 57 active+clean, 53 active+clean+snaptrim_wait; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-05-14T15:32:11.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 679797 10.37 MB/sec execute 212 sec latency 16.787 ms 2024-05-14T15:32:12.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 -- 172.21.15.5:0/1035481027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1029c0 msgr2=0x7f8a4c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/1035481027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c102dd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000aaa0 tx=0x7f8a4000adb0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 -- 172.21.15.5:0/1035481027 shutdown_connections 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/1035481027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c103310 0x7f8a4c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/1035481027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c1089f0 0x7f8a4c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/1035481027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c102dd0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.998+0000 7f8a51ef5700 1 -- 172.21.15.5:0/1035481027 >> 172.21.15.5:0/1035481027 conn(0x7f8a4c0fe520 msgr2=0x7f8a4c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.999+0000 7f8a51ef5700 1 -- 172.21.15.5:0/1035481027 shutdown_connections 2024-05-14T15:32:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.999+0000 7f8a51ef5700 1 -- 172.21.15.5:0/1035481027 wait complete. 2024-05-14T15:32:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:11.999+0000 7f8a51ef5700 1 Processor -- start 2024-05-14T15:32:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 -- start start 2024-05-14T15:32:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c103310 0x7f8a4c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1089f0 0x7f8a4c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1122e0 con 0x7f8a4c1089f0 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c112160 con 0x7f8a4c103310 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a51ef5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c112460 con 0x7f8a4c1029c0 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a4bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1089f0 0x7f8a4c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53138/0 (socket says 172.21.15.5:53138) 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c103310 0x7f8a4c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.000+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/2323515757 learned_addr learned my addr 172.21.15.5:0/2323515757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/2323515757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c103310 msgr2=0x7f8a4c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c103310 0x7f8a4c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1089f0 msgr2=0x7f8a4c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1089f0 0x7f8a4c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/2323515757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c19fd10 con 0x7f8a4c1029c0 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4affd700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c103310 0x7f8a4c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c19ae40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00ea30 tx=0x7f8a3c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a48ff9700 1 -- 172.21.15.5:0/2323515757 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00cbd0 con 0x7f8a4c1029c0 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a48ff9700 1 -- 172.21.15.5:0/2323515757 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0044d0 con 0x7f8a4c1029c0 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c1a0000 con 0x7f8a4c1029c0 2024-05-14T15:32:12.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a48ff9700 1 -- 172.21.15.5:0/2323515757 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c003dc0 con 0x7f8a4c1029c0 2024-05-14T15:32:12.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.001+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c1a7920 con 0x7f8a4c1029c0 2024-05-14T15:32:12.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.004+0000 7f8a48ff9700 1 -- 172.21.15.5:0/2323515757 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8a3c004000 con 0x7f8a4c1029c0 2024-05-14T15:32:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.005+0000 7f8a48ff9700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8a340b29f0 0x7f8a340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a48ff9700 1 -- 172.21.15.5:0/2323515757 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f8a3c0e2b90 con 0x7f8a4c1029c0 2024-05-14T15:32:12.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a4affd700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8a340b29f0 0x7f8a340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a60909b80 1 Processor -- start 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a60909b80 1 -- start start 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.006+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4ca880 0x56063c4cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a60909b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56063c3e3860 con 0x56063c4c6070 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a60909b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56063c3827c0 con 0x56063c4ca880 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a60909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56063c3df820 con 0x56063c343e40 2024-05-14T15:32:12.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4affd700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8a340b29f0 0x7f8a340b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000c6d0 tx=0x7f8a40005960 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4ca880 0x56063c4cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45728/0 (socket says 172.21.15.5:45728) 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4affd700 1 -- 172.21.15.5:0/3555849463 learned_addr learned my addr 172.21.15.5:0/3555849463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 -- 172.21.15.5:0/3555849463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 msgr2=0x56063c4c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 -- 172.21.15.5:0/3555849463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 msgr2=0x56063c4ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 -- 172.21.15.5:0/3555849463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063c3308d0 con 0x56063c4ca880 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4ca880 0x56063c4cac90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a44011580 tx=0x7f8a44009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac575285807ae90f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.007+0000 7f8a31ffb700 1 -- 172.21.15.5:0/3555849463 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44011e80 con 0x56063c4ca880 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.008+0000 7f8a31ffb700 1 -- 172.21.15.5:0/3555849463 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a44004bb0 con 0x56063c4ca880 2024-05-14T15:32:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.008+0000 7f8a31ffb700 1 -- 172.21.15.5:0/3555849463 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440046e0 con 0x56063c4ca880 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.008+0000 7f8a60909b80 1 -- 172.21.15.5:0/3555849463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4ca880 msgr2=0x56063c4cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.008+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4ca880 0x56063c4cac90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a44011580 tx=0x7f8a44009920 comp rx=0 tx=0).stop 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 -- 172.21.15.5:0/3555849463 shutdown_connections 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4ca880 0x56063c4cac90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3555849463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 -- 172.21.15.5:0/3555849463 >> 172.21.15.5:0/3555849463 conn(0x56063c38b600 msgr2=0x56063c4bc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:12.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 -- 172.21.15.5:0/3555849463 shutdown_connections 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 -- 172.21.15.5:0/3555849463 wait complete. 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.009+0000 7f8a60909b80 1 -- start start 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 Processor -- start 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c343e40 0x56063c4cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c4ca880 0x56063c4cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56063c4e3840 con 0x56063c4c6070 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56063c349c30 con 0x56063c343e40 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56063c4e39c0 con 0x56063c4ca880 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a60909b80 10 client.0 _ll_register_callbacks cb 0x56063c37b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c4ca880 0x56063c4cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45734/0 (socket says 172.21.15.5:45734) 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c343e40 0x56063c4cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4affd700 1 -- 172.21.15.5:0/3936490118 learned_addr learned my addr 172.21.15.5:0/3936490118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4bfff700 1 -- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c343e40 msgr2=0x56063c4cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c343e40 0x56063c4cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.010+0000 7f8a4bfff700 1 -- 172.21.15.5:0/3936490118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 msgr2=0x56063c4cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a4bfff700 1 -- 172.21.15.5:0/3936490118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063c37f690 con 0x56063c4ca880 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a4affd700 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c4ca880 0x56063c4cff50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b010 tx=0x7f8a44009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f39fdbfb39f085f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a2affd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a2affd700 1 -- 172.21.15.5:0/3936490118 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4402b060 con 0x56063c4ca880 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a44004bb0 con 0x56063c4ca880 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a2affd700 1 -- 172.21.15.5:0/3936490118 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f8a44003c00 con 0x56063c4ca880 2024-05-14T15:32:12.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a2affd700 1 -- 172.21.15.5:0/3936490118 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a44032050 con 0x56063c4ca880 2024-05-14T15:32:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a60909b80 10 client.26749 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.011+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56063c4ee380 con 0x56063c4ca880 2024-05-14T15:32:12.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a2affd700 1 -- 172.21.15.5:0/3936490118 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a44004bb0 con 0x56063c4ca880 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a2affd700 1 -- 172.21.15.5:0/3936490118 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8a44025030 con 0x56063c4ca880 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a60909b80 10 client.26749 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a60909b80 10 client.26749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a60909b80 20 client.26749 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c4f0910 0x56063c4f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a60909b80 4 client.26749 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063c3f1240 con 0x56063c4f0910 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c4f0910 0x56063c4f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.012+0000 7f8a2affd700 10 client.26749 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.013+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c4f0910 0x56063c4f2d30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136b3198f75df6cc server_cookie=fe4038995b8abfce in_seq=0 out_seq=0 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.013+0000 7f8a2affd700 1 -- 172.21.15.5:0/3936490118 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56063c3f1240 con 0x56063c4f0910 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.013+0000 7f8a2affd700 10 client.26749 handle_command_reply: tid=0 2024-05-14T15:32:12.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.013+0000 7f8a60909b80 1 client.26749 shutdown 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c4ca880 msgr2=0x56063c4cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c4ca880 0x56063c4cff50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b010 tx=0x7f8a44009dd0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 shutdown_connections 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c4f0910 0x56063c4f2d30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c4ca880 0x56063c4cff50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c343e40 0x56063c4cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3936490118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4c6070 0x56063c4cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a2affd700 0 client.26749 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 >> 172.21.15.5:0/3936490118 conn(0x56063c38b600 msgr2=0x56063c4c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 shutdown_connections 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 1 -- 172.21.15.5:0/3936490118 wait complete. 2024-05-14T15:32:12.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.014+0000 7f8a60909b80 20 client.26749 trim_cache size 0 max 16384 2024-05-14T15:32:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 Processor -- start 2024-05-14T15:32:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 -- start start 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1a87a0 con 0x56063c4ca880 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c073f20 con 0x56063c4c6070 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.042+0000 7f8a60909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c071d90 con 0x56063c343e40 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50532/0 (socket says 172.21.15.5:50532) 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53176/0 (socket says 172.21.15.5:53176) 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4affd700 1 -- 172.21.15.5:0/1802316923 learned_addr learned my addr 172.21.15.5:0/1802316923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45740/0 (socket says 172.21.15.5:45740) 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/1802316923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 msgr2=0x56063c4c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/1802316923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 msgr2=0x56063c4c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/1802316923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063c4b8ae0 con 0x56063c343e40 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4affd700 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4ce770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0dfe10 tx=0x7f8a3c00cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd4f3fba468e975 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a317fa700 1 -- 172.21.15.5:0/1802316923 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0b2140 con 0x56063c343e40 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.043+0000 7f8a317fa700 1 -- 172.21.15.5:0/1802316923 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0a94b0 con 0x56063c343e40 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.044+0000 7f8a317fa700 1 -- 172.21.15.5:0/1802316923 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0b2800 con 0x56063c343e40 2024-05-14T15:32:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.044+0000 7f8a60909b80 1 -- 172.21.15.5:0/1802316923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 msgr2=0x56063c4ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.044+0000 7f8a60909b80 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4ce770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0dfe10 tx=0x7f8a3c00cd70 comp rx=0 tx=0).stop 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.044+0000 7f8a60909b80 1 -- 172.21.15.5:0/1802316923 shutdown_connections 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.044+0000 7f8a60909b80 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4ce770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 --2- 172.21.15.5:0/1802316923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 -- 172.21.15.5:0/1802316923 >> 172.21.15.5:0/1802316923 conn(0x56063c38b600 msgr2=0x56063c4d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 -- 172.21.15.5:0/1802316923 shutdown_connections 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 -- 172.21.15.5:0/1802316923 wait complete. 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 -- start start 2024-05-14T15:32:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.045+0000 7f8a60909b80 1 Processor -- start 2024-05-14T15:32:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c0723e0 con 0x56063c4ca880 2024-05-14T15:32:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c073850 con 0x56063c4c6070 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c072ab0 con 0x56063c343e40 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a60909b80 10 client.0 _ll_register_callbacks cb 0x56063c37b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4e9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50544/0 (socket says 172.21.15.5:50544) 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4e8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53192/0 (socket says 172.21.15.5:53192) 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4affd700 1 -- 172.21.15.5:0/3326817232 learned_addr learned my addr 172.21.15.5:0/3326817232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4bfff700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 msgr2=0x56063c4e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/3326817232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 msgr2=0x56063c4cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 -- 172.21.15.5:0/3326817232 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063c4cbd50 con 0x56063c343e40 2024-05-14T15:32:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4affd700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a4b7fe700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4e8c50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0dfe10 tx=0x7f8a3c0b2140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a7908641665b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.046+0000 7f8a2affd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a2affd700 1 -- 172.21.15.5:0/3326817232 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0042f0 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a2affd700 1 -- 172.21.15.5:0/3326817232 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f8a3c0e0090 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3c0a94b0 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a2affd700 1 -- 172.21.15.5:0/3326817232 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a3c0aa0a0 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 10 client.26755 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56063c55a570 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a2affd700 1 -- 172.21.15.5:0/3326817232 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c0a94b0 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a2affd700 1 -- 172.21.15.5:0/3326817232 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8a3c0c6070 con 0x56063c343e40 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 10 client.26755 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 20 client.26755 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c55aba0 0x56063c55cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 4 client.26755 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:32:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.047+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56063c3eb2e0 con 0x56063c55aba0 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.048+0000 7f8a4affd700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c55aba0 0x56063c55cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.048+0000 7f8a4affd700 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c55aba0 0x56063c55cfc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f5a1653e990913 server_cookie=4440d463e76d9bde in_seq=0 out_seq=0 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.048+0000 7f8a2affd700 10 client.26755 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a2affd700 1 -- 172.21.15.5:0/3326817232 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56063c3eb2e0 con 0x56063c55aba0 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a2affd700 10 client.26755 handle_command_reply: tid=0 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 client.26755 shutdown 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 msgr2=0x56063c4e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4e8c50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0dfe10 tx=0x7f8a3c0b2140 comp rx=0 tx=0).stop 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 shutdown_connections 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x56063c55aba0 0x56063c55cfc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56063c4ca880 0x56063c4cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56063c343e40 0x56063c4e8c50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 --2- 172.21.15.5:0/3326817232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56063c4c6070 0x56063c4e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a2affd700 0 client.26755 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 >> 172.21.15.5:0/3326817232 conn(0x56063c38b600 msgr2=0x56063c4c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 shutdown_connections 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 1 -- 172.21.15.5:0/3326817232 wait complete. 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.049+0000 7f8a60909b80 20 client.26755 trim_cache size 0 max 16384 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:32:12.055 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8a340b29f0 msgr2=0x7f8a340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8a340b29f0 0x7f8a340b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000c6d0 tx=0x7f8a40005960 comp rx=0 tx=0).stop 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 msgr2=0x7f8a4c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c19ae40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00ea30 tx=0x7f8a3c00ed40 comp rx=0 tx=0).stop 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 shutdown_connections 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8a340b29f0 0x7f8a340b4f00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c1089f0 0x7f8a4c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a4c1029c0 0x7f8a4c19ae40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 --2- 172.21.15.5:0/2323515757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c103310 0x7f8a4c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 >> 172.21.15.5:0/2323515757 conn(0x7f8a4c0fe520 msgr2=0x7f8a4c100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 shutdown_connections 2024-05-14T15:32:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:12.050+0000 7f8a51ef5700 1 -- 172.21.15.5:0/2323515757 wait complete. 2024-05-14T15:32:12.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:11 smithi005 ceph-mon[31559]: pgmap v227: 129 pgs: 19 active+clean+snaptrim, 57 active+clean, 53 active+clean+snaptrim_wait; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-05-14T15:32:12.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:11 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:32:12.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 681917 10.38 MB/sec execute 213 sec latency 30.150 ms 2024-05-14T15:32:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:11 smithi144 ceph-mon[35650]: pgmap v227: 129 pgs: 19 active+clean+snaptrim, 57 active+clean, 53 active+clean+snaptrim_wait; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-05-14T15:32:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:11 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:32:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:11 smithi193 ceph-mon[36001]: pgmap v227: 129 pgs: 19 active+clean+snaptrim, 57 active+clean, 53 active+clean+snaptrim_wait; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 411 op/s 2024-05-14T15:32:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:11 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:32:13.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 683389 10.37 MB/sec execute 214 sec latency 14.705 ms 2024-05-14T15:32:13.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:12 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:32:13.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:12 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:32:13.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:12 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:32:13.268 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:32:13.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:32:13.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:12 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:32:13.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:12 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:32:13.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:12 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:32:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:12 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:32:13.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:12 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:32:13.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:12 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:32:13.522 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:13.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:13 smithi005 ceph-mon[31559]: pgmap v228: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-05-14T15:32:14.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 685687 10.38 MB/sec execute 215 sec latency 27.613 ms 2024-05-14T15:32:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.147+0000 7f9282051640 1 -- 172.21.15.5:0/1875326887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c111df0 msgr2=0x7f927c112280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.147+0000 7f9282051640 1 --2- 172.21.15.5:0/1875326887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c111df0 0x7f927c112280 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9270009860 tx=0x7f927002a3d0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 -- 172.21.15.5:0/1875326887 shutdown_connections 2024-05-14T15:32:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 --2- 172.21.15.5:0/1875326887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c112910 0x7f927c118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 --2- 172.21.15.5:0/1875326887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c111df0 0x7f927c112280 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 --2- 172.21.15.5:0/1875326887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927c10a4a0 0x7f927c10a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 -- 172.21.15.5:0/1875326887 >> 172.21.15.5:0/1875326887 conn(0x7f927c06e900 msgr2=0x7f927c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 -- 172.21.15.5:0/1875326887 shutdown_connections 2024-05-14T15:32:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.148+0000 7f9282051640 1 -- 172.21.15.5:0/1875326887 wait complete. 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.149+0000 7f9282051640 1 Processor -- start 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 -- start start 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927c10a4a0 0x7f927c1a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c111df0 0x7f927c1a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 0x7f927c1a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f927c11e8e0 con 0x7f927c112910 2024-05-14T15:32:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f927c11e760 con 0x7f927c111df0 2024-05-14T15:32:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.150+0000 7f9282051640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f927c11ea60 con 0x7f927c10a4a0 2024-05-14T15:32:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.151+0000 7f927bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 0x7f927c1a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.151+0000 7f927bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 0x7f927c1a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45780/0 (socket says 172.21.15.5:45780) 2024-05-14T15:32:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.151+0000 7f927bfff640 1 -- 172.21.15.5:0/871508363 learned_addr learned my addr 172.21.15.5:0/871508363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.151+0000 7f927b7fe640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927c10a4a0 0x7f927c1a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.152+0000 7f927affd640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c111df0 0x7f927c1a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.152+0000 7f927bfff640 1 -- 172.21.15.5:0/871508363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927c10a4a0 msgr2=0x7f927c1a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.152+0000 7f927bfff640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927c10a4a0 0x7f927c1a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.152+0000 7f927bfff640 1 -- 172.21.15.5:0/871508363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c111df0 msgr2=0x7f927c1a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.152+0000 7f927bfff640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c111df0 0x7f927c1a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.152+0000 7f927bfff640 1 -- 172.21.15.5:0/871508363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1aa180 con 0x7f927c112910 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.153+0000 7f927bfff640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 0x7f927c1a9b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f927400b9e0 tx=0x7f927400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.154+0000 7f9278ff9640 1 -- 172.21.15.5:0/871508363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274019070 con 0x7f927c112910 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.154+0000 7f9278ff9640 1 -- 172.21.15.5:0/871508363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9274014440 con 0x7f927c112910 2024-05-14T15:32:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.154+0000 7f9278ff9640 1 -- 172.21.15.5:0/871508363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740125a0 con 0x7f927c112910 2024-05-14T15:32:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.155+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c1aa470 con 0x7f927c112910 2024-05-14T15:32:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.155+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1aa970 con 0x7f927c112910 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.161+0000 7f9288fe6740 1 Processor -- start 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 -- start start 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f691280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f68cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f6917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e67f5ee5d0 con 0x55e67f2e81b0 2024-05-14T15:32:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e67f5a7890 con 0x55e67f2bf2f0 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f9288fe6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e67f5ea590 con 0x55e67f2c74d0 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f68cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f68cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50584/0 (socket says 172.21.15.5:50584) 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.162+0000 7f927b7fe640 1 -- 172.21.15.5:0/2377872346 learned_addr learned my addr 172.21.15.5:0/2377872346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f9278ff9640 1 -- 172.21.15.5:0/871508363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f927401c070 con 0x7f927c112910 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927affd640 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f6917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927b7fe640 1 -- 172.21.15.5:0/2377872346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 msgr2=0x55e67f6917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927b7fe640 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f6917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927b7fe640 1 -- 172.21.15.5:0/2377872346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 msgr2=0x55e67f691280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927b7fe640 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f691280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927b7fe640 1 -- 172.21.15.5:0/2377872346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e67f300210 con 0x55e67f2bf2f0 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927affd640 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f6917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:32:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.163+0000 7f927b7fe640 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f68cc90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f926c00b740 tx=0x7f926c00bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45aa318c3a8e4486 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.164+0000 7f9261ffb640 1 -- 172.21.15.5:0/2377872346 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c003dd0 con 0x55e67f2bf2f0 2024-05-14T15:32:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.164+0000 7f9261ffb640 1 -- 172.21.15.5:0/2377872346 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c015070 con 0x55e67f2bf2f0 2024-05-14T15:32:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 -- 172.21.15.5:0/2377872346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 msgr2=0x55e67f68cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f68cc90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f926c00b740 tx=0x7f926c00bc00 comp rx=0 tx=0).stop 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 -- 172.21.15.5:0/2377872346 shutdown_connections 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f6917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f68cc90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 --2- 172.21.15.5:0/2377872346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f691280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9288fe6740 1 -- 172.21.15.5:0/2377872346 >> 172.21.15.5:0/2377872346 conn(0x55e67f2dc300 msgr2=0x55e67f68db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.165+0000 7f9278ff9640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92640b2a20 0x7f92640b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.166+0000 7f9288fe6740 1 -- 172.21.15.5:0/2377872346 shutdown_connections 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.166+0000 7f9278ff9640 1 -- 172.21.15.5:0/871508363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f92740e1e30 con 0x7f927c112910 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.166+0000 7f9288fe6740 1 -- 172.21.15.5:0/2377872346 wait complete. 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.166+0000 7f927b7fe640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92640b2a20 0x7f92640b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.167+0000 7f9288fe6740 1 -- start start 2024-05-14T15:32:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.167+0000 7f9288fe6740 1 Processor -- start 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.167+0000 7f927b7fe640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92640b2a20 0x7f92640b4f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55e67f68ec50 tx=0x7f926c01ad50 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.167+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f68a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2c74d0 0x55e67f68ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2e81b0 0x55e67f6b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f68a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f68a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53226/0 (socket says 172.21.15.5:53226) 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f927b7fe640 1 -- 172.21.15.5:0/3970165279 learned_addr learned my addr 172.21.15.5:0/3970165279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e67f691e20 con 0x55e67f2c74d0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e67f5ab370 con 0x55e67f2e81b0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e67f691fa0 con 0x55e67f2bf2f0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f9288fe6740 10 client.0 _ll_register_callbacks cb 0x55e67f2ffa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f927bfff640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2e81b0 0x55e67f6b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.168+0000 7f927affd640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2c74d0 0x55e67f68ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.169+0000 7f927bfff640 1 -- 172.21.15.5:0/3970165279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 msgr2=0x55e67f68a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.169+0000 7f927bfff640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f68a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.169+0000 7f927bfff640 1 -- 172.21.15.5:0/3970165279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2c74d0 msgr2=0x55e67f68ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.169+0000 7f927bfff640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2c74d0 0x55e67f68ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.169+0000 7f927bfff640 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e67f3cc620 con 0x55e67f2e81b0 2024-05-14T15:32:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.169+0000 7f927affd640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2c74d0 0x55e67f68ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:32:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.173+0000 7f927bfff640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2e81b0 0x55e67f6b02d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f92740df1c0 tx=0x7f92740a9950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28008617d4c5759f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.173+0000 7f925affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:32:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.176+0000 7f925affd640 1 -- 172.21.15.5:0/3970165279 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740df540 con 0x55e67f2e81b0 2024-05-14T15:32:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.176+0000 7f925affd640 1 -- 172.21.15.5:0/3970165279 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f92740b26a0 con 0x55e67f2e81b0 2024-05-14T15:32:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.176+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c015070 con 0x55e67f2e81b0 2024-05-14T15:32:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.176+0000 7f925affd640 1 -- 172.21.15.5:0/3970165279 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92740b2b30 con 0x55e67f2e81b0 2024-05-14T15:32:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.177+0000 7f9288fe6740 10 client.26792 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.177+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e67f6b2e90 con 0x55e67f2e81b0 2024-05-14T15:32:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.177+0000 7f925affd640 1 -- 172.21.15.5:0/3970165279 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92740b28a0 con 0x55e67f2e81b0 2024-05-14T15:32:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f925affd640 1 -- 172.21.15.5:0/3970165279 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f92740a8a50 con 0x55e67f2e81b0 2024-05-14T15:32:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f9288fe6740 10 client.26792 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f9288fe6740 10 client.26792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f9288fe6740 20 client.26792 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f2a6d80 0x55e67f6b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f9288fe6740 4 client.26792 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f926c015070 con 0x55e67f2a6d80 2024-05-14T15:32:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.179+0000 7f927b7fe640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f2a6d80 0x55e67f6b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.180+0000 7f927b7fe640 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f2a6d80 0x55e67f6b67d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0c9e286e97e67d server_cookie=81c94c2c062cd964 in_seq=0 out_seq=0 2024-05-14T15:32:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.181+0000 7f925affd640 10 client.26792 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.190+0000 7f925affd640 1 -- 172.21.15.5:0/3970165279 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f926c015070 con 0x55e67f2a6d80 2024-05-14T15:32:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.190+0000 7f925affd640 10 client.26792 handle_command_reply: tid=0 2024-05-14T15:32:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.190+0000 7f9288fe6740 1 client.26792 shutdown 2024-05-14T15:32:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.196+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2e81b0 msgr2=0x55e67f6b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.196+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2e81b0 0x55e67f6b02d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f92740df1c0 tx=0x7f92740a9950 comp rx=0 tx=0).stop 2024-05-14T15:32:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.201+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 shutdown_connections 2024-05-14T15:32:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.201+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f2a6d80 0x55e67f6b67d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.201+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2e81b0 0x55e67f6b02d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.201+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2c74d0 0x55e67f68ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.201+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3970165279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f68a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.201+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 >> 172.21.15.5:0/3970165279 conn(0x55e67f2dc300 msgr2=0x55e67f693110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.202+0000 7f925affd640 0 client.26792 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.204+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 shutdown_connections 2024-05-14T15:32:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.205+0000 7f9288fe6740 1 -- 172.21.15.5:0/3970165279 wait complete. 2024-05-14T15:32:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.209+0000 7f9288fe6740 20 client.26792 trim_cache size 0 max 16384 2024-05-14T15:32:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.247+0000 7f9288fe6740 1 Processor -- start 2024-05-14T15:32:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.247+0000 7f9288fe6740 1 -- start start 2024-05-14T15:32:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.248+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f6895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.248+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f693730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.248+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f6895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.248+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f6895a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50606/0 (socket says 172.21.15.5:50606) 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.248+0000 7f927b7fe640 1 -- 172.21.15.5:0/3306079038 learned_addr learned my addr 172.21.15.5:0/3306079038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.248+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.249+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f927c083680 con 0x55e67f2e81b0 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.249+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f927c080f50 con 0x55e67f2bf2f0 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.249+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f927c07edc0 con 0x55e67f2c74d0 2024-05-14T15:32:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.249+0000 7f927bfff640 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.251+0000 7f927b7fe640 1 -- 172.21.15.5:0/3306079038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 msgr2=0x55e67f693730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:32:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.251+0000 7f927b7fe640 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f693730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.251+0000 7f927b7fe640 1 -- 172.21.15.5:0/3306079038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 msgr2=0x55e67f6a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.251+0000 7f927b7fe640 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.252+0000 7f927b7fe640 1 -- 172.21.15.5:0/3306079038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e67f693c70 con 0x55e67f2bf2f0 2024-05-14T15:32:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.252+0000 7f927b7fe640 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f6895a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f926c0039c0 tx=0x7f926c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07606f6c7b0d889 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.253+0000 7f92617fa640 1 -- 172.21.15.5:0/3306079038 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c011840 con 0x55e67f2bf2f0 2024-05-14T15:32:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.253+0000 7f92617fa640 1 -- 172.21.15.5:0/3306079038 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c015070 con 0x55e67f2bf2f0 2024-05-14T15:32:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.253+0000 7f92617fa640 1 -- 172.21.15.5:0/3306079038 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004600 con 0x55e67f2bf2f0 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 msgr2=0x55e67f6895a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f6895a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f926c0039c0 tx=0x7f926c01f000 comp rx=0 tx=0).stop 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 shutdown_connections 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2c74d0 0x55e67f693730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 --2- 172.21.15.5:0/3306079038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2bf2f0 0x55e67f6895a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 >> 172.21.15.5:0/3306079038 conn(0x55e67f2dc300 msgr2=0x55e67f693020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 shutdown_connections 2024-05-14T15:32:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.255+0000 7f9288fe6740 1 -- 172.21.15.5:0/3306079038 wait complete. 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 -- start start 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 Processor -- start 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f6ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2c74d0 0x55e67f6a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f927c07f410 con 0x55e67f2e81b0 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f927c080880 con 0x55e67f2c74d0 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f927c07fae0 con 0x55e67f2bf2f0 2024-05-14T15:32:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.256+0000 7f9288fe6740 10 client.0 _ll_register_callbacks cb 0x7f92740a5c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.257+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f6ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.257+0000 7f927b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f6ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53242/0 (socket says 172.21.15.5:53242) 2024-05-14T15:32:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.257+0000 7f927b7fe640 1 -- 172.21.15.5:0/904086848 learned_addr learned my addr 172.21.15.5:0/904086848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927bfff640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927affd640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2c74d0 0x55e67f6a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927b7fe640 1 -- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2c74d0 msgr2=0x55e67f6a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927b7fe640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2c74d0 0x55e67f6a76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927b7fe640 1 -- 172.21.15.5:0/904086848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 msgr2=0x55e67f6a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927b7fe640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927b7fe640 1 -- 172.21.15.5:0/904086848 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e67f6a8270 con 0x55e67f2bf2f0 2024-05-14T15:32:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f927b7fe640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f6ac370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c010 tx=0x7f926c01ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51282e04852af4a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.258+0000 7f925affd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.259+0000 7f925affd640 1 -- 172.21.15.5:0/904086848 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c043060 con 0x55e67f2bf2f0 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.259+0000 7f925affd640 1 -- 172.21.15.5:0/904086848 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f926c011840 con 0x55e67f2bf2f0 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.259+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c015070 con 0x55e67f2bf2f0 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f925affd640 1 -- 172.21.15.5:0/904086848 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f926c04a050 con 0x55e67f2bf2f0 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 10 client.26767 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e67f71e9c0 con 0x55e67f2bf2f0 2024-05-14T15:32:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f925affd640 1 -- 172.21.15.5:0/904086848 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c015070 con 0x55e67f2bf2f0 2024-05-14T15:32:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f925affd640 1 -- 172.21.15.5:0/904086848 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f926c03d070 con 0x55e67f2bf2f0 2024-05-14T15:32:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 10 client.26767 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 10 client.26767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 20 client.26767 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f27c1c0 0x55e67f720eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 4 client.26767 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:32:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e67f5fc1e0 con 0x55e67f27c1c0 2024-05-14T15:32:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.260+0000 7f927affd640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f27c1c0 0x55e67f720eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.261+0000 7f927affd640 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f27c1c0 0x55e67f720eb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f135a03daa8393d5 server_cookie=2ad73474020d34c6 in_seq=0 out_seq=0 2024-05-14T15:32:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.261+0000 7f925affd640 10 client.26767 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:13 smithi144 ceph-mon[35650]: pgmap v228: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-05-14T15:32:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:13 smithi193 ceph-mon[36001]: pgmap v228: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-05-14T15:32:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.574+0000 7f925affd640 1 -- 172.21.15.5:0/904086848 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e67f5fc1e0 con 0x55e67f27c1c0 2024-05-14T15:32:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.574+0000 7f925affd640 10 client.26767 handle_command_reply: tid=0 2024-05-14T15:32:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.574+0000 7f9288fe6740 1 client.26767 shutdown 2024-05-14T15:32:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 msgr2=0x55e67f6ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f6ac370 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f926c00c010 tx=0x7f926c01ab30 comp rx=0 tx=0).stop 2024-05-14T15:32:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 shutdown_connections 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e67f27c1c0 0x55e67f720eb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e67f2e81b0 0x55e67f6a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f925affd640 0 client.26767 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e67f2c74d0 0x55e67f6a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 --2- 172.21.15.5:0/904086848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e67f2bf2f0 0x55e67f6ac370 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 >> 172.21.15.5:0/904086848 conn(0x55e67f2dc300 msgr2=0x55e67f691180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 shutdown_connections 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 1 -- 172.21.15.5:0/904086848 wait complete. 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.575+0000 7f9288fe6740 20 client.26767 trim_cache size 0 max 16384 2024-05-14T15:32:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:32:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:32:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:32:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:32:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92640b2a20 msgr2=0x7f92640b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92640b2a20 0x7f92640b4f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55e67f68ec50 tx=0x7f926c01ad50 comp rx=0 tx=0).stop 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 msgr2=0x7f927c1a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 0x7f927c1a9b40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f927400b9e0 tx=0x7f927400bea0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 shutdown_connections 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f92640b2a20 0x7f92640b4f30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c112910 0x7f927c1a9b40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c111df0 0x7f927c1a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 --2- 172.21.15.5:0/871508363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927c10a4a0 0x7f927c1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 >> 172.21.15.5:0/871508363 conn(0x7f927c06e900 msgr2=0x7f927c0737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 shutdown_connections 2024-05-14T15:32:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:14.576+0000 7f9282051640 1 -- 172.21.15.5:0/871508363 wait complete. 2024-05-14T15:32:15.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 686617 10.34 MB/sec execute 216 sec latency 87.559 ms 2024-05-14T15:32:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 -- 172.21.15.5:0/4229697903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 msgr2=0x7f2c0c107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 --2- 172.21.15.5:0/4229697903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c107ff0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009990 tx=0x7f2bf8009ca0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 -- 172.21.15.5:0/4229697903 shutdown_connections 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 --2- 172.21.15.5:0/4229697903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c108530 0x7f2c0c10fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 --2- 172.21.15.5:0/4229697903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c1032b0 0x7f2c0c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 --2- 172.21.15.5:0/4229697903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c107ff0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 -- 172.21.15.5:0/4229697903 >> 172.21.15.5:0/4229697903 conn(0x7f2c0c0fed00 msgr2=0x7f2c0c101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 -- 172.21.15.5:0/4229697903 shutdown_connections 2024-05-14T15:32:15.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.388+0000 7f2c12467700 1 -- 172.21.15.5:0/4229697903 wait complete. 2024-05-14T15:32:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.389+0000 7f2c12467700 1 Processor -- start 2024-05-14T15:32:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 -- start start 2024-05-14T15:32:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c1032b0 0x7f2c0c1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c108530 0x7f2c0c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c114280 con 0x7f2c0c107be0 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c114100 con 0x7f2c0c108530 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c12467700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c114400 con 0x7f2c0c1032b0 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c1a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45832/0 (socket says 172.21.15.5:45832) 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c0b7fe700 1 -- 172.21.15.5:0/1022521020 learned_addr learned my addr 172.21.15.5:0/1022521020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c0bfff700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c1032b0 0x7f2c0c1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.390+0000 7f2c10a04700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c108530 0x7f2c0c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c0b7fe700 1 -- 172.21.15.5:0/1022521020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c1032b0 msgr2=0x7f2c0c1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c0b7fe700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c1032b0 0x7f2c0c1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c0b7fe700 1 -- 172.21.15.5:0/1022521020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c108530 msgr2=0x7f2c0c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:15.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c0b7fe700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c108530 0x7f2c0c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c0b7fe700 1 -- 172.21.15.5:0/1022521020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c19b130 con 0x7f2c0c107be0 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c10a04700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c108530 0x7f2c0c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c0b7fe700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c1a0da0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000c00 tx=0x7f2bf800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8027070 con 0x7f2c0c107be0 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf8022470 con 0x7f2c0c107be0 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c19b3c0 con 0x7f2c0c107be0 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.391+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8017650 con 0x7f2c0c107be0 2024-05-14T15:32:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.392+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c19b8a0 con 0x7f2c0c107be0 2024-05-14T15:32:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.393+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c06a620 con 0x7f2c0c107be0 2024-05-14T15:32:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.396+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2bf802a070 con 0x7f2c0c107be0 2024-05-14T15:32:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.396+0000 7f2c097fa700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2bfc0b6e00 0x7f2bfc0b9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.397+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f2bf80e3ff0 con 0x7f2c0c107be0 2024-05-14T15:32:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.397+0000 7f2c0bfff700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2bfc0b6e00 0x7f2bfc0b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.397+0000 7f2c0bfff700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2bfc0b6e00 0x7f2bfc0b9310 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c19bbc0 tx=0x7f2c00009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.400+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2bf8031020 con 0x7f2c0c107be0 2024-05-14T15:32:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.702+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c0c02e5f0 con 0x7f2c0c107be0 2024-05-14T15:32:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.703+0000 7f2c097fa700 1 -- 172.21.15.5:0/1022521020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7f2bf8013070 con 0x7f2c0c107be0 2024-05-14T15:32:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:32:15.706 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:32:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.705+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2bfc0b6e00 msgr2=0x7f2bfc0b9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.705+0000 7f2c12467700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2bfc0b6e00 0x7f2bfc0b9310 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c19bbc0 tx=0x7f2c00009260 comp rx=0 tx=0).stop 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.705+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 msgr2=0x7f2c0c1a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.705+0000 7f2c12467700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c1a0da0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8000c00 tx=0x7f2bf800bfd0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 shutdown_connections 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2bfc0b6e00 0x7f2bfc0b9310 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0c108530 0x7f2c0c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0c1032b0 0x7f2c0c1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 --2- 172.21.15.5:0/1022521020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0c107be0 0x7f2c0c1a0da0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 >> 172.21.15.5:0/1022521020 conn(0x7f2c0c0fed00 msgr2=0x7f2c0c109490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 shutdown_connections 2024-05-14T15:32:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:15.706+0000 7f2c12467700 1 -- 172.21.15.5:0/1022521020 wait complete. 2024-05-14T15:32:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:32:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:15 smithi005 ceph-mon[31559]: pgmap v229: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-05-14T15:32:16.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:15 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1022521020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:32:16.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 688586 10.35 MB/sec execute 217 sec latency 44.493 ms 2024-05-14T15:32:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:15 smithi144 ceph-mon[35650]: pgmap v229: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-05-14T15:32:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:15 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1022521020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:32:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:15 smithi193 ceph-mon[36001]: pgmap v229: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-05-14T15:32:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:15 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1022521020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:32:16.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:32:17.110 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:17.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 690180 10.34 MB/sec execute 218 sec latency 13.213 ms 2024-05-14T15:32:18.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 692587 10.36 MB/sec execute 219 sec latency 32.484 ms 2024-05-14T15:32:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:17 smithi144 ceph-mon[35650]: pgmap v230: 129 pgs: 129 active+clean; 77 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-05-14T15:32:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:17 smithi193 ceph-mon[36001]: pgmap v230: 129 pgs: 129 active+clean; 77 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-05-14T15:32:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:17 smithi005 ceph-mon[31559]: pgmap v230: 129 pgs: 129 active+clean; 77 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-05-14T15:32:19.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 694140 10.34 MB/sec execute 220 sec latency 16.094 ms 2024-05-14T15:32:19.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:19 smithi005 ceph-mon[31559]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T15:32:19.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:19 smithi005 ceph-mon[31559]: pgmap v232: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-05-14T15:32:19.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:19 smithi144 ceph-mon[35650]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T15:32:19.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:19 smithi144 ceph-mon[35650]: pgmap v232: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-05-14T15:32:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:19 smithi193 ceph-mon[36001]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T15:32:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:19 smithi193 ceph-mon[36001]: pgmap v232: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-05-14T15:32:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 -- 172.21.15.5:0/106000625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 msgr2=0x7fe7c8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/106000625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c8108200 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0099c0 tx=0x7fe7bc009cd0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 -- 172.21.15.5:0/106000625 shutdown_connections 2024-05-14T15:32:19.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/106000625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c8108740 0x7fe7c81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/106000625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7c81034c0 0x7fe7c81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/106000625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c8108200 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 -- 172.21.15.5:0/106000625 >> 172.21.15.5:0/106000625 conn(0x7fe7c80fedc0 msgr2=0x7fe7c81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:19.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 -- 172.21.15.5:0/106000625 shutdown_connections 2024-05-14T15:32:19.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.693+0000 7fe7cc98d700 1 -- 172.21.15.5:0/106000625 wait complete. 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.694+0000 7fe7cc98d700 1 Processor -- start 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 -- start start 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c81034c0 0x7fe7c819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7c8108740 0x7fe7c8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8113350 con 0x7fe7c8107df0 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81131d0 con 0x7fe7c81034c0 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7cc98d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81134d0 con 0x7fe7c8108740 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c819fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54150/0 (socket says 172.21.15.5:54150) 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c81034c0 0x7fe7c819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 -- 172.21.15.5:0/3447896663 learned_addr learned my addr 172.21.15.5:0/3447896663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7c8108740 0x7fe7c8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 -- 172.21.15.5:0/3447896663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7c8108740 msgr2=0x7fe7c8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7c8108740 0x7fe7c8199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 -- 172.21.15.5:0/3447896663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c81034c0 msgr2=0x7fe7c819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c81034c0 0x7fe7c819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c5d9b700 1 -- 172.21.15.5:0/3447896663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c819a260 con 0x7fe7c8107df0 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.695+0000 7fe7c659c700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c81034c0 0x7fe7c819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.696+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c819fee0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc006010 tx=0x7fe7bc005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.696+0000 7fe7b77fe700 1 -- 172.21.15.5:0/3447896663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc026070 con 0x7fe7c8107df0 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.696+0000 7fe7b77fe700 1 -- 172.21.15.5:0/3447896663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7bc005680 con 0x7fe7c8107df0 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.696+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c819a4f0 con 0x7fe7c8107df0 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.696+0000 7fe7b77fe700 1 -- 172.21.15.5:0/3447896663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00fb70 con 0x7fe7c8107df0 2024-05-14T15:32:19.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.696+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a6750 con 0x7fe7c8107df0 2024-05-14T15:32:19.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.699+0000 7fe7b77fe700 1 -- 172.21.15.5:0/3447896663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe7bc029070 con 0x7fe7c8107df0 2024-05-14T15:32:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.700+0000 7fe7b77fe700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe7b00b29f0 0x7fe7b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.700+0000 7fe7db3a1b80 1 Processor -- start 2024-05-14T15:32:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.700+0000 7fe7c659c700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe7b00b29f0 0x7fe7b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.700+0000 7fe7db3a1b80 1 -- start start 2024-05-14T15:32:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.700+0000 7fe7b77fe700 1 -- 172.21.15.5:0/3447896663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fe7bc0e30a0 con 0x7fe7c8107df0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbea9a0 0x56182fbeadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe7b00b29f0 0x7fe7b00b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009680 tx=0x7fe7b8009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbf8ae0 0x56182fbfaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56182fb0c860 con 0x56182fa6ce40 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56182faab7c0 con 0x56182fbf8ae0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56182fb08820 con 0x56182fbea9a0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbea460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54166/0 (socket says 172.21.15.5:54166) 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 -- 172.21.15.5:0/176392665 learned_addr learned my addr 172.21.15.5:0/176392665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbea9a0 0x56182fbeadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbf8ae0 0x56182fbfaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 -- 172.21.15.5:0/176392665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbea9a0 msgr2=0x56182fbeadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbea9a0 0x56182fbeadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 -- 172.21.15.5:0/176392665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbf8ae0 msgr2=0x56182fbfaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbf8ae0 0x56182fbfaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 -- 172.21.15.5:0/176392665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182fa598d0 con 0x56182fa6ce40 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbea9a0 0x56182fbeadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.701+0000 7fe7c659c700 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbea460 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e6e0 tx=0x7fe7b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e157fae628d5f1c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.702+0000 7fe7b4ff9700 1 -- 172.21.15.5:0/176392665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8026630 con 0x56182fa6ce40 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.702+0000 7fe7b4ff9700 1 -- 172.21.15.5:0/176392665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b801c8c0 con 0x56182fa6ce40 2024-05-14T15:32:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.702+0000 7fe7b4ff9700 1 -- 172.21.15.5:0/176392665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b801d870 con 0x56182fa6ce40 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/176392665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 msgr2=0x56182fbea460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbea460 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e6e0 tx=0x7fe7b8014000 comp rx=0 tx=0).stop 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/176392665 shutdown_connections 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbf8ae0 0x56182fbfaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbea460 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/176392665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbea9a0 0x56182fbeadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/176392665 >> 172.21.15.5:0/176392665 conn(0x56182fab4600 msgr2=0x56182fbe54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:19.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/176392665 shutdown_connections 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/176392665 wait complete. 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 -- start start 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.703+0000 7fe7db3a1b80 1 Processor -- start 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbf59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbf6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56182fbfb810 con 0x56182fa6ce40 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56182fa72c30 con 0x56182fbea9a0 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56182fbfb990 con 0x56182fbf8ae0 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7db3a1b80 10 client.0 _ll_register_callbacks cb 0x56182faa4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbf59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbf59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54178/0 (socket says 172.21.15.5:54178) 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 -- 172.21.15.5:0/3537542027 learned_addr learned my addr 172.21.15.5:0/3537542027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:19.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbf6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 -- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 msgr2=0x56182fbf6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbf6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 -- 172.21.15.5:0/3537542027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 msgr2=0x56182fbf5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 -- 172.21.15.5:0/3537542027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182faa8690 con 0x56182fa6ce40 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7c659c700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbf59e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800ecd0 tx=0x7fe7b801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d44e3251627f425 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.704+0000 7fe7adffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7adffb700 1 -- 172.21.15.5:0/3537542027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8034060 con 0x56182fa6ce40 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7adffb700 1 -- 172.21.15.5:0/3537542027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fe7b801cc80 con 0x56182fa6ce40 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b801c8c0 con 0x56182fa6ce40 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7adffb700 1 -- 172.21.15.5:0/3537542027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7b803b050 con 0x56182fa6ce40 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 10 client.17190 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56182fc06380 con 0x56182fa6ce40 2024-05-14T15:32:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7adffb700 1 -- 172.21.15.5:0/3537542027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b801c8c0 con 0x56182fa6ce40 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7adffb700 1 -- 172.21.15.5:0/3537542027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe7b8004a70 con 0x56182fa6ce40 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 10 client.17190 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 10 client.17190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 20 client.17190 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc08900 0x56182fc0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 4 client.17190 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.705+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182fb1a240 con 0x56182fc08900 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.706+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc08900 0x56182fc0ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.706+0000 7fe7adffb700 10 client.17190 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:32:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.706+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc08900 0x56182fc0ad20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64ae4c104b4fdc24 server_cookie=1e5e4f0589b34a8a in_seq=0 out_seq=0 2024-05-14T15:32:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7adffb700 1 -- 172.21.15.5:0/3537542027 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56182fb1a240 con 0x56182fc08900 2024-05-14T15:32:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7adffb700 10 client.17190 handle_command_reply: tid=0 2024-05-14T15:32:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 client.17190 shutdown 2024-05-14T15:32:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 msgr2=0x56182fbf59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbf59e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800ecd0 tx=0x7fe7b801c560 comp rx=0 tx=0).stop 2024-05-14T15:32:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 shutdown_connections 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbf6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fbf59e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc08900 0x56182fc0ad20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/3537542027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7adffb700 0 client.17190 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 >> 172.21.15.5:0/3537542027 conn(0x56182fab4600 msgr2=0x56182fbe7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 shutdown_connections 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/3537542027 wait complete. 2024-05-14T15:32:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.707+0000 7fe7db3a1b80 20 client.17190 trim_cache size 0 max 16384 2024-05-14T15:32:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.722+0000 7fe7db3a1b80 1 Processor -- start 2024-05-14T15:32:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.722+0000 7fe7db3a1b80 1 -- start start 2024-05-14T15:32:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fc04c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbe52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81b0d30 con 0x56182fa6ce40 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81ae5c0 con 0x56182fbea9a0 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81ac430 con 0x56182fbf8ae0 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fc04c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fc04c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54182/0 (socket says 172.21.15.5:54182) 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 -- 172.21.15.5:0/1194374580 learned_addr learned my addr 172.21.15.5:0/1194374580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbe52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c5d9b700 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 -- 172.21.15.5:0/1194374580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 msgr2=0x56182fbe52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbe52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 -- 172.21.15.5:0/1194374580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 msgr2=0x56182fbf62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 -- 172.21.15.5:0/1194374580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182fbf67e0 con 0x56182fa6ce40 2024-05-14T15:32:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.723+0000 7fe7c659c700 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fc04c30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8016040 tx=0x7fe7b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272397048691a03f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7affff700 1 -- 172.21.15.5:0/1194374580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8044070 con 0x56182fa6ce40 2024-05-14T15:32:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7affff700 1 -- 172.21.15.5:0/1194374580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b801de30 con 0x56182fa6ce40 2024-05-14T15:32:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7affff700 1 -- 172.21.15.5:0/1194374580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8034040 con 0x56182fa6ce40 2024-05-14T15:32:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1194374580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 msgr2=0x56182fc04c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fc04c30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8016040 tx=0x7fe7b800eea0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1194374580 shutdown_connections 2024-05-14T15:32:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fbf8ae0 0x56182fbe52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fa6ce40 0x56182fc04c30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1194374580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fbf62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1194374580 >> 172.21.15.5:0/1194374580 conn(0x56182fab4600 msgr2=0x56182fbe4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1194374580 shutdown_connections 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.724+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1194374580 wait complete. 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 -- start start 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 Processor -- start 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fa6ce40 0x56182fc0b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fc0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 0x56182fc70e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81aca80 con 0x56182fbf8ae0 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81adef0 con 0x56182fbea9a0 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81ad150 con 0x56182fa6ce40 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7db3a1b80 10 client.0 _ll_register_callbacks cb 0x56182faa4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 0x56182fc70e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.725+0000 7fe7c5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fc0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 0x56182fc70e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54194/0 (socket says 172.21.15.5:54194) 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 -- 172.21.15.5:0/1115807388 learned_addr learned my addr 172.21.15.5:0/1115807388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c659c700 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fa6ce40 0x56182fc0b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 -- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fa6ce40 msgr2=0x56182fc0b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fa6ce40 0x56182fc0b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 -- 172.21.15.5:0/1115807388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 msgr2=0x56182fc0b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fc0b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 -- 172.21.15.5:0/1115807388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182fc71380 con 0x56182fbf8ae0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7c6d9d700 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 0x56182fc70e40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00092a0 tx=0x7fe7c00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85413c5ee6c536e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7adffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7adffb700 1 -- 172.21.15.5:0/1115807388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0007cf0 con 0x56182fbf8ae0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7adffb700 1 -- 172.21.15.5:0/1115807388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fe7c0004bb0 con 0x56182fbf8ae0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b801de30 con 0x56182fbf8ae0 2024-05-14T15:32:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7adffb700 1 -- 172.21.15.5:0/1115807388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7c002a050 con 0x56182fbf8ae0 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7db3a1b80 10 client.17196 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.726+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56182fc72590 con 0x56182fbf8ae0 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7adffb700 1 -- 172.21.15.5:0/1115807388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b801de30 con 0x56182fbf8ae0 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7adffb700 1 -- 172.21.15.5:0/1115807388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe7c0016420 con 0x56182fbf8ae0 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7db3a1b80 10 client.17196 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7db3a1b80 10 client.17196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7db3a1b80 20 client.17196 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc72cf0 0x56182fc75110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7db3a1b80 4 client.17196 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56182fb142e0 con 0x56182fc72cf0 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7c659c700 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc72cf0 0x56182fc75110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7adffb700 10 client.17196 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:32:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.727+0000 7fe7c659c700 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc72cf0 0x56182fc75110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1b8ce577107bfd server_cookie=dfa129979ce243d1 in_seq=0 out_seq=0 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7adffb700 1 -- 172.21.15.5:0/1115807388 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56182fb142e0 con 0x56182fc72cf0 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7adffb700 10 client.17196 handle_command_reply: tid=0 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 client.17196 shutdown 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 msgr2=0x56182fc70e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 0x56182fc70e40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00092a0 tx=0x7fe7c00077e0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 shutdown_connections 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182fbf8ae0 0x56182fc70e40 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56182fa6ce40 0x56182fc0b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x56182fc72cf0 0x56182fc75110 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 --2- 172.21.15.5:0/1115807388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56182fbea9a0 0x56182fc0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 >> 172.21.15.5:0/1115807388 conn(0x56182fab4600 msgr2=0x56182fbe9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7adffb700 0 client.17196 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 shutdown_connections 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 1 -- 172.21.15.5:0/1115807388 wait complete. 2024-05-14T15:32:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.728+0000 7fe7db3a1b80 20 client.17196 trim_cache size 0 max 16384 2024-05-14T15:32:19.736 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:32:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe7b00b29f0 msgr2=0x7fe7b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe7b00b29f0 0x7fe7b00b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009680 tx=0x7fe7b8009260 comp rx=0 tx=0).stop 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 msgr2=0x7fe7c819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c819fee0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc006010 tx=0x7fe7bc005280 comp rx=0 tx=0).stop 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 shutdown_connections 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe7b00b29f0 0x7fe7b00b4f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7c8108740 0x7fe7c8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7c81034c0 0x7fe7c819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 --2- 172.21.15.5:0/3447896663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c8107df0 0x7fe7c819fee0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 >> 172.21.15.5:0/3447896663 conn(0x7fe7c80fedc0 msgr2=0x7fe7c81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 shutdown_connections 2024-05-14T15:32:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:19.729+0000 7fe7cc98d700 1 -- 172.21.15.5:0/3447896663 wait complete. 2024-05-14T15:32:20.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 696241 10.36 MB/sec execute 221 sec latency 167.050 ms 2024-05-14T15:32:20.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:32:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:21.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 697571 10.34 MB/sec execute 222 sec latency 19.988 ms 2024-05-14T15:32:22.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:21 smithi005 ceph-mon[31559]: pgmap v233: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-05-14T15:32:22.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 699910 10.35 MB/sec execute 223 sec latency 32.016 ms 2024-05-14T15:32:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:21 smithi144 ceph-mon[35650]: pgmap v233: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-05-14T15:32:22.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:21 smithi193 ceph-mon[36001]: pgmap v233: 129 pgs: 129 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-05-14T15:32:23.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 701566 10.33 MB/sec execute 224 sec latency 20.226 ms 2024-05-14T15:32:23.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 -- 172.21.15.5:0/3577390370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b401034c0 msgr2=0x7f6b401038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 --2- 172.21.15.5:0/3577390370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b401034c0 0x7f6b401038d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0099c0 tx=0x7f6b3c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 -- 172.21.15.5:0/3577390370 shutdown_connections 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 --2- 172.21.15.5:0/3577390370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 0x7f6b401100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 --2- 172.21.15.5:0/3577390370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b401034c0 0x7f6b401038d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 --2- 172.21.15.5:0/3577390370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b40107df0 0x7f6b40108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 -- 172.21.15.5:0/3577390370 >> 172.21.15.5:0/3577390370 conn(0x7f6b400fedc0 msgr2=0x7f6b401011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 -- 172.21.15.5:0/3577390370 shutdown_connections 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 -- 172.21.15.5:0/3577390370 wait complete. 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.704+0000 7f6b47595700 1 Processor -- start 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 -- start start 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b401034c0 0x7f6b4019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 0x7f6b4019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 0x7f6b401a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401144c0 con 0x7f6b40107df0 2024-05-14T15:32:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40114340 con 0x7f6b401034c0 2024-05-14T15:32:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b47595700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40114640 con 0x7f6b40108740 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b44b30700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 0x7f6b4019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45331700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b401034c0 0x7f6b4019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 0x7f6b401a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b44b30700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 0x7f6b4019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54200/0 (socket says 172.21.15.5:54200) 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b44b30700 1 -- 172.21.15.5:0/1877852405 learned_addr learned my addr 172.21.15.5:0/1877852405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45b32700 1 -- 172.21.15.5:0/1877852405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b401034c0 msgr2=0x7f6b4019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45b32700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b401034c0 0x7f6b4019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45b32700 1 -- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 msgr2=0x7f6b4019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45b32700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 0x7f6b4019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45b32700 1 -- 172.21.15.5:0/1877852405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b401a1ea0 con 0x7f6b40108740 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b45331700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b401034c0 0x7f6b4019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.705+0000 7f6b44b30700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 0x7f6b4019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.706+0000 7f6b45b32700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 0x7f6b401a1800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b38007f10 tx=0x7f6b3800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.706+0000 7f6b327fc700 1 -- 172.21.15.5:0/1877852405 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38004db0 con 0x7f6b40108740 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.706+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b401a2190 con 0x7f6b40108740 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.706+0000 7f6b327fc700 1 -- 172.21.15.5:0/1877852405 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b38009e20 con 0x7f6b40108740 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.706+0000 7f6b327fc700 1 -- 172.21.15.5:0/1877852405 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38015400 con 0x7f6b40108740 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.706+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b401a9ab0 con 0x7f6b40108740 2024-05-14T15:32:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.709+0000 7f6b327fc700 1 -- 172.21.15.5:0/1877852405 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6b38005260 con 0x7f6b40108740 2024-05-14T15:32:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b327fc700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6b2c0b29f0 0x7f6b2c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 Processor -- start 2024-05-14T15:32:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b327fc700 1 -- 172.21.15.5:0/1877852405 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f6b380e2330 con 0x7f6b40108740 2024-05-14T15:32:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b45331700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6b2c0b29f0 0x7f6b2c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 -- start start 2024-05-14T15:32:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe06840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe06d80 0x5607fbe008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 0x5607fbe01220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5607fbd22940 con 0x5607fbe00e10 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5607fbc87da0 con 0x5607fbe06d80 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607fbd1e900 con 0x5607fbc82e40 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.710+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 0x5607fbe01220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45331700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6b2c0b29f0 0x7f6b2c0b4f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c000c00 tx=0x7f6b3c01a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 0x5607fbe01220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54210/0 (socket says 172.21.15.5:54210) 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 -- 172.21.15.5:0/52421892 learned_addr learned my addr 172.21.15.5:0/52421892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b44b30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe06d80 0x5607fbe008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45331700 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe06840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 -- 172.21.15.5:0/52421892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 msgr2=0x5607fbe06840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe06840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 -- 172.21.15.5:0/52421892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe06d80 msgr2=0x5607fbe008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe06d80 0x5607fbe008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 -- 172.21.15.5:0/52421892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fbc6f8d0 con 0x5607fbe00e10 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b44b30700 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe06d80 0x5607fbe008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b45b32700 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 0x5607fbe01220 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6b380a5280 tx=0x7f6b380a8d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec757ffd7a797c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b2b7fe700 1 -- 172.21.15.5:0/52421892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380a9200 con 0x5607fbe00e10 2024-05-14T15:32:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b2b7fe700 1 -- 172.21.15.5:0/52421892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b380ad5b0 con 0x5607fbe00e10 2024-05-14T15:32:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.711+0000 7f6b2b7fe700 1 -- 172.21.15.5:0/52421892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380ad120 con 0x5607fbe00e10 2024-05-14T15:32:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/52421892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 msgr2=0x5607fbe01220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 0x5607fbe01220 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f6b380a5280 tx=0x7f6b380a8d60 comp rx=0 tx=0).stop 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/52421892 shutdown_connections 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe00e10 0x5607fbe01220 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe06840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/52421892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe06d80 0x5607fbe008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/52421892 >> 172.21.15.5:0/52421892 conn(0x5607fbcca630 msgr2=0x5607fbdfba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/52421892 shutdown_connections 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.712+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/52421892 wait complete. 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 -- start start 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 Processor -- start 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe0aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5607fbe0d7c0 con 0x5607fbe06d80 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5607fbcdbed0 con 0x5607fbe00e10 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607fbe0d940 con 0x5607fbc82e40 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.713+0000 7f6b55fa9b80 10 client.0 _ll_register_callbacks cb 0x5607fbcba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45331700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54212/0 (socket says 172.21.15.5:54212) 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 -- 172.21.15.5:0/83064704 learned_addr learned my addr 172.21.15.5:0/83064704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b44b30700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe0aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 -- 172.21.15.5:0/83064704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 msgr2=0x5607fbe0a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe0a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 -- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 msgr2=0x5607fbe0aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe0aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 -- 172.21.15.5:0/83064704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fbcd9760 con 0x5607fbe06d80 2024-05-14T15:32:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45331700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe0a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b44b30700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b45b32700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0af90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6b380df720 tx=0x7f6b380ad3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a6edffd00d2df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b28ff9700 1 -- 172.21.15.5:0/83064704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380c2070 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b28ff9700 1 -- 172.21.15.5:0/83064704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f6b380b1de0 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b380ad5b0 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b28ff9700 1 -- 172.21.15.5:0/83064704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b380b2060 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b55fa9b80 10 client.17208 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.714+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607fbe18410 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b28ff9700 1 -- 172.21.15.5:0/83064704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b380ad5b0 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b28ff9700 1 -- 172.21.15.5:0/83064704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6b380a4b70 con 0x5607fbe06d80 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b55fa9b80 10 client.17208 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b55fa9b80 10 client.17208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:32:23.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b55fa9b80 20 client.17208 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe1a8e0 0x5607fbe1cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b55fa9b80 4 client.17208 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607fbd30320 con 0x5607fbe1a8e0 2024-05-14T15:32:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.715+0000 7f6b45331700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe1a8e0 0x5607fbe1cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.716+0000 7f6b45331700 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe1a8e0 0x5607fbe1cd00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2cf3e2f1cba1dfb7 server_cookie=d188e8b120a93d95 in_seq=0 out_seq=0 2024-05-14T15:32:23.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.716+0000 7f6b28ff9700 10 client.17208 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b28ff9700 1 -- 172.21.15.5:0/83064704 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607fbd30320 con 0x5607fbe1a8e0 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b28ff9700 10 client.17208 handle_command_reply: tid=0 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 client.17208 shutdown 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 msgr2=0x5607fbe0af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0af90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6b380df720 tx=0x7f6b380ad3d0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 shutdown_connections 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe1a8e0 0x5607fbe1cd00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0af90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbc82e40 0x5607fbe0a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/83064704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 >> 172.21.15.5:0/83064704 conn(0x5607fbcca630 msgr2=0x5607fbdff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b28ff9700 0 client.17208 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 shutdown_connections 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/83064704 wait complete. 2024-05-14T15:32:23.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.717+0000 7f6b55fa9b80 20 client.17208 trim_cache size 0 max 16384 2024-05-14T15:32:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.733+0000 7f6b55fa9b80 1 Processor -- start 2024-05-14T15:32:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.733+0000 7f6b55fa9b80 1 -- start start 2024-05-14T15:32:23.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbc82e40 0x5607fbe008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe00e10 0x5607fbe05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401b5230 con 0x5607fbe06d80 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401b2ac0 con 0x5607fbc82e40 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401b0930 con 0x5607fbe00e10 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45331700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbc82e40 0x5607fbe008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54224/0 (socket says 172.21.15.5:54224) 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 -- 172.21.15.5:0/3497192510 learned_addr learned my addr 172.21.15.5:0/3497192510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:23.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b44b30700 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe00e10 0x5607fbe05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 -- 172.21.15.5:0/3497192510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe00e10 msgr2=0x5607fbe05da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe00e10 0x5607fbe05da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 -- 172.21.15.5:0/3497192510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbc82e40 msgr2=0x5607fbe008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbc82e40 0x5607fbe008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 -- 172.21.15.5:0/3497192510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fbe06bb0 con 0x5607fbe06d80 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b44b30700 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe00e10 0x5607fbe05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.734+0000 7f6b45b32700 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0c9b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6b380b2220 tx=0x7f6b380df770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4a4c46f083b974 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.735+0000 7f6b2affd700 1 -- 172.21.15.5:0/3497192510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380c2070 con 0x5607fbe06d80 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.735+0000 7f6b2affd700 1 -- 172.21.15.5:0/3497192510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b380a9200 con 0x5607fbe06d80 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.735+0000 7f6b2affd700 1 -- 172.21.15.5:0/3497192510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380a8150 con 0x5607fbe06d80 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.735+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/3497192510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 msgr2=0x5607fbe0c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.735+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0c9b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6b380b2220 tx=0x7f6b380df770 comp rx=0 tx=0).stop 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/3497192510 shutdown_connections 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbe06d80 0x5607fbe0c9b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbc82e40 0x5607fbe008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/3497192510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe00e10 0x5607fbe05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/3497192510 >> 172.21.15.5:0/3497192510 conn(0x5607fbcca630 msgr2=0x5607fbe08e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/3497192510 shutdown_connections 2024-05-14T15:32:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/3497192510 wait complete. 2024-05-14T15:32:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 -- start start 2024-05-14T15:32:23.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.736+0000 7f6b55fa9b80 1 Processor -- start 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 0x5607fbe1c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe1ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe06d80 0x5607fbe1d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401b0f80 con 0x5607fbc82e40 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401b23f0 con 0x5607fbe00e10 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401b1650 con 0x5607fbe06d80 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b45331700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 0x5607fbe1c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b45331700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 0x5607fbe1c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54226/0 (socket says 172.21.15.5:54226) 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b45331700 1 -- 172.21.15.5:0/687176705 learned_addr learned my addr 172.21.15.5:0/687176705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b55fa9b80 10 client.0 _ll_register_callbacks cb 0x5607fbcba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b44b30700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe1ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.737+0000 7f6b45b32700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe06d80 0x5607fbe1d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45331700 1 -- 172.21.15.5:0/687176705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe06d80 msgr2=0x5607fbe1d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45331700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe06d80 0x5607fbe1d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45331700 1 -- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 msgr2=0x5607fbe1ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45331700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe1ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45331700 1 -- 172.21.15.5:0/687176705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fbe82f20 con 0x5607fbc82e40 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45b32700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe06d80 0x5607fbe1d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b45331700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 0x5607fbe1c8f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009fd0 tx=0x7f6b3c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2308594f69048b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b28ff9700 1 -- 172.21.15.5:0/687176705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c038060 con 0x5607fbc82e40 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b28ff9700 1 -- 172.21.15.5:0/687176705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f6b3c00f460 con 0x5607fbc82e40 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b380a9200 con 0x5607fbc82e40 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b28ff9700 1 -- 172.21.15.5:0/687176705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b3c03f050 con 0x5607fbc82e40 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b55fa9b80 10 client.17214 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:23.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.738+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607fbe841e0 con 0x5607fbc82e40 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b28ff9700 1 -- 172.21.15.5:0/687176705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b380a9200 con 0x5607fbc82e40 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b28ff9700 1 -- 172.21.15.5:0/687176705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6b3c0327b0 con 0x5607fbc82e40 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b55fa9b80 10 client.17214 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b55fa9b80 10 client.17214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b55fa9b80 20 client.17214 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe84870 0x5607fbe86c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b55fa9b80 4 client.17214 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607fbd2a3c0 con 0x5607fbe84870 2024-05-14T15:32:23.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.739+0000 7f6b44b30700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe84870 0x5607fbe86c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b28ff9700 10 client.17214 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:32:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b44b30700 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe84870 0x5607fbe86c90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb7f1055981a3ce9 server_cookie=11d4e57f18faa7a7 in_seq=0 out_seq=0 2024-05-14T15:32:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b28ff9700 1 -- 172.21.15.5:0/687176705 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607fbd2a3c0 con 0x5607fbe84870 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b28ff9700 10 client.17214 handle_command_reply: tid=0 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 client.17214 shutdown 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 msgr2=0x5607fbe1c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 0x5607fbe1c8f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009fd0 tx=0x7f6b3c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 shutdown_connections 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5607fbe84870 0x5607fbe86c90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607fbe06d80 0x5607fbe1d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b28ff9700 0 client.17214 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fbc82e40 0x5607fbe1c8f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 --2- 172.21.15.5:0/687176705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607fbe00e10 0x5607fbe1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 >> 172.21.15.5:0/687176705 conn(0x5607fbcca630 msgr2=0x5607fbdf5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 shutdown_connections 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 1 -- 172.21.15.5:0/687176705 wait complete. 2024-05-14T15:32:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.740+0000 7f6b55fa9b80 20 client.17214 trim_cache size 0 max 16384 2024-05-14T15:32:23.747 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:32:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6b2c0b29f0 msgr2=0x7f6b2c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6b2c0b29f0 0x7f6b2c0b4f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c000c00 tx=0x7f6b3c01a040 comp rx=0 tx=0).stop 2024-05-14T15:32:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 msgr2=0x7f6b401a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 0x7f6b401a1800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b38007f10 tx=0x7f6b3800d3b0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 shutdown_connections 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6b2c0b29f0 0x7f6b2c0b4f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b40108740 0x7f6b401a1800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b401034c0 0x7f6b4019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 --2- 172.21.15.5:0/1877852405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b40107df0 0x7f6b4019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 >> 172.21.15.5:0/1877852405 conn(0x7f6b400fedc0 msgr2=0x7f6b401075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 shutdown_connections 2024-05-14T15:32:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:23.741+0000 7f6b47595700 1 -- 172.21.15.5:0/1877852405 wait complete. 2024-05-14T15:32:24.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:23 smithi005 ceph-mon[31559]: pgmap v234: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-05-14T15:32:24.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 703670 10.35 MB/sec execute 225 sec latency 23.658 ms 2024-05-14T15:32:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:23 smithi144 ceph-mon[35650]: pgmap v234: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-05-14T15:32:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:23 smithi193 ceph-mon[36001]: pgmap v234: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-05-14T15:32:24.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:32:24.814 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:25.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 704956 10.32 MB/sec execute 226 sec latency 106.830 ms 2024-05-14T15:32:26.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:25 smithi005 ceph-mon[31559]: pgmap v235: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-05-14T15:32:26.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 707061 10.34 MB/sec execute 227 sec latency 46.557 ms 2024-05-14T15:32:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:25 smithi144 ceph-mon[35650]: pgmap v235: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-05-14T15:32:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:25 smithi193 ceph-mon[36001]: pgmap v235: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-05-14T15:32:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 -- 172.21.15.5:0/508544885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc1034c0 msgr2=0x7f0cfc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 --2- 172.21.15.5:0/508544885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc1038d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80099c0 tx=0x7f0cf8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 -- 172.21.15.5:0/508544885 shutdown_connections 2024-05-14T15:32:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 --2- 172.21.15.5:0/508544885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cfc108740 0x7f0cfc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 --2- 172.21.15.5:0/508544885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc1038d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 --2- 172.21.15.5:0/508544885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc107df0 0x7f0cfc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 -- 172.21.15.5:0/508544885 >> 172.21.15.5:0/508544885 conn(0x7f0cfc0fedf0 msgr2=0x7f0cfc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 -- 172.21.15.5:0/508544885 shutdown_connections 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.628+0000 7f0d02def700 1 -- 172.21.15.5:0/508544885 wait complete. 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 Processor -- start 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 -- start start 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc107df0 0x7f0cfc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cfc108740 0x7f0cfc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc113370 con 0x7f0cfc107df0 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1131f0 con 0x7f0cfc1034c0 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d02def700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1134f0 con 0x7f0cfc108740 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37728/0 (socket says 172.21.15.5:37728) 2024-05-14T15:32:26.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d00b8b700 1 -- 172.21.15.5:0/268452418 learned_addr learned my addr 172.21.15.5:0/268452418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0d0138c700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cfc108740 0x7f0cfc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.629+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc107df0 0x7f0cfc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d00b8b700 1 -- 172.21.15.5:0/268452418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cfc108740 msgr2=0x7f0cfc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cfc108740 0x7f0cfc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d00b8b700 1 -- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc107df0 msgr2=0x7f0cfc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc107df0 0x7f0cfc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d00b8b700 1 -- 172.21.15.5:0/268452418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc19fcc0 con 0x7f0cfc1034c0 2024-05-14T15:32:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc19adf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006010 tx=0x7f0cf80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0cf1ffb700 1 -- 172.21.15.5:0/268452418 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8026070 con 0x7f0cfc1034c0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc19ff50 con 0x7f0cfc1034c0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0cf1ffb700 1 -- 172.21.15.5:0/268452418 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf800f460 con 0x7f0cfc1034c0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.630+0000 7f0cf1ffb700 1 -- 172.21.15.5:0/268452418 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8009e60 con 0x7f0cfc1034c0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.631+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a78d0 con 0x7f0cfc1034c0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.633+0000 7f0cf1ffb700 1 -- 172.21.15.5:0/268452418 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0cf8029070 con 0x7f0cfc1034c0 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.634+0000 7f0cf1ffb700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce40b2a70 0x7f0ce40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.634+0000 7f0d11803b80 1 Processor -- start 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.634+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce40b2a70 0x7f0ce40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 -- start start 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0cf1ffb700 1 -- 172.21.15.5:0/268452418 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f0cf80e3160 con 0x7f0cfc1034c0 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b56e10 0x561106b57220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561106a78940 con 0x5611069d8e40 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5611069ddda0 con 0x561106b56e10 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d11803b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561106a74900 con 0x561106b5cd80 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce40b2a70 0x7f0ce40b4f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc109970 tx=0x7f0cec009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d0138c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b56e10 0x561106b57220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b5c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58078/0 (socket says 172.21.15.5:58078) 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d00b8b700 1 -- 172.21.15.5:0/3507988527 learned_addr learned my addr 172.21.15.5:0/3507988527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d00b8b700 1 -- 172.21.15.5:0/3507988527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 msgr2=0x561106b568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.635+0000 7f0d00b8b700 1 -- 172.21.15.5:0/3507988527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b56e10 msgr2=0x561106b57220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b56e10 0x561106b57220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0d00b8b700 1 -- 172.21.15.5:0/3507988527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611069c58d0 con 0x5611069d8e40 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0d0138c700 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b56e10 0x561106b57220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b5c840 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80a6140 tx=0x7f0cf80a9a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598a47aba4a71e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0ce2ffd700 1 -- 172.21.15.5:0/3507988527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80aa7c0 con 0x5611069d8e40 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0ce2ffd700 1 -- 172.21.15.5:0/3507988527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf80aa0c0 con 0x5611069d8e40 2024-05-14T15:32:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.636+0000 7f0ce2ffd700 1 -- 172.21.15.5:0/3507988527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80aa320 con 0x5611069d8e40 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 -- 172.21.15.5:0/3507988527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 msgr2=0x561106b5c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b5c840 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80a6140 tx=0x7f0cf80a9a10 comp rx=0 tx=0).stop 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 -- 172.21.15.5:0/3507988527 shutdown_connections 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b56e10 0x561106b57220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b5c840 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3507988527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 -- 172.21.15.5:0/3507988527 >> 172.21.15.5:0/3507988527 conn(0x561106a20630 msgr2=0x561106b51a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 -- 172.21.15.5:0/3507988527 shutdown_connections 2024-05-14T15:32:26.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 -- 172.21.15.5:0/3507988527 wait complete. 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.637+0000 7f0d11803b80 1 -- start start 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 Processor -- start 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561106b635e0 con 0x5611069d8e40 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561106a31ed0 con 0x561106b5cd80 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561106b63760 con 0x561106b56e10 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d11803b80 10 client.0 _ll_register_callbacks cb 0x561106a10950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d0138c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d0138c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37738/0 (socket says 172.21.15.5:37738) 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58086/0 (socket says 172.21.15.5:58086) 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.638+0000 7f0d0138c700 1 -- 172.21.15.5:0/3482875973 learned_addr learned my addr 172.21.15.5:0/3482875973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d0138c700 1 -- 172.21.15.5:0/3482875973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 msgr2=0x561106b60a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d0138c700 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b60a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d0138c700 1 -- 172.21.15.5:0/3482875973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 msgr2=0x561106b604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d0138c700 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d0138c700 1 -- 172.21.15.5:0/3482875973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561106a2f760 con 0x561106b5cd80 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d0138c700 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40040e0 tx=0x7f0cf4004110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0432779d0a6028 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0cdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0cdbfff700 1 -- 172.21.15.5:0/3482875973 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf401a070 con 0x561106b5cd80 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0cdbfff700 1 -- 172.21.15.5:0/3482875973 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f0cf4007700 con 0x561106b5cd80 2024-05-14T15:32:26.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.639+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf80aa0c0 con 0x561106b5cd80 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0cdbfff700 1 -- 172.21.15.5:0/3482875973 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cf4004bb0 con 0x561106b5cd80 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 10 client.26849 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561106b6e180 con 0x561106b5cd80 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0cdbfff700 1 -- 172.21.15.5:0/3482875973 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf80aa0c0 con 0x561106b5cd80 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0cdbfff700 1 -- 172.21.15.5:0/3482875973 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0cf40167a0 con 0x561106b5cd80 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 10 client.26849 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 20 client.26849 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106b72c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 4 client.26849 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.640+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561106a86320 con 0x561106b70760 2024-05-14T15:32:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.641+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106b72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.641+0000 7f0cdbfff700 10 client.26849 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.641+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106b72c00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78df9224d86af1ed server_cookie=27abf40a75969687 in_seq=0 out_seq=0 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0cdbfff700 1 -- 172.21.15.5:0/3482875973 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561106a86320 con 0x561106b70760 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0cdbfff700 10 client.26849 handle_command_reply: tid=0 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 client.26849 shutdown 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 msgr2=0x561106b60f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40040e0 tx=0x7f0cf4004110 comp rx=0 tx=0).stop 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 shutdown_connections 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106b72c00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 --2- 172.21.15.5:0/3482875973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0cdbfff700 0 client.26849 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 >> 172.21.15.5:0/3482875973 conn(0x561106a20630 msgr2=0x561106b558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:26.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 shutdown_connections 2024-05-14T15:32:26.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.642+0000 7f0d11803b80 1 -- 172.21.15.5:0/3482875973 wait complete. 2024-05-14T15:32:26.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.643+0000 7f0d11803b80 20 client.26849 trim_cache size 0 max 16384 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.659+0000 7f0d11803b80 1 Processor -- start 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.659+0000 7f0d11803b80 1 -- start start 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b5bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d11803b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1b2fd0 con 0x5611069d8e40 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d11803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1b0860 con 0x561106b5cd80 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d11803b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1ae6d0 con 0x561106b56e10 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58102/0 (socket says 172.21.15.5:58102) 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b5bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 -- 172.21.15.5:0/976605977 learned_addr learned my addr 172.21.15.5:0/976605977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:26.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d0138c700 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 -- 172.21.15.5:0/976605977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 msgr2=0x561106b5bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b5bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 -- 172.21.15.5:0/976605977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 msgr2=0x561106b60bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.660+0000 7f0d00b8b700 1 -- 172.21.15.5:0/976605977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561106b73190 con 0x5611069d8e40 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.661+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b568d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80a7330 tx=0x7f0cf80a9b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8432e1f0441e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:26.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.661+0000 7f0ce27fc700 1 -- 172.21.15.5:0/976605977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80c3070 con 0x5611069d8e40 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.661+0000 7f0ce27fc700 1 -- 172.21.15.5:0/976605977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf80ae470 con 0x5611069d8e40 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.661+0000 7f0ce27fc700 1 -- 172.21.15.5:0/976605977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800f910 con 0x5611069d8e40 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 -- 172.21.15.5:0/976605977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 msgr2=0x561106b568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b568d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80a7330 tx=0x7f0cf80a9b40 comp rx=0 tx=0).stop 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 -- 172.21.15.5:0/976605977 shutdown_connections 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561106b5cd80 0x561106b60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611069d8e40 0x561106b568d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 --2- 172.21.15.5:0/976605977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b56e10 0x561106b5bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 -- 172.21.15.5:0/976605977 >> 172.21.15.5:0/976605977 conn(0x561106a20630 msgr2=0x561106b604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 -- 172.21.15.5:0/976605977 shutdown_connections 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 -- 172.21.15.5:0/976605977 wait complete. 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 -- start start 2024-05-14T15:32:26.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.662+0000 7f0d11803b80 1 Processor -- start 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 0x561106b689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561106b56e10 0x561106b68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1aed20 con 0x561106b56e10 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1b0190 con 0x5611069d8e40 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1af3f0 con 0x561106b5cd80 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 0x561106b689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561106b56e10 0x561106b68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d11803b80 10 client.0 _ll_register_callbacks cb 0x561106a10950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d00b8b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 0x561106b689b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37756/0 (socket says 172.21.15.5:37756) 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d00b8b700 1 -- 172.21.15.5:0/1853809866 learned_addr learned my addr 172.21.15.5:0/1853809866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d0138c700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.663+0000 7f0d00b8b700 1 -- 172.21.15.5:0/1853809866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 msgr2=0x561106b773b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b773b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d00b8b700 1 -- 172.21.15.5:0/1853809866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561106b56e10 msgr2=0x561106b68ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561106b56e10 0x561106b68ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d00b8b700 1 -- 172.21.15.5:0/1853809866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561106b78da0 con 0x5611069d8e40 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d0138c700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d00b8b700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 0x561106b689b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80e0aa0 tx=0x7f0cf800fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41676206cf0cf310 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0cdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0cdbfff700 1 -- 172.21.15.5:0/1853809866 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80c3070 con 0x5611069d8e40 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561106a803c0 con 0x5611069d8e40 2024-05-14T15:32:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0cdbfff700 1 -- 172.21.15.5:0/1853809866 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f0cf800f910 con 0x5611069d8e40 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0cdbfff700 1 -- 172.21.15.5:0/1853809866 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80aa7c0 con 0x5611069d8e40 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d11803b80 10 client.26855 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561106bde8d0 con 0x5611069d8e40 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.664+0000 7f0cdbfff700 1 -- 172.21.15.5:0/1853809866 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561106a803c0 con 0x5611069d8e40 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0cdbfff700 1 -- 172.21.15.5:0/1853809866 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f0cf80b2a00 con 0x5611069d8e40 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0d11803b80 10 client.26855 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0d11803b80 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0d11803b80 20 client.26855 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0d11803b80 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106be0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0d11803b80 4 client.26855 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611068f8250 con 0x561106b70760 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106be0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0cdbfff700 10 client.26855 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.665+0000 7f0cf3fff700 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106be0c00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fef4573cbb52400 server_cookie=4aebfc2eea444609 in_seq=0 out_seq=0 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0cdbfff700 1 -- 172.21.15.5:0/1853809866 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611068f8250 con 0x561106b70760 2024-05-14T15:32:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0cdbfff700 10 client.26855 handle_command_reply: tid=0 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 client.26855 shutdown 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 msgr2=0x561106b689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 0x561106b689b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80e0aa0 tx=0x7f0cf800fcb0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 shutdown_connections 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561106b70760 0x561106be0c00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561106b5cd80 0x561106b773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611069d8e40 0x561106b689b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0cdbfff700 0 client.26855 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 --2- 172.21.15.5:0/1853809866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561106b56e10 0x561106b68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 >> 172.21.15.5:0/1853809866 conn(0x561106a20630 msgr2=0x561106b4bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 shutdown_connections 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 1 -- 172.21.15.5:0/1853809866 wait complete. 2024-05-14T15:32:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.666+0000 7f0d11803b80 20 client.26855 trim_cache size 0 max 16384 2024-05-14T15:32:26.673 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:32:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce40b2a70 msgr2=0x7f0ce40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce40b2a70 0x7f0ce40b4f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc109970 tx=0x7f0cec009260 comp rx=0 tx=0).stop 2024-05-14T15:32:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 msgr2=0x7f0cfc19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc19adf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006010 tx=0x7f0cf80045e0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 shutdown_connections 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0ce40b2a70 0x7f0ce40b4f80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cfc108740 0x7f0cfc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cfc1034c0 0x7f0cfc19adf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 --2- 172.21.15.5:0/268452418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cfc107df0 0x7f0cfc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 >> 172.21.15.5:0/268452418 conn(0x7f0cfc0fedf0 msgr2=0x7f0cfc075d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 shutdown_connections 2024-05-14T15:32:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:26.667+0000 7f0d02def700 1 -- 172.21.15.5:0/268452418 wait complete. 2024-05-14T15:32:27.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 708764 10.32 MB/sec execute 228 sec latency 34.141 ms 2024-05-14T15:32:27.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 -- 172.21.15.5:0/3151961472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec108ba0 msgr2=0x7fa4ec102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/3151961472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec102740 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009900 tx=0x7fa4dc02a410 comp rx=0 tx=0).stop 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 -- 172.21.15.5:0/3151961472 shutdown_connections 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/3151961472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4ec109040 0x7fa4ec102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/3151961472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/3151961472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec102740 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 -- 172.21.15.5:0/3151961472 >> 172.21.15.5:0/3151961472 conn(0x7fa4ec0782e0 msgr2=0x7fa4ec078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 -- 172.21.15.5:0/3151961472 shutdown_connections 2024-05-14T15:32:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.684+0000 7fa4f0a51640 1 -- 172.21.15.5:0/3151961472 wait complete. 2024-05-14T15:32:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.685+0000 7fa4f0a51640 1 Processor -- start 2024-05-14T15:32:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.685+0000 7fa4f0a51640 1 -- start start 2024-05-14T15:32:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4f0a51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec0ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4f0a51640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec0ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4f0a51640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4ec109040 0x7fa4ec0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4f0a51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec112de0 con 0x7fa4ec1032f0 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4f0a51640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec112c60 con 0x7fa4ec108ba0 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4f0a51640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec112f60 con 0x7fa4ec109040 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec0ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec0ff380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58116/0 (socket says 172.21.15.5:58116) 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2856314824 learned_addr learned my addr 172.21.15.5:0/2856314824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec0ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4ec109040 0x7fa4ec0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2856314824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4ec109040 msgr2=0x7fa4ec0ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4ec109040 0x7fa4ec0ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2856314824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec108ba0 msgr2=0x7fa4ec0ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec0ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2856314824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec1a8c10 con 0x7fa4ec1032f0 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.686+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec0ff380 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005ab0 tx=0x7fa4dc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.687+0000 7fa4db7fe640 1 -- 172.21.15.5:0/2856314824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc03f070 con 0x7fa4ec1032f0 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.687+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec1a8ea0 con 0x7fa4ec1032f0 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.687+0000 7fa4db7fe640 1 -- 172.21.15.5:0/2856314824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4dc031b50 con 0x7fa4ec1032f0 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.687+0000 7fa4db7fe640 1 -- 172.21.15.5:0/2856314824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc035030 con 0x7fa4ec1032f0 2024-05-14T15:32:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.687+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a92a0 con 0x7fa4ec1032f0 2024-05-14T15:32:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.689+0000 7fa4f79e8740 1 Processor -- start 2024-05-14T15:32:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 -- start start 2024-05-14T15:32:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4db7fe640 1 -- 172.21.15.5:0/2856314824 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa4dc042070 con 0x7fa4ec1032f0 2024-05-14T15:32:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620f26195d0 con 0x5620f23131b0 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5620f25d2890 con 0x5620f22ea2f0 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620f2615590 con 0x5620f22f24d0 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37792/0 (socket says 172.21.15.5:37792) 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3313871042 learned_addr learned my addr 172.21.15.5:0/3313871042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58122/0 (socket says 172.21.15.5:58122) 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.690+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3313871042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 msgr2=0x5620f26b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3313871042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 msgr2=0x5620f26b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3313871042 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f232b210 con 0x5620f22ea2f0 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4db7fe640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa4d40b2950 0x7fa4d40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26b4120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc100290 tx=0x7fa4dc10b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a588f5ecf23f99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa4d40b2950 0x7fa4d40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4d8ff9640 1 -- 172.21.15.5:0/3313871042 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc0c5e20 con 0x5620f22ea2f0 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4db7fe640 1 -- 172.21.15.5:0/2856314824 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fa4dc102fe0 con 0x7fa4ec1032f0 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4d8ff9640 1 -- 172.21.15.5:0/3313871042 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4dc0c9a70 con 0x5620f22ea2f0 2024-05-14T15:32:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.691+0000 7fa4d8ff9640 1 -- 172.21.15.5:0/3313871042 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc0c96d0 con 0x5620f22ea2f0 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3313871042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 msgr2=0x5620f26b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26b4120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc100290 tx=0x7fa4dc10b320 comp rx=0 tx=0).stop 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa4d40b2950 0x7fa4d40b4e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5620f26b0d00 tx=0x7fa4e4009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3313871042 shutdown_connections 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26b4120 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3313871042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3313871042 >> 172.21.15.5:0/3313871042 conn(0x5620f2307300 msgr2=0x5620f26b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3313871042 shutdown_connections 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3313871042 wait complete. 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.692+0000 7fa4f79e8740 1 -- start start 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 Processor -- start 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26be180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26be6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620f26c0ad0 con 0x5620f23131b0 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5620f25d6370 con 0x5620f22ea2f0 2024-05-14T15:32:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620f26c0c50 con 0x5620f22f24d0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4f79e8740 10 client.0 _ll_register_callbacks cb 0x5620f232aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26be180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26be180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37800/0 (socket says 172.21.15.5:37800) 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2917051414 learned_addr learned my addr 172.21.15.5:0/2917051414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26be6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2917051414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 msgr2=0x5620f26be6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26be6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2917051414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 msgr2=0x5620f26c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ea59c640 1 -- 172.21.15.5:0/2917051414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f23f7620 con 0x5620f22ea2f0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.693+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26be180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc02a9d0 tx=0x7fa4dc031f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d61c9cae8888ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/2917051414 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc049070 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4dc0c9a70 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/2917051414 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fa4dc031d00 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/2917051414 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0c98b0 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4f79e8740 10 client.26867 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5620f26c0ad0 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/2917051414 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4dc0c9a70 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/2917051414 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa4dc1005a0 con 0x5620f22ea2f0 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4f79e8740 10 client.26867 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4f79e8740 10 client.26867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.694+0000 7fa4f79e8740 20 client.26867 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.695+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22d1d80 0x5620f26cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.695+0000 7fa4f79e8740 4 client.26867 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.695+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f26271e0 con 0x5620f22d1d80 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.695+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22d1d80 0x5620f26cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.695+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22d1d80 0x5620f26cff50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e36112f066cb749 server_cookie=ef1d31358ea2615f in_seq=0 out_seq=0 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.695+0000 7fa4d1ffb640 10 client.26867 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/2917051414 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620f26271e0 con 0x5620f22d1d80 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4d1ffb640 10 client.26867 handle_command_reply: tid=0 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 client.26867 shutdown 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 msgr2=0x5620f26be180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26be180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc02a9d0 tx=0x7fa4dc031f50 comp rx=0 tx=0).stop 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 shutdown_connections 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22d1d80 0x5620f26cff50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f26be6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/2917051414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22ea2f0 0x5620f26be180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4d1ffb640 0 client.26867 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 >> 172.21.15.5:0/2917051414 conn(0x5620f2307300 msgr2=0x5620f26b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 shutdown_connections 2024-05-14T15:32:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.698+0000 7fa4f79e8740 1 -- 172.21.15.5:0/2917051414 wait complete. 2024-05-14T15:32:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.699+0000 7fa4f79e8740 20 client.26867 trim_cache size 0 max 16384 2024-05-14T15:32:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.714+0000 7fa4f79e8740 1 Processor -- start 2024-05-14T15:32:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.714+0000 7fa4f79e8740 1 -- start start 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22ea2f0 0x5620f26b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22f24d0 0x5620f26bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1b5fa0 con 0x5620f23131b0 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1b3870 con 0x5620f22f24d0 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1b16e0 con 0x5620f22ea2f0 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26d5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58140/0 (socket says 172.21.15.5:58140) 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 -- 172.21.15.5:0/431359526 learned_addr learned my addr 172.21.15.5:0/431359526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22ea2f0 0x5620f26b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22f24d0 0x5620f26bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 -- 172.21.15.5:0/431359526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22ea2f0 msgr2=0x5620f26b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22ea2f0 0x5620f26b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 -- 172.21.15.5:0/431359526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22f24d0 msgr2=0x5620f26bfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22f24d0 0x5620f26bfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 -- 172.21.15.5:0/431359526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f26d0280 con 0x5620f23131b0 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22f24d0 0x5620f26bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22ea2f0 0x5620f26b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.715+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26d5700 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4004390 tx=0x7fa4e401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7cb21d9531990b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.716+0000 7fa4d3fff640 1 -- 172.21.15.5:0/431359526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4004b20 con 0x5620f23131b0 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.716+0000 7fa4d3fff640 1 -- 172.21.15.5:0/431359526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e4004cc0 con 0x5620f23131b0 2024-05-14T15:32:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.716+0000 7fa4d3fff640 1 -- 172.21.15.5:0/431359526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40057b0 con 0x5620f23131b0 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- 172.21.15.5:0/431359526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 msgr2=0x5620f26d5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26d5700 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4004390 tx=0x7fa4e401a000 comp rx=0 tx=0).stop 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- 172.21.15.5:0/431359526 shutdown_connections 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f23131b0 0x5620f26d5700 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f22f24d0 0x5620f26bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/431359526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22ea2f0 0x5620f26b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- 172.21.15.5:0/431359526 >> 172.21.15.5:0/431359526 conn(0x5620f2307300 msgr2=0x5620f26bf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- 172.21.15.5:0/431359526 shutdown_connections 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- 172.21.15.5:0/431359526 wait complete. 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- start start 2024-05-14T15:32:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 Processor -- start 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 0x5620f2730660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f2730ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f23131b0 0x5620f26c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1b1d30 con 0x5620f22ea2f0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1b31a0 con 0x5620f23131b0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.717+0000 7fa4f79e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1b2400 con 0x5620f22f24d0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4f79e8740 10 client.0 _ll_register_callbacks cb 0x7fa4e4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 0x5620f2730660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 0x5620f2730660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58146/0 (socket says 172.21.15.5:58146) 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3870318791 learned_addr learned my addr 172.21.15.5:0/3870318791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f2730ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f23131b0 0x5620f26c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3870318791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 msgr2=0x5620f2730ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f2730ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f23131b0 msgr2=0x5620f26c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f23131b0 0x5620f26c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3870318791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f26c1860 con 0x5620f22ea2f0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f2730ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 0x5620f2730660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0d2de0 tx=0x7fa4dc106c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cc2b71fd414288 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/3870318791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc049020 con 0x5620f22ea2f0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/3870318791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fa4dc106d20 con 0x5620f22ea2f0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.718+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e4004cc0 con 0x5620f22ea2f0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/3870318791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa4dc0c9d30 con 0x5620f22ea2f0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 10 client.17250 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5620f26c1860 con 0x5620f22ea2f0 2024-05-14T15:32:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/3870318791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e4004cc0 con 0x5620f22ea2f0 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/3870318791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa4dc0ecce0 con 0x5620f22ea2f0 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 10 client.17250 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 20 client.17250 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22a71c0 0x5620f273a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 4 client.17250 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620f2621050 con 0x5620f22a71c0 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.719+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22a71c0 0x5620f273a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.720+0000 7fa4d1ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:27.720+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22a71c0 0x5620f273a000 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95426ecca3e91142 server_cookie=d2ac628b1c4505c4 in_seq=0 out_seq=0 2024-05-14T15:32:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/3870318791 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620f2621050 con 0x5620f22a71c0 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4d1ffb640 10 client.17250 handle_command_reply: tid=0 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 client.17250 shutdown 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 msgr2=0x5620f2730660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 0x5620f2730660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0d2de0 tx=0x7fa4dc106c60 comp rx=0 tx=0).stop 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 shutdown_connections 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5620f22a71c0 0x5620f273a000 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5620f22f24d0 0x5620f2730ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5620f23131b0 0x5620f26c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4d1ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 --2- 172.21.15.5:0/3870318791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620f22ea2f0 0x5620f2730660 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.010+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 >> 172.21.15.5:0/3870318791 conn(0x5620f2307300 msgr2=0x5620f26b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.011+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 shutdown_connections 2024-05-14T15:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.011+0000 7fa4f79e8740 1 -- 172.21.15.5:0/3870318791 wait complete. 2024-05-14T15:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.011+0000 7fa4f79e8740 20 client.17250 trim_cache size 0 max 16384 2024-05-14T15:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:32:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa4d40b2950 msgr2=0x7fa4d40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa4d40b2950 0x7fa4d40b4e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5620f26b0d00 tx=0x7fa4e4009500 comp rx=0 tx=0).stop 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 msgr2=0x7fa4ec0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec0ff380 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005ab0 tx=0x7fa4dc003b90 comp rx=0 tx=0).stop 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 shutdown_connections 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa4d40b2950 0x7fa4d40b4e60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4ec109040 0x7fa4ec0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4ec108ba0 0x7fa4ec0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 --2- 172.21.15.5:0/2856314824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1032f0 0x7fa4ec0ff380 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 >> 172.21.15.5:0/2856314824 conn(0x7fa4ec0782e0 msgr2=0x7fa4ec06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 shutdown_connections 2024-05-14T15:32:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:28.012+0000 7fa4f0a51640 1 -- 172.21.15.5:0/2856314824 wait complete. 2024-05-14T15:32:28.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 710490 10.33 MB/sec execute 229 sec latency 193.791 ms 2024-05-14T15:32:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:27 smithi144 ceph-mon[35650]: pgmap v236: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-05-14T15:32:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:27 smithi193 ceph-mon[36001]: pgmap v236: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-05-14T15:32:28.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:27 smithi005 ceph-mon[31559]: pgmap v236: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-05-14T15:32:28.481 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:32:28.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:32:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:29.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 712003 10.31 MB/sec execute 230 sec latency 18.650 ms 2024-05-14T15:32:30.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 714060 10.32 MB/sec execute 231 sec latency 46.397 ms 2024-05-14T15:32:30.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:29 smithi005 ceph-mon[31559]: pgmap v237: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-05-14T15:32:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:29 smithi144 ceph-mon[35650]: pgmap v237: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-05-14T15:32:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:29 smithi193 ceph-mon[36001]: pgmap v237: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-05-14T15:32:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.755+0000 7faceed9f700 1 -- 172.21.15.5:0/2527861233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face810a3e0 msgr2=0x7face810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.755+0000 7faceed9f700 1 --2- 172.21.15.5:0/2527861233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face810a3e0 0x7face810a7f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7facdc009990 tx=0x7facdc009ca0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 -- 172.21.15.5:0/2527861233 shutdown_connections 2024-05-14T15:32:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 --2- 172.21.15.5:0/2527861233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7face8103130 0x7face8111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 --2- 172.21.15.5:0/2527861233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face810a3e0 0x7face810a7f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 --2- 172.21.15.5:0/2527861233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face81027e0 0x7face8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 -- 172.21.15.5:0/2527861233 >> 172.21.15.5:0/2527861233 conn(0x7face80fe6f0 msgr2=0x7face8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 -- 172.21.15.5:0/2527861233 shutdown_connections 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.756+0000 7faceed9f700 1 -- 172.21.15.5:0/2527861233 wait complete. 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.757+0000 7faceed9f700 1 Processor -- start 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.757+0000 7faceed9f700 1 -- start start 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.757+0000 7faceed9f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face81027e0 0x7face8105560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.757+0000 7faceed9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 0x7face81039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.757+0000 7faceed9f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7face810a3e0 0x7face8103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7faceed9f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7face8114480 con 0x7face81027e0 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7facecb3b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face81027e0 0x7face8105560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 0x7face81039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7faceed9f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7face8114300 con 0x7face8103130 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7faceed9f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7face8114600 con 0x7face810a3e0 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 0x7face81039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37824/0 (socket says 172.21.15.5:37824) 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 -- 172.21.15.5:0/292190295 learned_addr learned my addr 172.21.15.5:0/292190295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7facecb3b700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face81027e0 0x7face8105560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58168/0 (socket says 172.21.15.5:58168) 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7faced33c700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7face810a3e0 0x7face8103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 -- 172.21.15.5:0/292190295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7face810a3e0 msgr2=0x7face8103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7face810a3e0 0x7face8103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 -- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face81027e0 msgr2=0x7face8105560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face81027e0 0x7face8105560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 -- 172.21.15.5:0/292190295 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face81047d0 con 0x7face8103130 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.758+0000 7face7fff700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 0x7face81039c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7facd400ea30 tx=0x7facd400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.759+0000 7face5ffb700 1 -- 172.21.15.5:0/292190295 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd400cbd0 con 0x7face8103130 2024-05-14T15:32:30.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.759+0000 7face5ffb700 1 -- 172.21.15.5:0/292190295 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd40044d0 con 0x7face8103130 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.759+0000 7face5ffb700 1 -- 172.21.15.5:0/292190295 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4003dc0 con 0x7face8103130 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.759+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81ad8e0 con 0x7face8103130 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.760+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81add40 con 0x7face8103130 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.763+0000 7face5ffb700 1 -- 172.21.15.5:0/292190295 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7facd4003f80 con 0x7face8103130 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.764+0000 7face5ffb700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7facd80b2af0 0x7facd80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.764+0000 7face5ffb700 1 -- 172.21.15.5:0/292190295 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7facd40e2ae0 con 0x7face8103130 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.764+0000 7facecb3b700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7facd80b2af0 0x7facd80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.764+0000 7facfd7b3b80 1 Processor -- start 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.764+0000 7facfd7b3b80 1 -- start start 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7facd80b2af0 0x7facd80b5000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facdc006010 tx=0x7facdc00b260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:30.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560cea969c0 con 0x5560ceb74e90 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560ce9fbe20 con 0x5560ceb7ae00 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560cea92980 con 0x5560ce9f6ec0 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7faced33c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7face7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45932/0 (socket says 172.21.15.5:45932) 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 -- 172.21.15.5:0/509570458 learned_addr learned my addr 172.21.15.5:0/509570458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 -- 172.21.15.5:0/509570458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 msgr2=0x5560ceb74950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb74950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 -- 172.21.15.5:0/509570458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 msgr2=0x5560ceb752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 -- 172.21.15.5:0/509570458 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ce9e3950 con 0x5560ce9f6ec0 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.765+0000 7facecb3b700 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7a8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facdc0057e0 tx=0x7facdc01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5f59493442d6aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.766+0000 7facd2ffd700 1 -- 172.21.15.5:0/509570458 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc003880 con 0x5560ce9f6ec0 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.766+0000 7facd2ffd700 1 -- 172.21.15.5:0/509570458 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facdc02f480 con 0x5560ce9f6ec0 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.766+0000 7facd2ffd700 1 -- 172.21.15.5:0/509570458 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc00f6c0 con 0x5560ce9f6ec0 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 -- 172.21.15.5:0/509570458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 msgr2=0x5560ceb7a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7a8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facdc0057e0 tx=0x7facdc01e000 comp rx=0 tx=0).stop 2024-05-14T15:32:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 -- 172.21.15.5:0/509570458 shutdown_connections 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7a8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/509570458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 -- 172.21.15.5:0/509570458 >> 172.21.15.5:0/509570458 conn(0x5560cea3e6b0 msgr2=0x5560ceb6fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 -- 172.21.15.5:0/509570458 shutdown_connections 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 -- 172.21.15.5:0/509570458 wait complete. 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 -- start start 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.767+0000 7facfd7b3b80 1 Processor -- start 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560ceb81840 con 0x5560ceb74e90 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560cea4ff50 con 0x5560ceb7ae00 2024-05-14T15:32:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560ceb819c0 con 0x5560ce9f6ec0 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facecb3b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7faced33c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58186/0 (socket says 172.21.15.5:58186) 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 -- 172.21.15.5:0/735631968 learned_addr learned my addr 172.21.15.5:0/735631968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facfd7b3b80 10 client.0 _ll_register_callbacks cb 0x5560cea2e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 -- 172.21.15.5:0/735631968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 msgr2=0x5560ceb7e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 -- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 msgr2=0x5560ceb7efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb7efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 -- 172.21.15.5:0/735631968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cea4d7e0 con 0x5560ceb74e90 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7faced33c700 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7face7fff700 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ea60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7facd4009fd0 tx=0x7facd400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c34018363d136c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.768+0000 7facc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facc7fff700 1 -- 172.21.15.5:0/735631968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40c3060 con 0x5560ceb74e90 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facc7fff700 1 -- 172.21.15.5:0/735631968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7facd40a9bb0 con 0x5560ceb74e90 2024-05-14T15:32:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facdc02f480 con 0x5560ceb74e90 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facc7fff700 1 -- 172.21.15.5:0/735631968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7facd40a9460 con 0x5560ceb74e90 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facfd7b3b80 10 client.17262 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560ceb8c400 con 0x5560ceb74e90 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.769+0000 7facc7fff700 1 -- 172.21.15.5:0/735631968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facdc02f480 con 0x5560ceb74e90 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facc7fff700 1 -- 172.21.15.5:0/735631968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7facd40b2320 con 0x5560ceb74e90 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facfd7b3b80 10 client.17262 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facfd7b3b80 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facfd7b3b80 20 client.17262 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560ceb90e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facfd7b3b80 4 client.17262 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560ceaa43a0 con 0x5560ceb8ea40 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facecb3b700 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560ceb90e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facc7fff700 10 client.17262 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.770+0000 7facecb3b700 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560ceb90e60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5c83b2b6267db0 server_cookie=ba77e4e940ce7475 in_seq=0 out_seq=0 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.771+0000 7facc7fff700 1 -- 172.21.15.5:0/735631968 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560ceaa43a0 con 0x5560ceb8ea40 2024-05-14T15:32:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.771+0000 7facc7fff700 10 client.17262 handle_command_reply: tid=0 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.771+0000 7facfd7b3b80 1 client.17262 shutdown 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.771+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 msgr2=0x5560ceb7ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.771+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ea60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7facd4009fd0 tx=0x7facd400ce80 comp rx=0 tx=0).stop 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 shutdown_connections 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560ceb90e60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb7e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facc7fff700 0 client.17262 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/735631968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ea60 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 >> 172.21.15.5:0/735631968 conn(0x5560cea3e6b0 msgr2=0x5560ceb73940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 shutdown_connections 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 1 -- 172.21.15.5:0/735631968 wait complete. 2024-05-14T15:32:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.772+0000 7facfd7b3b80 20 client.17262 trim_cache size 0 max 16384 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 Processor -- start 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 -- start start 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7face81b9620 con 0x5560ceb74e90 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7face81b6eb0 con 0x5560ceb7ae00 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7face81b4d20 con 0x5560ce9f6ec0 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7face7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.794+0000 7face7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58190/0 (socket says 172.21.15.5:58190) 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 -- 172.21.15.5:0/1377724905 learned_addr learned my addr 172.21.15.5:0/1377724905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7faced33c700 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7facecb3b700 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 -- 172.21.15.5:0/1377724905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 msgr2=0x5560ceb74950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb74950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 -- 172.21.15.5:0/1377724905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 msgr2=0x5560ceb802e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb802e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 -- 172.21.15.5:0/1377724905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ceb7a410 con 0x5560ceb74e90 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7face7fff700 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ede0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7facd40a9600 tx=0x7facd40a5940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4ccc087a9bb2a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7facd27fc700 1 -- 172.21.15.5:0/1377724905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40c3070 con 0x5560ceb74e90 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7facd27fc700 1 -- 172.21.15.5:0/1377724905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd40aa050 con 0x5560ceb74e90 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.795+0000 7facd27fc700 1 -- 172.21.15.5:0/1377724905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40a9950 con 0x5560ceb74e90 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 -- 172.21.15.5:0/1377724905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 msgr2=0x5560ceb7ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ede0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7facd40a9600 tx=0x7facd40a5940 comp rx=0 tx=0).stop 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 -- 172.21.15.5:0/1377724905 shutdown_connections 2024-05-14T15:32:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ceb7ae00 0x5560ceb802e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ce9f6ec0 0x5560ceb74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/1377724905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb7ede0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 -- 172.21.15.5:0/1377724905 >> 172.21.15.5:0/1377724905 conn(0x5560cea3e6b0 msgr2=0x5560ceb7fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 -- 172.21.15.5:0/1377724905 shutdown_connections 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.796+0000 7facfd7b3b80 1 -- 172.21.15.5:0/1377724905 wait complete. 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.797+0000 7facfd7b3b80 1 -- start start 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.797+0000 7facfd7b3b80 1 Processor -- start 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ce9f6ec0 0x5560ceb96080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ceb7ae00 0x5560ceb96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7face81b5370 con 0x5560ceb74e90 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7face81b67e0 con 0x5560ce9f6ec0 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7face81b5a40 con 0x5560ceb7ae00 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 10 client.0 _ll_register_callbacks cb 0x5560cea2e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58194/0 (socket says 172.21.15.5:58194) 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facecb3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ce9f6ec0 0x5560ceb96080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 -- 172.21.15.5:0/4210399256 learned_addr learned my addr 172.21.15.5:0/4210399256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7faced33c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ceb7ae00 0x5560ceb96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 -- 172.21.15.5:0/4210399256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ceb7ae00 msgr2=0x5560ceb96b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ceb7ae00 0x5560ceb96b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 -- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ce9f6ec0 msgr2=0x5560ceb96080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ce9f6ec0 0x5560ceb96080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 -- 172.21.15.5:0/4210399256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cebf83a0 con 0x5560ceb74e90 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7face7fff700 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb965c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7facd40a59d0 tx=0x7facd40a51b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2761a22a586cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facc7fff700 1 -- 172.21.15.5:0/4210399256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40c3030 con 0x5560ceb74e90 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd40aa050 con 0x5560ceb74e90 2024-05-14T15:32:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.798+0000 7facc7fff700 1 -- 172.21.15.5:0/4210399256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7facd40a8dc0 con 0x5560ceb74e90 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facc7fff700 1 -- 172.21.15.5:0/4210399256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7facd40a9720 con 0x5560ceb74e90 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 10 client.17268 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560cebf9620 con 0x5560ceb74e90 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facc7fff700 1 -- 172.21.15.5:0/4210399256 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd40aa050 con 0x5560ceb74e90 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facc7fff700 1 -- 172.21.15.5:0/4210399256 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7facd40c7020 con 0x5560ceb74e90 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 10 client.17268 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 20 client.17268 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560cebfb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 4 client.17268 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560cea9e440 con 0x5560ceb8ea40 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.799+0000 7facecb3b700 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560cebfb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.800+0000 7facc7fff700 10 client.17268 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.800+0000 7facecb3b700 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560cebfb9f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af98a95860cbfe64 server_cookie=201c9e3a6d3e0f00 in_seq=0 out_seq=0 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facc7fff700 1 -- 172.21.15.5:0/4210399256 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560cea9e440 con 0x5560ceb8ea40 2024-05-14T15:32:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facc7fff700 10 client.17268 handle_command_reply: tid=0 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 client.17268 shutdown 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 msgr2=0x5560ceb965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb965c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7facd40a59d0 tx=0x7facd40a51b0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "b21ed6e1-d4aa-4c8c-b2c5-158b24107b86", 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:32:30.916 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 shutdown_connections 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5560ceb8ea40 0x5560cebfb9f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560ceb7ae00 0x5560ceb96b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560ce9f6ec0 0x5560ceb96080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facc7fff700 0 client.17268 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 --2- 172.21.15.5:0/4210399256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ceb74e90 0x5560ceb965c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 >> 172.21.15.5:0/4210399256 conn(0x5560cea3e6b0 msgr2=0x5560ceb69d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 shutdown_connections 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 1 -- 172.21.15.5:0/4210399256 wait complete. 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.801+0000 7facfd7b3b80 20 client.17268 trim_cache size 0 max 16384 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7facd80b2af0 msgr2=0x7facd80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7facd80b2af0 0x7facd80b5000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facdc006010 tx=0x7facdc00b260 comp rx=0 tx=0).stop 2024-05-14T15:32:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 msgr2=0x7face81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 0x7face81039c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7facd400ea30 tx=0x7facd400ed40 comp rx=0 tx=0).stop 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 shutdown_connections 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7facd80b2af0 0x7facd80b5000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7face810a3e0 0x7face8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7face81027e0 0x7face8105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 --2- 172.21.15.5:0/292190295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7face8103130 0x7face81039c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 >> 172.21.15.5:0/292190295 conn(0x7face80fe6f0 msgr2=0x7face8100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 shutdown_connections 2024-05-14T15:32:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:30.802+0000 7faceed9f700 1 -- 172.21.15.5:0/292190295 wait complete. 2024-05-14T15:32:31.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 715330 10.30 MB/sec execute 232 sec latency 39.207 ms 2024-05-14T15:32:31.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:32:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:31 smithi005 ceph-mon[31559]: pgmap v238: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-05-14T15:32:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:31 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:32:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:31 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:32:32.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:31 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:32:32.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:31 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:32:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:32:32.100 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:31 smithi144 ceph-mon[35650]: pgmap v238: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-05-14T15:32:32.100 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:31 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:32:32.100 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:31 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:32:32.101 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:31 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:32:32.101 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:31 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:32:32.105 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:31 smithi193 ceph-mon[36001]: pgmap v238: 129 pgs: 129 active+clean; 85 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-05-14T15:32:32.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:31 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:32:32.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:31 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:32:32.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:31 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:32:32.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:31 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:32:32.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 717590 10.32 MB/sec execute 233 sec latency 32.559 ms 2024-05-14T15:32:33.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 719174 10.30 MB/sec execute 234 sec latency 21.501 ms 2024-05-14T15:32:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 -- 172.21.15.5:0/1041428093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7041034c0 msgr2=0x7ff7041038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 --2- 172.21.15.5:0/1041428093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7041034c0 0x7ff7041038d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40099c0 tx=0x7ff6f4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 -- 172.21.15.5:0/1041428093 shutdown_connections 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 --2- 172.21.15.5:0/1041428093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704108740 0x7ff7041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 --2- 172.21.15.5:0/1041428093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7041034c0 0x7ff7041038d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 --2- 172.21.15.5:0/1041428093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff704107df0 0x7ff704108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 -- 172.21.15.5:0/1041428093 >> 172.21.15.5:0/1041428093 conn(0x7ff7040fedc0 msgr2=0x7ff7041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 -- 172.21.15.5:0/1041428093 shutdown_connections 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.916+0000 7ff709bd1700 1 -- 172.21.15.5:0/1041428093 wait complete. 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 Processor -- start 2024-05-14T15:32:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 -- start start 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 0x7ff70419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 0x7ff70419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff704108740 0x7ff70419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff704113370 con 0x7ff704108740 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041131f0 con 0x7ff7041034c0 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff709bd1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041134f0 con 0x7ff704107df0 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff702ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 0x7ff70419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 0x7ff70419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff703fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff704108740 0x7ff70419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff702ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 0x7ff70419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45968/0 (socket says 172.21.15.5:45968) 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff703fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff704108740 0x7ff70419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58208/0 (socket says 172.21.15.5:58208) 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 0x7ff70419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:37870/0 (socket says 172.21.15.5:37870) 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.917+0000 7ff703fff700 1 -- 172.21.15.5:0/2669350918 learned_addr learned my addr 172.21.15.5:0/2669350918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff7037fe700 1 -- 172.21.15.5:0/2669350918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 msgr2=0x7ff70419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff7037fe700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 0x7ff70419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff7037fe700 1 -- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff704108740 msgr2=0x7ff70419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff7037fe700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff704108740 0x7ff70419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff7037fe700 1 -- 172.21.15.5:0/2669350918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70419fce0 con 0x7ff7041034c0 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff702ffd700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 0x7ff70419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff7037fe700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 0x7ff70419ae10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40098e0 tx=0x7ff6f40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff700ff9700 1 -- 172.21.15.5:0/2669350918 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4026070 con 0x7ff7041034c0 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70419ff70 con 0x7ff7041034c0 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff700ff9700 1 -- 172.21.15.5:0/2669350918 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f400f460 con 0x7ff7041034c0 2024-05-14T15:32:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff700ff9700 1 -- 172.21.15.5:0/2669350918 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4009e60 con 0x7ff7041034c0 2024-05-14T15:32:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.918+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041a78f0 con 0x7ff7041034c0 2024-05-14T15:32:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.922+0000 7ff700ff9700 1 -- 172.21.15.5:0/2669350918 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff6f4029070 con 0x7ff7041034c0 2024-05-14T15:32:33.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:33 smithi005 ceph-mon[31559]: pgmap v239: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.922+0000 7ff7185e5b80 1 Processor -- start 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.922+0000 7ff7185e5b80 1 -- start start 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff700ff9700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff6f00b29f0 0x7ff6f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff703fff700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff6f00b29f0 0x7ff6f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af99634840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff700ff9700 1 -- 172.21.15.5:0/2669350918 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff6f40e33a0 con 0x7ff7041034c0 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af9962e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af9962f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55af99550940 con 0x55af9962ee10 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55af994b5da0 con 0x55af99634d80 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af9954c900 con 0x55af994b0e40 2024-05-14T15:32:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af99634840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af99634840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:45980/0 (socket says 172.21.15.5:45980) 2024-05-14T15:32:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 -- 172.21.15.5:0/3445186539 learned_addr learned my addr 172.21.15.5:0/3445186539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 -- 172.21.15.5:0/3445186539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 msgr2=0x55af9962e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af9962e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 -- 172.21.15.5:0/3445186539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 msgr2=0x55af9962f220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af9962f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.923+0000 7ff7037fe700 1 -- 172.21.15.5:0/3445186539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9949d8d0 con 0x55af994b0e40 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.924+0000 7ff703fff700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff6f00b29f0 0x7ff6f00b4f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009550 tx=0x7ff6fc009040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.924+0000 7ff7037fe700 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af99634840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40a6320 tx=0x7ff6f40a9a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9110eff39b7c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.924+0000 7ff6f9ffb700 1 -- 172.21.15.5:0/3445186539 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40e3160 con 0x55af994b0e40 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.924+0000 7ff6f9ffb700 1 -- 172.21.15.5:0/3445186539 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f40aa2a0 con 0x55af994b0e40 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.924+0000 7ff6f9ffb700 1 -- 172.21.15.5:0/3445186539 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40aa760 con 0x55af994b0e40 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 -- 172.21.15.5:0/3445186539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 msgr2=0x55af99634840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af99634840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40a6320 tx=0x7ff6f40a9a10 comp rx=0 tx=0).stop 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 -- 172.21.15.5:0/3445186539 shutdown_connections 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af9962f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af99634840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/3445186539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af9962e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 -- 172.21.15.5:0/3445186539 >> 172.21.15.5:0/3445186539 conn(0x55af994f8630 msgr2=0x55af99629a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:33.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 -- 172.21.15.5:0/3445186539 shutdown_connections 2024-05-14T15:32:33.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.925+0000 7ff7185e5b80 1 -- 172.21.15.5:0/3445186539 wait complete. 2024-05-14T15:32:33.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 -- start start 2024-05-14T15:32:33.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 Processor -- start 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af9962ee10 0x55af99638ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99639020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55af9963b5e0 con 0x55af994b0e40 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55af99509ed0 con 0x55af99634d80 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af9963b760 con 0x55af9962ee10 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7185e5b80 10 client.0 _ll_register_callbacks cb 0x55af994e8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58230/0 (socket says 172.21.15.5:58230) 2024-05-14T15:32:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff7037fe700 1 -- 172.21.15.5:0/1146432325 learned_addr learned my addr 172.21.15.5:0/1146432325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff702ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af9962ee10 0x55af99638ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.926+0000 7ff703fff700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99639020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff702ffd700 1 -- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 msgr2=0x55af99639020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99639020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff702ffd700 1 -- 172.21.15.5:0/1146432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 msgr2=0x55af996385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff702ffd700 1 -- 172.21.15.5:0/1146432325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af99507760 con 0x55af9962ee10 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff703fff700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99639020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af9962ee10 0x55af99638ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55af9962b1f0 tx=0x7ff6ec011cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80ffb0ea21cbab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff6e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1146432325 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec01e060 con 0x55af9962ee10 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f40aa2a0 con 0x55af9962ee10 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1146432325 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff6ec00f850 con 0x55af9962ee10 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.927+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1146432325 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6ec025050 con 0x55af9962ee10 2024-05-14T15:32:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 10 client.26875 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af99646260 con 0x55af9962ee10 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1146432325 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f40aa2a0 con 0x55af9962ee10 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1146432325 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff6ec018c00 con 0x55af9962ee10 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 10 client.26875 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 10 client.26875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 20 client.26875 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996488a0 0x55af9964acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 4 client.26875 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9955e320 con 0x55af996488a0 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.928+0000 7ff7037fe700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996488a0 0x55af9964acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.929+0000 7ff7037fe700 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996488a0 0x55af9964acc0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9881182524ae9351 server_cookie=c7024ef03512fdff in_seq=0 out_seq=0 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.929+0000 7ff6e2ffd700 10 client.26875 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1146432325 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af9955e320 con 0x55af996488a0 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff6e2ffd700 10 client.26875 handle_command_reply: tid=0 2024-05-14T15:32:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 client.26875 shutdown 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af9962ee10 msgr2=0x55af99638ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af9962ee10 0x55af99638ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55af9962b1f0 tx=0x7ff6ec011cb0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 shutdown_connections 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996488a0 0x55af9964acc0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99639020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1146432325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af9962ee10 0x55af99638ae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff6e2ffd700 0 client.26875 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 >> 172.21.15.5:0/1146432325 conn(0x55af994f8630 msgr2=0x55af9962d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 shutdown_connections 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1146432325 wait complete. 2024-05-14T15:32:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.930+0000 7ff7185e5b80 20 client.26875 trim_cache size 0 max 16384 2024-05-14T15:32:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 Processor -- start 2024-05-14T15:32:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 -- start start 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af996361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af99637500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99638e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b2fc0 con 0x55af9962ee10 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b0850 con 0x55af99634d80 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041ae6c0 con 0x55af994b0e40 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff702ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af99637500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff702ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af99637500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58232/0 (socket says 172.21.15.5:58232) 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff702ffd700 1 -- 172.21.15.5:0/1707712081 learned_addr learned my addr 172.21.15.5:0/1707712081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff7037fe700 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af996361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.952+0000 7ff703fff700 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99638e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff702ffd700 1 -- 172.21.15.5:0/1707712081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 msgr2=0x55af996361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af996361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff702ffd700 1 -- 172.21.15.5:0/1707712081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 msgr2=0x55af99638e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99638e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff702ffd700 1 -- 172.21.15.5:0/1707712081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af996343c0 con 0x55af9962ee10 2024-05-14T15:32:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af99637500 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec014040 tx=0x7ff6ec00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a2dab4ef013ed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff6f97fa700 1 -- 172.21.15.5:0/1707712081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec02e070 con 0x55af9962ee10 2024-05-14T15:32:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff6f97fa700 1 -- 172.21.15.5:0/1707712081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ec0044f0 con 0x55af9962ee10 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.953+0000 7ff6f97fa700 1 -- 172.21.15.5:0/1707712081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec00fc30 con 0x55af9962ee10 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1707712081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 msgr2=0x55af99637500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af99637500 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec014040 tx=0x7ff6ec00f3c0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1707712081 shutdown_connections 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af99634d80 0x55af99638e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af994b0e40 0x55af996361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1707712081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af9962ee10 0x55af99637500 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1707712081 >> 172.21.15.5:0/1707712081 conn(0x55af994f8630 msgr2=0x55af99639c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1707712081 shutdown_connections 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.954+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1707712081 wait complete. 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 -- start start 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 Processor -- start 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af9962ee10 0x55af99640ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af99634d80 0x55af9964f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041aed10 con 0x55af994b0e40 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b0180 con 0x55af9962ee10 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041af3e0 con 0x55af99634d80 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7185e5b80 10 client.0 _ll_register_callbacks cb 0x55af994e8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58234/0 (socket says 172.21.15.5:58234) 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff702ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af9962ee10 0x55af99640ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff7037fe700 1 -- 172.21.15.5:0/1340679330 learned_addr learned my addr 172.21.15.5:0/1340679330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.955+0000 7ff703fff700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af99634d80 0x55af9964f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7037fe700 1 -- 172.21.15.5:0/1340679330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af99634d80 msgr2=0x55af9964f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7037fe700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af99634d80 0x55af9964f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7037fe700 1 -- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af9962ee10 msgr2=0x55af99640ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7037fe700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af9962ee10 0x55af99640ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7037fe700 1 -- 172.21.15.5:0/1340679330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9964fae0 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff703fff700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af99634d80 0x55af9964f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7037fe700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996409b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40a62d0 tx=0x7ff6f40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bc53550f176147 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff6e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1340679330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40c3020 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1340679330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff6f40aa500 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ec0044f0 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1340679330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6f40aa2a0 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7185e5b80 10 client.17283 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.956+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af996b28a0 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1340679330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ec0044f0 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1340679330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff6f40b2be0 con 0x55af994b0e40 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff7185e5b80 10 client.17283 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff7185e5b80 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff7185e5b80 20 client.17283 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996b2d60 0x55af996b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff7185e5b80 4 client.17283 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af995583c0 con 0x55af996b2d60 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.957+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996b2d60 0x55af996b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.958+0000 7ff6e2ffd700 10 client.17283 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.958+0000 7ff702ffd700 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996b2d60 0x55af996b5180 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc01f1955746e95 server_cookie=2583d6ee4f189493 in_seq=0 out_seq=0 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff6e2ffd700 1 -- 172.21.15.5:0/1340679330 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55af995583c0 con 0x55af996b2d60 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff6e2ffd700 10 client.17283 handle_command_reply: tid=0 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 client.17283 shutdown 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 msgr2=0x55af996409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996409b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40a62d0 tx=0x7ff6f40092e0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 shutdown_connections 2024-05-14T15:32:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55af996b2d60 0x55af996b5180 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af99634d80 0x55af9964f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af994b0e40 0x55af996409b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 --2- 172.21.15.5:0/1340679330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af9962ee10 0x55af99640ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff6e2ffd700 0 client.17283 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 >> 172.21.15.5:0/1340679330 conn(0x55af994f8630 msgr2=0x55af99629a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 shutdown_connections 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 1 -- 172.21.15.5:0/1340679330 wait complete. 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.959+0000 7ff7185e5b80 20 client.17283 trim_cache size 0 max 16384 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:32:33.965 INFO:teuthology.orchestra.run.smithi005.stdout: "b21ed6e1-d4aa-4c8c-b2c5-158b24107b86": { 2024-05-14T15:32:33.966 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:32:33.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "b21ed6e1-d4aa-4c8c-b2c5-158b24107b86", 2024-05-14T15:32:33.966 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:32:33.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:32:33.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:32:33.966 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff6f00b29f0 msgr2=0x7ff6f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff6f00b29f0 0x7ff6f00b4f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009550 tx=0x7ff6fc009040 comp rx=0 tx=0).stop 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 msgr2=0x7ff70419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 0x7ff70419ae10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40098e0 tx=0x7ff6f40045e0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 shutdown_connections 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff6f00b29f0 0x7ff6f00b4f00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff704108740 0x7ff70419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7041034c0 0x7ff70419ae10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 --2- 172.21.15.5:0/2669350918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff704107df0 0x7ff70419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 >> 172.21.15.5:0/2669350918 conn(0x7ff7040fedc0 msgr2=0x7ff7041096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 shutdown_connections 2024-05-14T15:32:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:32:33.960+0000 7ff709bd1700 1 -- 172.21.15.5:0/2669350918 wait complete. 2024-05-14T15:32:34.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 721333 10.31 MB/sec execute 235 sec latency 18.644 ms 2024-05-14T15:32:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:33 smithi144 ceph-mon[35650]: pgmap v239: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-05-14T15:32:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:33 smithi193 ceph-mon[36001]: pgmap v239: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-05-14T15:32:34.772 INFO:tasks.cephfs.filesystem:scrub status for tag:b21ed6e1-d4aa-4c8c-b2c5-158b24107b86 - {'path': '/', 'tag': 'b21ed6e1-d4aa-4c8c-b2c5-158b24107b86', 'options': 'recursive,force'} 2024-05-14T15:32:35.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 723057 10.29 MB/sec execute 236 sec latency 55.034 ms 2024-05-14T15:32:36.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:35 smithi005 ceph-mon[31559]: pgmap v240: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-05-14T15:32:36.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:35 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:32:36.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 724911 10.31 MB/sec execute 237 sec latency 19.057 ms 2024-05-14T15:32:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:35 smithi144 ceph-mon[35650]: pgmap v240: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-05-14T15:32:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:35 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:32:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:35 smithi193 ceph-mon[36001]: pgmap v240: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 332 op/s 2024-05-14T15:32:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:35 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:32:37.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 726963 10.30 MB/sec execute 238 sec latency 15.015 ms 2024-05-14T15:32:38.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 728870 10.30 MB/sec execute 239 sec latency 13.210 ms 2024-05-14T15:32:38.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:37 smithi144 ceph-mon[35650]: pgmap v241: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 381 op/s 2024-05-14T15:32:38.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:37 smithi193 ceph-mon[36001]: pgmap v241: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 381 op/s 2024-05-14T15:32:38.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:37 smithi005 ceph-mon[31559]: pgmap v241: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 381 op/s 2024-05-14T15:32:39.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 731466 10.33 MB/sec execute 240 sec latency 19.792 ms 2024-05-14T15:32:40.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 732746 10.30 MB/sec execute 241 sec latency 48.805 ms 2024-05-14T15:32:40.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:39 smithi144 ceph-mon[35650]: pgmap v242: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-05-14T15:32:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:39 smithi193 ceph-mon[36001]: pgmap v242: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-05-14T15:32:40.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:39 smithi005 ceph-mon[31559]: pgmap v242: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 -- 172.21.15.5:0/2401277398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 msgr2=0x7fbd7c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 --2- 172.21.15.5:0/2401277398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c102c80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009900 tx=0x7fbd6802a410 comp rx=0 tx=0).stop 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 -- 172.21.15.5:0/2401277398 shutdown_connections 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 --2- 172.21.15.5:0/2401277398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c10f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 --2- 172.21.15.5:0/2401277398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c102c80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 --2- 172.21.15.5:0/2401277398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c108ba0 0x7fbd7c102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 -- 172.21.15.5:0/2401277398 >> 172.21.15.5:0/2401277398 conn(0x7fbd7c0782e0 msgr2=0x7fbd7c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 -- 172.21.15.5:0/2401277398 shutdown_connections 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.121+0000 7fbd82f38640 1 -- 172.21.15.5:0/2401277398 wait complete. 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 Processor -- start 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 -- start start 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd7c108ba0 0x7fbd7c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c112d90 con 0x7fbd7c109040 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c112c10 con 0x7fbd7c1032f0 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.122+0000 7fbd82f38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c112f10 con 0x7fbd7c108ba0 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:38014/0 (socket says 172.21.15.5:38014) 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c19c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56106/0 (socket says 172.21.15.5:56106) 2024-05-14T15:32:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 -- 172.21.15.5:0/3712194080 learned_addr learned my addr 172.21.15.5:0/3712194080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd7c108ba0 0x7fbd7c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 -- 172.21.15.5:0/3712194080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd7c108ba0 msgr2=0x7fbd7c19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd7c108ba0 0x7fbd7c19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 -- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 msgr2=0x7fbd7c19c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c19c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 -- 172.21.15.5:0/3712194080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c1a6a60 con 0x7fbd7c1032f0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd80cd4640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c19b750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000e9c0 tx=0x7fbd7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd79ffb640 1 -- 172.21.15.5:0/3712194080 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000cd80 con 0x7fbd7c1032f0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c1a6d50 con 0x7fbd7c1032f0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd79ffb640 1 -- 172.21.15.5:0/3712194080 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd700054d0 con 0x7fbd7c1032f0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.123+0000 7fbd79ffb640 1 -- 172.21.15.5:0/3712194080 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70010430 con 0x7fbd7c1032f0 2024-05-14T15:32:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.124+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c1a7200 con 0x7fbd7c1032f0 2024-05-14T15:32:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 Processor -- start 2024-05-14T15:32:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 -- start start 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd79ffb640 1 -- 172.21.15.5:0/3712194080 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbd70004fd0 con 0x7fbd7c1032f0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 0x562cea2cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f052f0 0x562cea2cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cea2345d0 con 0x562ce9f2e1b0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562cea1ed890 con 0x562ce9f052f0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.126+0000 7fbd89ded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562cea230590 con 0x562ce9f0d4d0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 0x562cea2cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 0x562cea2cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56116/0 (socket says 172.21.15.5:56116) 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd80cd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f052f0 0x562cea2cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 -- 172.21.15.5:0/271236120 learned_addr learned my addr 172.21.15.5:0/271236120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 -- 172.21.15.5:0/271236120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 msgr2=0x562cea2cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 -- 172.21.15.5:0/271236120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f052f0 msgr2=0x562cea2cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f052f0 0x562cea2cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 -- 172.21.15.5:0/271236120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce9f46210 con 0x562ce9f2e1b0 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd814d5640 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 0x562cea2cebe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009f90 tx=0x7fbd740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb212c14915e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd66ffd640 1 -- 172.21.15.5:0/271236120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74004d20 con 0x562ce9f2e1b0 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd66ffd640 1 -- 172.21.15.5:0/271236120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd74004050 con 0x562ce9f2e1b0 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd66ffd640 1 -- 172.21.15.5:0/271236120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74005240 con 0x562ce9f2e1b0 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.127+0000 7fbd79ffb640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd6c0b2950 0x7fbd6c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd80cd4640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd6c0b2950 0x7fbd6c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd79ffb640 1 -- 172.21.15.5:0/3712194080 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fbd700e2db0 con 0x7fbd7c1032f0 2024-05-14T15:32:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 -- 172.21.15.5:0/271236120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 msgr2=0x562cea2cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd80cd4640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd6c0b2950 0x7fbd6c0b4e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd700e01a0 tx=0x7fbd700e6900 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 0x562cea2cebe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009f90 tx=0x7fbd740118d0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 -- 172.21.15.5:0/271236120 shutdown_connections 2024-05-14T15:32:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f052f0 0x562cea2cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 --2- 172.21.15.5:0/271236120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f2e1b0 0x562cea2cebe0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.128+0000 7fbd89ded740 1 -- 172.21.15.5:0/271236120 >> 172.21.15.5:0/271236120 conn(0x562ce9f22300 msgr2=0x562cea2cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.129+0000 7fbd89ded740 1 -- 172.21.15.5:0/271236120 shutdown_connections 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.129+0000 7fbd89ded740 1 -- 172.21.15.5:0/271236120 wait complete. 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.129+0000 7fbd89ded740 1 -- start start 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.129+0000 7fbd89ded740 1 Processor -- start 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.129+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.129+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f2e1b0 0x562cea2e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd89ded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cea2dbd10 con 0x562ce9f052f0 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd89ded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562cea1f1370 con 0x562ce9f2e1b0 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd89ded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562cea2dbe90 con 0x562ce9f0d4d0 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd89ded740 10 client.0 _ll_register_callbacks cb 0x562ce9f45a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2d91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56126/0 (socket says 172.21.15.5:56126) 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 -- 172.21.15.5:0/569203837 learned_addr learned my addr 172.21.15.5:0/569203837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f2e1b0 0x562cea2e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 -- 172.21.15.5:0/569203837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 msgr2=0x562cea2d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 -- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f2e1b0 msgr2=0x562cea2e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f2e1b0 0x562cea2e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 -- 172.21.15.5:0/569203837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cea012620 con 0x562ce9f052f0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd814d5640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2d91b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbd7401b040 tx=0x7fbd74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d166d7e52837fe7a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd5bfff640 1 -- 172.21.15.5:0/569203837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7402c020 con 0x562ce9f052f0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd5bfff640 1 -- 172.21.15.5:0/569203837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fbd74005040 con 0x562ce9f052f0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.130+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd74004050 con 0x562ce9f052f0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd5bfff640 1 -- 172.21.15.5:0/569203837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd74037050 con 0x562ce9f052f0 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 10 client.17295 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562cea2e8040 con 0x562ce9f052f0 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd5bfff640 1 -- 172.21.15.5:0/569203837 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd74004050 con 0x562ce9f052f0 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd5bfff640 1 -- 172.21.15.5:0/569203837 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbd7400f570 con 0x562ce9f052f0 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 10 client.17295 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 20 client.17295 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9eecd80 0x562cea2eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 4 client.17295 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cea2421e0 con 0x562ce9eecd80 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.131+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9eecd80 0x562cea2eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.132+0000 7fbd5bfff640 10 client.17295 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.132+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9eecd80 0x562cea2eb510 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff770a03c28cc956 server_cookie=425b7aedfe5a9d76 in_seq=0 out_seq=0 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.132+0000 7fbd5bfff640 1 -- 172.21.15.5:0/569203837 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562cea2421e0 con 0x562ce9eecd80 2024-05-14T15:32:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.132+0000 7fbd5bfff640 10 client.17295 handle_command_reply: tid=0 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 client.17295 shutdown 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 msgr2=0x562cea2d91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2d91b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbd7401b040 tx=0x7fbd74009c00 comp rx=0 tx=0).stop 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 shutdown_connections 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9eecd80 0x562cea2eb510 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9f0d4d0 0x562cea2d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f2e1b0 0x562cea2e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd5bfff640 0 client.17295 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 --2- 172.21.15.5:0/569203837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2d91b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 >> 172.21.15.5:0/569203837 conn(0x562ce9f22300 msgr2=0x562cea2cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 shutdown_connections 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 1 -- 172.21.15.5:0/569203837 wait complete. 2024-05-14T15:32:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.133+0000 7fbd89ded740 20 client.17295 trim_cache size 0 max 16384 2024-05-14T15:32:41.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 735117 10.32 MB/sec execute 242 sec latency 40.372 ms 2024-05-14T15:32:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.149+0000 7fbd89ded740 1 Processor -- start 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 -- start start 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b5f80 con 0x562ce9f052f0 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b3850 con 0x562ce9f0d4d0 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd89ded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b16c0 con 0x562ce9eecd80 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd814d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2f0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56128/0 (socket says 172.21.15.5:56128) 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 -- 172.21.15.5:0/155820694 learned_addr learned my addr 172.21.15.5:0/155820694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd814d5640 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 -- 172.21.15.5:0/155820694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 msgr2=0x562cea2db190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2db190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 -- 172.21.15.5:0/155820694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 msgr2=0x562cea2f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.150+0000 7fbd7bfff640 1 -- 172.21.15.5:0/155820694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cea2c69a0 con 0x562ce9f052f0 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.151+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2f0b40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005940 tx=0x7fbd68031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b4a6d74f993714 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.151+0000 7fbd667fc640 1 -- 172.21.15.5:0/155820694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6804b070 con 0x562ce9f052f0 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.151+0000 7fbd667fc640 1 -- 172.21.15.5:0/155820694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd68004380 con 0x562ce9f052f0 2024-05-14T15:32:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.151+0000 7fbd667fc640 1 -- 172.21.15.5:0/155820694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68035040 con 0x562ce9f052f0 2024-05-14T15:32:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 -- 172.21.15.5:0/155820694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 msgr2=0x562cea2f0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2f0b40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005940 tx=0x7fbd68031400 comp rx=0 tx=0).stop 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 -- 172.21.15.5:0/155820694 shutdown_connections 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2f0b40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 --2- 172.21.15.5:0/155820694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2db190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 -- 172.21.15.5:0/155820694 >> 172.21.15.5:0/155820694 conn(0x562ce9f22300 msgr2=0x562cea2ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 -- 172.21.15.5:0/155820694 shutdown_connections 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 -- 172.21.15.5:0/155820694 wait complete. 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 -- start start 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.152+0000 7fbd89ded740 1 Processor -- start 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2dc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b1d10 con 0x562ce9f052f0 2024-05-14T15:32:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b3180 con 0x562ce9f0d4d0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c1b23e0 con 0x562ce9eecd80 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd89ded740 10 client.0 _ll_register_callbacks cb 0x7fbd700e67d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2dc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2dc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56134/0 (socket says 172.21.15.5:56134) 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 -- 172.21.15.5:0/3651710619 learned_addr learned my addr 172.21.15.5:0/3651710619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd814d5640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd814d5640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 -- 172.21.15.5:0/3651710619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 msgr2=0x562cea2e1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2e1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 -- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 msgr2=0x562cea2dcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2dcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 -- 172.21.15.5:0/3651710619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cea2dd140 con 0x562ce9f052f0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd7bfff640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2dc6c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbd68049040 tx=0x7fbd680045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2be40a44646bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:32:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.153+0000 7fbd5bfff640 1 -- 172.21.15.5:0/3651710619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6804b040 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd5bfff640 1 -- 172.21.15.5:0/3651710619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fbd680462e0 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd68004380 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd5bfff640 1 -- 172.21.15.5:0/3651710619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd6805c050 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 10 client.17301 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562cea3532c0 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd5bfff640 1 -- 172.21.15.5:0/3651710619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd68004380 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd5bfff640 1 -- 172.21.15.5:0/3651710619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbd68057030 con 0x562ce9f052f0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 10 client.17301 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 20 client.17301 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9f2e1b0 0x562cea3559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 4 client.17301 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cea23c050 con 0x562ce9f2e1b0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.154+0000 7fbd814d5640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9f2e1b0 0x562cea3559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.155+0000 7fbd5bfff640 10 client.17301 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.155+0000 7fbd814d5640 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9f2e1b0 0x562cea3559e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dcce58236cb247 server_cookie=42a05d54d21e82dd in_seq=0 out_seq=0 2024-05-14T15:32:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd5bfff640 1 -- 172.21.15.5:0/3651710619 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cea23c050 con 0x562ce9f2e1b0 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd5bfff640 10 client.17301 handle_command_reply: tid=0 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 client.17301 shutdown 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 msgr2=0x562cea2dc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2dc6c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbd68049040 tx=0x7fbd680045e0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 shutdown_connections 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x562ce9f2e1b0 0x562cea3559e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd5bfff640 0 client.17301 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:41.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ce9f0d4d0 0x562cea2dcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce9f052f0 0x562cea2dc6c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 --2- 172.21.15.5:0/3651710619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ce9eecd80 0x562cea2e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 >> 172.21.15.5:0/3651710619 conn(0x562ce9f22300 msgr2=0x562cea2ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 shutdown_connections 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.436+0000 7fbd89ded740 1 -- 172.21.15.5:0/3651710619 wait complete. 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.437+0000 7fbd89ded740 20 client.17301 trim_cache size 0 max 16384 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:32:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.437+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd6c0b2950 msgr2=0x7fbd6c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.437+0000 7fbd82f38640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd6c0b2950 0x7fbd6c0b4e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd700e01a0 tx=0x7fbd700e6900 comp rx=0 tx=0).stop 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.437+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 msgr2=0x7fbd7c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.437+0000 7fbd82f38640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c19b750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000e9c0 tx=0x7fbd7000ee80 comp rx=0 tx=0).stop 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 shutdown_connections 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd6c0b2950 0x7fbd6c0b4e60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd7c108ba0 0x7fbd7c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd7c109040 0x7fbd7c19c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 --2- 172.21.15.5:0/3712194080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd7c1032f0 0x7fbd7c19b750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 >> 172.21.15.5:0/3712194080 conn(0x7fbd7c0782e0 msgr2=0x7fbd7c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 shutdown_connections 2024-05-14T15:32:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:41.438+0000 7fbd82f38640 1 -- 172.21.15.5:0/3712194080 wait complete. 2024-05-14T15:32:42.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 736158 10.30 MB/sec execute 243 sec latency 96.447 ms 2024-05-14T15:32:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:41 smithi144 ceph-mon[35650]: pgmap v243: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 325 op/s 2024-05-14T15:32:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:41 smithi193 ceph-mon[36001]: pgmap v243: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 325 op/s 2024-05-14T15:32:42.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:41 smithi005 ceph-mon[31559]: pgmap v243: 129 pgs: 129 active+clean; 77 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 325 op/s 2024-05-14T15:32:43.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 738592 10.32 MB/sec execute 244 sec latency 29.706 ms 2024-05-14T15:32:44.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 740055 10.30 MB/sec execute 245 sec latency 20.565 ms 2024-05-14T15:32:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:43 smithi144 ceph-mon[35650]: pgmap v244: 129 pgs: 129 active+clean; 68 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 531 op/s 2024-05-14T15:32:44.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:43 smithi193 ceph-mon[36001]: pgmap v244: 129 pgs: 129 active+clean; 68 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 531 op/s 2024-05-14T15:32:44.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:43 smithi005 ceph-mon[31559]: pgmap v244: 129 pgs: 129 active+clean; 68 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 531 op/s 2024-05-14T15:32:45.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 742236 10.31 MB/sec execute 246 sec latency 26.242 ms 2024-05-14T15:32:46.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 743513 10.30 MB/sec execute 247 sec latency 58.124 ms 2024-05-14T15:32:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:45 smithi144 ceph-mon[35650]: pgmap v245: 129 pgs: 129 active+clean; 68 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T15:32:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:45 smithi193 ceph-mon[36001]: pgmap v245: 129 pgs: 129 active+clean; 68 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T15:32:46.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:45 smithi005 ceph-mon[31559]: pgmap v245: 129 pgs: 129 active+clean; 68 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T15:32:47.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 745806 10.31 MB/sec execute 248 sec latency 22.738 ms 2024-05-14T15:32:48.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 747344 10.29 MB/sec execute 249 sec latency 18.155 ms 2024-05-14T15:32:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:47 smithi144 ceph-mon[35650]: pgmap v246: 129 pgs: 129 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-05-14T15:32:48.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:47 smithi193 ceph-mon[36001]: pgmap v246: 129 pgs: 129 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-05-14T15:32:48.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:47 smithi005 ceph-mon[31559]: pgmap v246: 129 pgs: 129 active+clean; 74 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 363 op/s 2024-05-14T15:32:49.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 749607 10.31 MB/sec execute 250 sec latency 28.660 ms 2024-05-14T15:32:50.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 750914 10.29 MB/sec execute 251 sec latency 157.080 ms 2024-05-14T15:32:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:49 smithi144 ceph-mon[35650]: pgmap v247: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 346 op/s 2024-05-14T15:32:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:49 smithi193 ceph-mon[36001]: pgmap v247: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 346 op/s 2024-05-14T15:32:50.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:49 smithi005 ceph-mon[31559]: pgmap v247: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 346 op/s 2024-05-14T15:32:51.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 753012 10.30 MB/sec execute 252 sec latency 48.118 ms 2024-05-14T15:32:52.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 754630 10.28 MB/sec execute 253 sec latency 15.369 ms 2024-05-14T15:32:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:51 smithi144 ceph-mon[35650]: pgmap v248: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T15:32:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:51 smithi193 ceph-mon[36001]: pgmap v248: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T15:32:52.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:51 smithi005 ceph-mon[31559]: pgmap v248: 129 pgs: 129 active+clean; 75 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-05-14T15:32:53.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 756878 10.29 MB/sec execute 254 sec latency 13.867 ms 2024-05-14T15:32:54.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 758767 10.28 MB/sec execute 255 sec latency 16.719 ms 2024-05-14T15:32:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:53 smithi144 ceph-mon[35650]: pgmap v249: 129 pgs: 129 active+clean; 81 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 403 op/s 2024-05-14T15:32:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:53 smithi193 ceph-mon[36001]: pgmap v249: 129 pgs: 129 active+clean; 81 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 403 op/s 2024-05-14T15:32:54.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:53 smithi005 ceph-mon[31559]: pgmap v249: 129 pgs: 129 active+clean; 81 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 403 op/s 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 -- 172.21.15.5:0/3045212076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168109030 msgr2=0x7f0168102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 --2- 172.21.15.5:0/3045212076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168109030 0x7f0168102c70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f015c009900 tx=0x7f015c02a410 comp rx=0 tx=0).stop 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 -- 172.21.15.5:0/3045212076 shutdown_connections 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 --2- 172.21.15.5:0/3045212076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 0x7f016810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 --2- 172.21.15.5:0/3045212076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168109030 0x7f0168102c70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 --2- 172.21.15.5:0/3045212076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0168108b90 0x7f0168102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 -- 172.21.15.5:0/3045212076 >> 172.21.15.5:0/3045212076 conn(0x7f01680fe390 msgr2=0x7f01681007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 -- 172.21.15.5:0/3045212076 shutdown_connections 2024-05-14T15:32:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.542+0000 7f016d843640 1 -- 172.21.15.5:0/3045212076 wait complete. 2024-05-14T15:32:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 Processor -- start 2024-05-14T15:32:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 -- start start 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 0x7f01681a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 0x7f01681a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0168109030 0x7f016819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0168113ed0 con 0x7f0168108b90 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0168113d50 con 0x7f01681032e0 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f016d843640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0168114050 con 0x7f0168109030 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f01667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 0x7f01681a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f01667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 0x7f01681a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60858/0 (socket says 172.21.15.5:60858) 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f01667fc640 1 -- 172.21.15.5:0/1227007836 learned_addr learned my addr 172.21.15.5:0/1227007836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f0166ffd640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 0x7f01681a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.543+0000 7f01677fe640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0168109030 0x7f016819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f0166ffd640 1 -- 172.21.15.5:0/1227007836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0168109030 msgr2=0x7f016819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f0166ffd640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0168109030 0x7f016819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f0166ffd640 1 -- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 msgr2=0x7f01681a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f0166ffd640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 0x7f01681a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f0166ffd640 1 -- 172.21.15.5:0/1227007836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016819aeb0 con 0x7f01681032e0 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f01667fc640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 0x7f01681a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f0166ffd640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 0x7f01681a05a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0168103870 tx=0x7f015800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f016c841640 1 -- 172.21.15.5:0/1227007836 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158017070 con 0x7f01681032e0 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016819b1a0 con 0x7f01681032e0 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f016c841640 1 -- 172.21.15.5:0/1227007836 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01580099d0 con 0x7f01681032e0 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f016c841640 1 -- 172.21.15.5:0/1227007836 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580047a0 con 0x7f01681032e0 2024-05-14T15:32:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.544+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a73f0 con 0x7f01681032e0 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.546+0000 7f01747db740 1 Processor -- start 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 -- start start 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f016c841640 1 -- 172.21.15.5:0/1227007836 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0158004940 con 0x7f01681032e0 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b8858be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b848f2f0 0x5650b8859120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8859660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650b87be5d0 con 0x5650b848f2f0 2024-05-14T15:32:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650b8777890 con 0x5650b84974d0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01747db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650b87ba590 con 0x5650b84b81b0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f0166ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b848f2f0 0x5650b8859120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8859660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f0166ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b848f2f0 0x5650b8859120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60872/0 (socket says 172.21.15.5:60872) 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8859660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39444/0 (socket says 172.21.15.5:39444) 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f0166ffd640 1 -- 172.21.15.5:0/2015246261 learned_addr learned my addr 172.21.15.5:0/2015246261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01677fe640 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b8858be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 -- 172.21.15.5:0/2015246261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 msgr2=0x5650b8858be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b8858be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 -- 172.21.15.5:0/2015246261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b848f2f0 msgr2=0x5650b8859120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b848f2f0 0x5650b8859120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.547+0000 7f01667fc640 1 -- 172.21.15.5:0/2015246261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b84d0210 con 0x5650b84974d0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f01667fc640 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8859660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f015c005ab0 tx=0x7f015c0044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde8d5afcaaa48fd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f016c841640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f01500b2950 0x7f01500b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f014dffb640 1 -- 172.21.15.5:0/2015246261 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c040070 con 0x5650b84974d0 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f014dffb640 1 -- 172.21.15.5:0/2015246261 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c0047a0 con 0x5650b84974d0 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f014dffb640 1 -- 172.21.15.5:0/2015246261 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c035040 con 0x5650b84974d0 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f01677fe640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f01500b2950 0x7f01500b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.548+0000 7f016c841640 1 -- 172.21.15.5:0/1227007836 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f01580e25c0 con 0x7f01681032e0 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 -- 172.21.15.5:0/2015246261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 msgr2=0x5650b8859660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01677fe640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f01500b2950 0x7f01500b4e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5650b8855d00 tx=0x7f0160009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8859660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f015c005ab0 tx=0x7f015c0044a0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 -- 172.21.15.5:0/2015246261 shutdown_connections 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8859660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b848f2f0 0x5650b8859120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 --2- 172.21.15.5:0/2015246261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b8858be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 -- 172.21.15.5:0/2015246261 >> 172.21.15.5:0/2015246261 conn(0x5650b84ac300 msgr2=0x5650b8855340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 -- 172.21.15.5:0/2015246261 shutdown_connections 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 -- 172.21.15.5:0/2015246261 wait complete. 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.549+0000 7f01747db740 1 -- start start 2024-05-14T15:32:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 Processor -- start 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8863200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8863740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b886f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650b8865cb0 con 0x5650b84b81b0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650b877b370 con 0x5650b84974d0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650b8865e30 con 0x5650b848f2f0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01747db740 10 client.0 _ll_register_callbacks cb 0x5650b84cfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8863200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01667fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8863740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f01677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b886f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8863200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50546/0 (socket says 172.21.15.5:50546) 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 -- 172.21.15.5:0/3938550252 learned_addr learned my addr 172.21.15.5:0/3938550252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 -- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 msgr2=0x5650b8863740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8863740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 -- 172.21.15.5:0/3938550252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 msgr2=0x5650b886f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.550+0000 7f0166ffd640 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b886f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0166ffd640 1 -- 172.21.15.5:0/3938550252 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b859c620 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0166ffd640 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8863200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01580a9190 tx=0x7f01580057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807cd91987311d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0146ffd640 1 -- 172.21.15.5:0/3938550252 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580a5350 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c0047a0 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0146ffd640 1 -- 172.21.15.5:0/3938550252 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f01580ad5d0 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0146ffd640 1 -- 172.21.15.5:0/3938550252 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f01580a8c00 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f01747db740 10 client.26911 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650b8872080 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0146ffd640 1 -- 172.21.15.5:0/3938550252 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c0047a0 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.551+0000 7f0146ffd640 1 -- 172.21.15.5:0/3938550252 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f01580ad1b0 con 0x5650b848f2f0 2024-05-14T15:32:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01747db740 10 client.26911 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01747db740 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01747db740 20 client.26911 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01747db740 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b8476d80 0x5650b8875550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01747db740 4 client.26911 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b87cc1e0 con 0x5650b8476d80 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01667fc640 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b8476d80 0x5650b8875550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f0146ffd640 10 client.26911 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.552+0000 7f01667fc640 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b8476d80 0x5650b8875550 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c15ea0a751abda6 server_cookie=13d29d91018314e6 in_seq=0 out_seq=0 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f0146ffd640 1 -- 172.21.15.5:0/3938550252 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650b87cc1e0 con 0x5650b8476d80 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f0146ffd640 10 client.26911 handle_command_reply: tid=0 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 client.26911 shutdown 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 msgr2=0x5650b8863200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8863200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01580a9190 tx=0x7f01580057e0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 shutdown_connections 2024-05-14T15:32:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b8476d80 0x5650b8875550 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b886f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b8863740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 --2- 172.21.15.5:0/3938550252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8863200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f0146ffd640 0 client.26911 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 >> 172.21.15.5:0/3938550252 conn(0x5650b84ac300 msgr2=0x5650b885a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 shutdown_connections 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 1 -- 172.21.15.5:0/3938550252 wait complete. 2024-05-14T15:32:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.554+0000 7f01747db740 20 client.26911 trim_cache size 0 max 16384 2024-05-14T15:32:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 Processor -- start 2024-05-14T15:32:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 -- start start 2024-05-14T15:32:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b848f2f0 0x5650b8859b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 0x5650b8865140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b886ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b3da0 con 0x5650b84974d0 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b1670 con 0x5650b848f2f0 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01747db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01681af4e0 con 0x5650b84b81b0 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 0x5650b8865140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 0x5650b8865140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60892/0 (socket says 172.21.15.5:60892) 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 -- 172.21.15.5:0/2408119503 learned_addr learned my addr 172.21.15.5:0/2408119503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f0166ffd640 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b848f2f0 0x5650b8859b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01677fe640 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b886ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 -- 172.21.15.5:0/2408119503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 msgr2=0x5650b886ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b886ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 -- 172.21.15.5:0/2408119503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b848f2f0 msgr2=0x5650b8859b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b848f2f0 0x5650b8859b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.571+0000 7f01667fc640 1 -- 172.21.15.5:0/2408119503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b8862090 con 0x5650b84974d0 2024-05-14T15:32:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.572+0000 7f01667fc640 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 0x5650b8865140 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f015c005e00 tx=0x7f015c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9a7931fcae6b0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.572+0000 7f014d7fa640 1 -- 172.21.15.5:0/2408119503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c055020 con 0x5650b84974d0 2024-05-14T15:32:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.572+0000 7f014d7fa640 1 -- 172.21.15.5:0/2408119503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c009520 con 0x5650b84974d0 2024-05-14T15:32:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.572+0000 7f014d7fa640 1 -- 172.21.15.5:0/2408119503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c035040 con 0x5650b84974d0 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 -- 172.21.15.5:0/2408119503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 msgr2=0x5650b8865140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 0x5650b8865140 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f015c005e00 tx=0x7f015c000bc0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 -- 172.21.15.5:0/2408119503 shutdown_connections 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b84b81b0 0x5650b886ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84974d0 0x5650b8865140 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 --2- 172.21.15.5:0/2408119503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b848f2f0 0x5650b8859b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 -- 172.21.15.5:0/2408119503 >> 172.21.15.5:0/2408119503 conn(0x5650b84ac300 msgr2=0x5650b8853970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 -- 172.21.15.5:0/2408119503 shutdown_connections 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 -- 172.21.15.5:0/2408119503 wait complete. 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.573+0000 7f01747db740 1 -- start start 2024-05-14T15:32:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 Processor -- start 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8866080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b88665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b88db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01681afb30 con 0x5650b84b81b0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b0fa0 con 0x5650b84974d0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b0200 con 0x5650b848f2f0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01747db740 10 client.0 _ll_register_callbacks cb 0x7f0158009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b88db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b88665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f0166ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8866080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b88665e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:39472/0 (socket says 172.21.15.5:39472) 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 -- 172.21.15.5:0/1319367688 learned_addr learned my addr 172.21.15.5:0/1319367688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 -- 172.21.15.5:0/1319367688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 msgr2=0x5650b8866080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8866080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 -- 172.21.15.5:0/1319367688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 msgr2=0x5650b88db8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b88db8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.574+0000 7f01667fc640 1 -- 172.21.15.5:0/1319367688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b88dbe30 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f01667fc640 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b88665e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f015c02aa00 tx=0x7f015c004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2382d9f2395a7dad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f0146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f0146ffd640 1 -- 172.21.15.5:0/1319367688 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c055030 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f0146ffd640 1 -- 172.21.15.5:0/1319367688 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f015c031a50 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c009520 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f0146ffd640 1 -- 172.21.15.5:0/1319367688 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f015c05e070 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f01747db740 10 client.26939 fetch_fsmap learned FSMap version 32 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650b88dcf20 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.575+0000 7f0146ffd640 1 -- 172.21.15.5:0/1319367688 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f015c009520 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f0146ffd640 1 -- 172.21.15.5:0/1319367688 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f015c0510f0 con 0x5650b84974d0 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f01747db740 10 client.26939 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f01747db740 10 client.26939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f01747db740 20 client.26939 populate_metadata read hostname 'smithi005' 2024-05-14T15:32:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f01747db740 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b844c1c0 0x5650b88df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:32:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f01747db740 4 client.26939 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:32:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650b87c6050 con 0x5650b844c1c0 2024-05-14T15:32:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f0166ffd640 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b844c1c0 0x5650b88df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:32:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f0146ffd640 10 client.26939 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.576+0000 7f0166ffd640 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b844c1c0 0x5650b88df160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85823e4e3dc6012 server_cookie=39001d2288f1d13d in_seq=0 out_seq=0 2024-05-14T15:32:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f0146ffd640 1 -- 172.21.15.5:0/1319367688 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650b87c6050 con 0x5650b844c1c0 2024-05-14T15:32:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f0146ffd640 10 client.26939 handle_command_reply: tid=0 2024-05-14T15:32:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 client.26939 shutdown 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 msgr2=0x5650b88665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b88665e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f015c02aa00 tx=0x7f015c004590 comp rx=0 tx=0).stop 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 shutdown_connections 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650b844c1c0 0x5650b88df160 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b84b81b0 0x5650b88db8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f0146ffd640 0 client.26939 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b84974d0 0x5650b88665e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 --2- 172.21.15.5:0/1319367688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b848f2f0 0x5650b8866080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 >> 172.21.15.5:0/1319367688 conn(0x5650b84ac300 msgr2=0x5650b8853970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 shutdown_connections 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 1 -- 172.21.15.5:0/1319367688 wait complete. 2024-05-14T15:32:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.860+0000 7f01747db740 20 client.26939 trim_cache size 0 max 16384 2024-05-14T15:32:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:32:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:32:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:32:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:32:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f01500b2950 msgr2=0x7f01500b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f01500b2950 0x7f01500b4e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5650b8855d00 tx=0x7f0160009500 comp rx=0 tx=0).stop 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 msgr2=0x7f01681a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 0x7f01681a05a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0168103870 tx=0x7f015800dbb0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 shutdown_connections 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f01500b2950 0x7f01500b4e60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0168109030 0x7f016819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0168108b90 0x7f01681a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 --2- 172.21.15.5:0/1227007836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01681032e0 0x7f01681a05a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 >> 172.21.15.5:0/1227007836 conn(0x7f01680fe390 msgr2=0x7f016810e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 shutdown_connections 2024-05-14T15:32:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:32:54.861+0000 7f016d843640 1 -- 172.21.15.5:0/1227007836 wait complete. 2024-05-14T15:32:55.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 760242 10.29 MB/sec execute 256 sec latency 225.390 ms 2024-05-14T15:32:56.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 761534 10.27 MB/sec execute 257 sec latency 68.827 ms 2024-05-14T15:32:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:55 smithi144 ceph-mon[35650]: pgmap v250: 129 pgs: 129 active+clean; 81 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-05-14T15:32:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:55 smithi193 ceph-mon[36001]: pgmap v250: 129 pgs: 129 active+clean; 81 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-05-14T15:32:56.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:55 smithi005 ceph-mon[31559]: pgmap v250: 129 pgs: 129 active+clean; 81 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-05-14T15:32:57.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 763827 10.28 MB/sec execute 258 sec latency 16.482 ms 2024-05-14T15:32:58.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 765745 10.27 MB/sec execute 259 sec latency 30.057 ms 2024-05-14T15:32:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:57 smithi144 ceph-mon[35650]: pgmap v251: 129 pgs: 129 active+clean; 67 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-05-14T15:32:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:57 smithi193 ceph-mon[36001]: pgmap v251: 129 pgs: 129 active+clean; 67 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-05-14T15:32:58.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:57 smithi005 ceph-mon[31559]: pgmap v251: 129 pgs: 129 active+clean; 67 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-05-14T15:32:59.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 767576 10.28 MB/sec execute 260 sec latency 13.681 ms 2024-05-14T15:33:00.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 769676 10.28 MB/sec execute 261 sec latency 19.641 ms 2024-05-14T15:33:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:32:59 smithi144 ceph-mon[35650]: pgmap v252: 129 pgs: 129 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-05-14T15:33:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:32:59 smithi193 ceph-mon[36001]: pgmap v252: 129 pgs: 129 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-05-14T15:33:00.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:32:59 smithi005 ceph-mon[31559]: pgmap v252: 129 pgs: 129 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-05-14T15:33:01.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 770212 10.25 MB/sec execute 262 sec latency 270.521 ms 2024-05-14T15:33:02.106 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:01 smithi193 ceph-mon[36001]: pgmap v253: 129 pgs: 129 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-05-14T15:33:02.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 771040 10.23 MB/sec execute 263 sec latency 110.489 ms 2024-05-14T15:33:02.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:01 smithi144 ceph-mon[35650]: pgmap v253: 129 pgs: 129 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-05-14T15:33:02.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:01 smithi005 ceph-mon[31559]: pgmap v253: 129 pgs: 129 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-05-14T15:33:03.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 772889 10.22 MB/sec execute 264 sec latency 17.802 ms 2024-05-14T15:33:04.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:03 smithi005 ceph-mon[31559]: pgmap v254: 129 pgs: 129 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 440 op/s 2024-05-14T15:33:04.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 774807 10.23 MB/sec execute 265 sec latency 17.868 ms 2024-05-14T15:33:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:03 smithi144 ceph-mon[35650]: pgmap v254: 129 pgs: 129 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 440 op/s 2024-05-14T15:33:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:03 smithi193 ceph-mon[36001]: pgmap v254: 129 pgs: 129 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 440 op/s 2024-05-14T15:33:04.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:33:05.062 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:05.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 777038 10.24 MB/sec execute 266 sec latency 17.364 ms 2024-05-14T15:33:06.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 778210 10.22 MB/sec execute 267 sec latency 58.238 ms 2024-05-14T15:33:06.208 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:05 smithi005 ceph-mon[31559]: pgmap v255: 129 pgs: 129 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 336 op/s 2024-05-14T15:33:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:05 smithi144 ceph-mon[35650]: pgmap v255: 129 pgs: 129 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 336 op/s 2024-05-14T15:33:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:05 smithi193 ceph-mon[36001]: pgmap v255: 129 pgs: 129 active+clean; 76 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 336 op/s 2024-05-14T15:33:07.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:06 smithi005 ceph-mon[31559]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T15:33:07.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 778474 10.19 MB/sec execute 268 sec latency 120.082 ms 2024-05-14T15:33:07.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.237+0000 7fca5ee29700 1 -- 172.21.15.5:0/741211028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca581086a0 msgr2=0x7fca58108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.237+0000 7fca5ee29700 1 --2- 172.21.15.5:0/741211028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca581086a0 0x7fca58108b70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca48009990 tx=0x7fca48009ca0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.239+0000 7fca5ee29700 1 -- 172.21.15.5:0/741211028 shutdown_connections 2024-05-14T15:33:07.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.239+0000 7fca5ee29700 1 --2- 172.21.15.5:0/741211028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca581086a0 0x7fca58108b70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.239+0000 7fca5ee29700 1 --2- 172.21.15.5:0/741211028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103250 0x7fca58103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.239+0000 7fca5ee29700 1 --2- 172.21.15.5:0/741211028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca58103ba0 0x7fca581080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.239+0000 7fca5ee29700 1 -- 172.21.15.5:0/741211028 >> 172.21.15.5:0/741211028 conn(0x7fca580fedc0 msgr2=0x7fca581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:07.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.241+0000 7fca5ee29700 1 -- 172.21.15.5:0/741211028 shutdown_connections 2024-05-14T15:33:07.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.242+0000 7fca5ee29700 1 -- 172.21.15.5:0/741211028 wait complete. 2024-05-14T15:33:07.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.243+0000 7fca5ee29700 1 Processor -- start 2024-05-14T15:33:07.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 -- start start 2024-05-14T15:33:07.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca58103250 0x7fca581a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 0x7fca581a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581086a0 0x7fca581a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca5811aa70 con 0x7fca58103ba0 2024-05-14T15:33:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca5811a8f0 con 0x7fca581086a0 2024-05-14T15:33:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.245+0000 7fca5ee29700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fca5811abf0 con 0x7fca58103250 2024-05-14T15:33:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.246+0000 7fca57fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 0x7fca581a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.246+0000 7fca57fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 0x7fca581a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50656/0 (socket says 172.21.15.5:50656) 2024-05-14T15:33:07.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.246+0000 7fca57fff700 1 -- 172.21.15.5:0/2298663433 learned_addr learned my addr 172.21.15.5:0/2298663433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.247+0000 7fca57fff700 1 -- 172.21.15.5:0/2298663433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca58103250 msgr2=0x7fca581a13a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T15:33:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.247+0000 7fca57fff700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca58103250 0x7fca581a13a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.247+0000 7fca57fff700 1 -- 172.21.15.5:0/2298663433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581086a0 msgr2=0x7fca581a5bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:33:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.247+0000 7fca57fff700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581086a0 0x7fca581a5bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.247+0000 7fca57fff700 1 -- 172.21.15.5:0/2298663433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca581a6270 con 0x7fca58103ba0 2024-05-14T15:33:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.247+0000 7fca57fff700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 0x7fca581a1940 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fca5000ef10 tx=0x7fca5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.248+0000 7fca55ffb700 1 -- 172.21.15.5:0/2298663433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cad0 con 0x7fca58103ba0 2024-05-14T15:33:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.248+0000 7fca55ffb700 1 -- 172.21.15.5:0/2298663433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca50019070 con 0x7fca58103ba0 2024-05-14T15:33:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.249+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca581a6500 con 0x7fca58103ba0 2024-05-14T15:33:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.249+0000 7fca55ffb700 1 -- 172.21.15.5:0/2298663433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cad0 con 0x7fca58103ba0 2024-05-14T15:33:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.249+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca581ade80 con 0x7fca58103ba0 2024-05-14T15:33:07.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.256+0000 7fca6d83db80 1 Processor -- start 2024-05-14T15:33:07.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.257+0000 7fca55ffb700 1 -- 172.21.15.5:0/2298663433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fca50012050 con 0x7fca58103ba0 2024-05-14T15:33:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.258+0000 7fca55ffb700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fca400b2e40 0x7fca400b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.259+0000 7fca55ffb700 1 -- 172.21.15.5:0/2298663433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7fca500092e0 con 0x7fca58103ba0 2024-05-14T15:33:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 -- start start 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 0x5632b198b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fca400b2e40 0x7fca400b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b19858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1985e10 0x5632b1986220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632b18a7940 con 0x5632b198bd80 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5632b180cda0 con 0x5632b1807e40 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.260+0000 7fca6d83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632b18a3900 con 0x5632b1985e10 2024-05-14T15:33:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.261+0000 7fca5cbc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 0x5632b198b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.261+0000 7fca5d3c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1985e10 0x5632b1986220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.261+0000 7fca5cbc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 0x5632b198b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48356/0 (socket says 172.21.15.5:48356) 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.261+0000 7fca5cbc5700 1 -- 172.21.15.5:0/1107742093 learned_addr learned my addr 172.21.15.5:0/1107742093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.261+0000 7fca57fff700 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b19858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca57fff700 1 -- 172.21.15.5:0/1107742093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1985e10 msgr2=0x5632b1986220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca57fff700 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1985e10 0x5632b1986220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca57fff700 1 -- 172.21.15.5:0/1107742093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 msgr2=0x5632b198b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fca400b2e40 0x7fca400b5350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca480098e0 tx=0x7fca48005100 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca57fff700 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 0x5632b198b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca57fff700 1 -- 172.21.15.5:0/1107742093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b17f48d0 con 0x5632b198bd80 2024-05-14T15:33:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.262+0000 7fca5cbc5700 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 0x5632b198b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.265+0000 7fca57fff700 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b19858d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fca500dea60 tx=0x7fca500adb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3727ebc9df334244 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:07.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.265+0000 7fca3effd700 1 -- 172.21.15.5:0/1107742093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500a4e70 con 0x5632b198bd80 2024-05-14T15:33:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.268+0000 7fca3effd700 1 -- 172.21.15.5:0/1107742093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca50015ec0 con 0x5632b198bd80 2024-05-14T15:33:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.269+0000 7fca6d83db80 1 -- 172.21.15.5:0/1107742093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 msgr2=0x5632b19858d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.269+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b19858d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fca500dea60 tx=0x7fca500adb50 comp rx=0 tx=0).stop 2024-05-14T15:33:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.270+0000 7fca6d83db80 1 -- 172.21.15.5:0/1107742093 shutdown_connections 2024-05-14T15:33:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.270+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1985e10 0x5632b1986220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.270+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1807e40 0x5632b198b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.270+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1107742093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b19858d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.270+0000 7fca6d83db80 1 -- 172.21.15.5:0/1107742093 >> 172.21.15.5:0/1107742093 conn(0x5632b184f630 msgr2=0x5632b1980a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:07.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.272+0000 7fca6d83db80 1 -- 172.21.15.5:0/1107742093 shutdown_connections 2024-05-14T15:33:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.273+0000 7fca6d83db80 1 -- 172.21.15.5:0/1107742093 wait complete. 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.273+0000 7fca6d83db80 1 -- start start 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 Processor -- start 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1807e40 0x5632b198f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b198fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632b19926d0 con 0x5632b198bd80 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5632b1860ed0 con 0x5632b1985e10 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.274+0000 7fca6d83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632b1992850 con 0x5632b1807e40 2024-05-14T15:33:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.275+0000 7fca6d83db80 10 client.0 _ll_register_callbacks cb 0x5632b183f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.276+0000 7fca57fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.276+0000 7fca57fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48372/0 (socket says 172.21.15.5:48372) 2024-05-14T15:33:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.276+0000 7fca57fff700 1 -- 172.21.15.5:0/1147212063 learned_addr learned my addr 172.21.15.5:0/1147212063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.276+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b198fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.276+0000 7fca5cbc5700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1807e40 0x5632b198f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.278+0000 7fca57fff700 1 -- 172.21.15.5:0/1147212063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1807e40 msgr2=0x5632b198f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.278+0000 7fca57fff700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1807e40 0x5632b198f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.278+0000 7fca57fff700 1 -- 172.21.15.5:0/1147212063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 msgr2=0x5632b198fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.278+0000 7fca57fff700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b198fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.278+0000 7fca57fff700 1 -- 172.21.15.5:0/1147212063 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b185e760 con 0x5632b1985e10 2024-05-14T15:33:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.279+0000 7fca57fff700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198f910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca5000eee0 tx=0x7fca500def00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ee9a8dc610c52d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.279+0000 7fca37fff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.279+0000 7fca37fff700 1 -- 172.21.15.5:0/1147212063 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500a51e0 con 0x5632b1985e10 2024-05-14T15:33:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.279+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca50015ec0 con 0x5632b1985e10 2024-05-14T15:33:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.279+0000 7fca37fff700 1 -- 172.21.15.5:0/1147212063 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7fca500b1380 con 0x5632b1985e10 2024-05-14T15:33:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.281+0000 7fca37fff700 1 -- 172.21.15.5:0/1147212063 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca500a5380 con 0x5632b1985e10 2024-05-14T15:33:07.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.281+0000 7fca6d83db80 10 client.26948 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:07.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.281+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5632b19926d0 con 0x5632b1985e10 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca37fff700 1 -- 172.21.15.5:0/1147212063 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca50015ec0 con 0x5632b1985e10 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca37fff700 1 -- 172.21.15.5:0/1147212063 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fca500adc90 con 0x5632b1985e10 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca6d83db80 10 client.26948 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca6d83db80 10 client.26948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca6d83db80 20 client.26948 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1995530 0x5632b1997950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca6d83db80 4 client.26948 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:07.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632b18b5320 con 0x5632b1995530 2024-05-14T15:33:07.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.286+0000 7fca5cbc5700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1995530 0x5632b1997950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.288+0000 7fca5cbc5700 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1995530 0x5632b1997950 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f91b29a2e831232 server_cookie=341eee1364c584e6 in_seq=0 out_seq=0 2024-05-14T15:33:07.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.288+0000 7fca37fff700 10 client.26948 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:07.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.290+0000 7fca37fff700 1 -- 172.21.15.5:0/1147212063 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632b18b5320 con 0x5632b1995530 2024-05-14T15:33:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.290+0000 7fca37fff700 10 client.26948 handle_command_reply: tid=0 2024-05-14T15:33:07.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.290+0000 7fca6d83db80 1 client.26948 shutdown 2024-05-14T15:33:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:06 smithi144 ceph-mon[35650]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T15:33:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.297+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 msgr2=0x5632b198f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.297+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198f910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca5000eee0 tx=0x7fca500def00 comp rx=0 tx=0).stop 2024-05-14T15:33:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 shutdown_connections 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1995530 0x5632b1997950 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b198bd80 0x5632b198fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b1807e40 0x5632b198f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca6d83db80 1 --2- 172.21.15.5:0/1147212063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198f910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 >> 172.21.15.5:0/1147212063 conn(0x5632b184f630 msgr2=0x5632b19848c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.300+0000 7fca37fff700 0 client.26948 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.301+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 shutdown_connections 2024-05-14T15:33:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.301+0000 7fca6d83db80 1 -- 172.21.15.5:0/1147212063 wait complete. 2024-05-14T15:33:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:06 smithi193 ceph-mon[36001]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T15:33:07.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.301+0000 7fca6d83db80 20 client.26948 trim_cache size 0 max 16384 2024-05-14T15:33:07.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.343+0000 7fca6d83db80 1 Processor -- start 2024-05-14T15:33:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.348+0000 7fca6d83db80 1 -- start start 2024-05-14T15:33:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.348+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 0x5632b19858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.349+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.349+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b19ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.349+0000 7fca6d83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca581b1020 con 0x5632b1807e40 2024-05-14T15:33:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.349+0000 7fca5d3c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b19ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.349+0000 7fca6d83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca581ae8b0 con 0x5632b1985e10 2024-05-14T15:33:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.350+0000 7fca6d83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fca580740b0 con 0x5632b198bd80 2024-05-14T15:33:07.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.352+0000 7fca5d3c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b19ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50242/0 (socket says 172.21.15.5:50242) 2024-05-14T15:33:07.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.352+0000 7fca5d3c6700 1 -- 172.21.15.5:0/2480480895 learned_addr learned my addr 172.21.15.5:0/2480480895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.353+0000 7fca5d3c6700 1 -- 172.21.15.5:0/2480480895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 msgr2=0x5632b198ac80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:33:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.353+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.353+0000 7fca5d3c6700 1 -- 172.21.15.5:0/2480480895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 msgr2=0x5632b19858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.353+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 0x5632b19858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.353+0000 7fca5d3c6700 1 -- 172.21.15.5:0/2480480895 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b1981620 con 0x5632b198bd80 2024-05-14T15:33:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.354+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b19ac7e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca4800b840 tx=0x7fca48004000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=349cdf817bb89933 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:07.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.354+0000 7fca3e7fc700 1 -- 172.21.15.5:0/2480480895 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48034070 con 0x5632b198bd80 2024-05-14T15:33:07.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.354+0000 7fca3e7fc700 1 -- 172.21.15.5:0/2480480895 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca48003db0 con 0x5632b198bd80 2024-05-14T15:33:07.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.354+0000 7fca3e7fc700 1 -- 172.21.15.5:0/2480480895 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4802b040 con 0x5632b198bd80 2024-05-14T15:33:07.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.355+0000 7fca6d83db80 1 -- 172.21.15.5:0/2480480895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 msgr2=0x5632b19ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.355+0000 7fca6d83db80 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b19ac7e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fca4800b840 tx=0x7fca48004000 comp rx=0 tx=0).stop 2024-05-14T15:33:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 -- 172.21.15.5:0/2480480895 shutdown_connections 2024-05-14T15:33:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b19ac7e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 0x5632b19858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 --2- 172.21.15.5:0/2480480895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b198ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 -- 172.21.15.5:0/2480480895 >> 172.21.15.5:0/2480480895 conn(0x5632b184f630 msgr2=0x5632b1990a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 -- 172.21.15.5:0/2480480895 shutdown_connections 2024-05-14T15:33:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.358+0000 7fca6d83db80 1 -- 172.21.15.5:0/2480480895 wait complete. 2024-05-14T15:33:07.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.359+0000 7fca6d83db80 1 -- start start 2024-05-14T15:33:07.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.359+0000 7fca6d83db80 1 Processor -- start 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 0x5632b1a24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b1a246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b1999b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca58074700 con 0x5632b1807e40 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca581ae1e0 con 0x5632b1985e10 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fca58074dd0 con 0x5632b198bd80 2024-05-14T15:33:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.361+0000 7fca6d83db80 10 client.0 _ll_register_callbacks cb 0x5632b183f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.363+0000 7fca57fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b1a246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.363+0000 7fca57fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b1a246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48392/0 (socket says 172.21.15.5:48392) 2024-05-14T15:33:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca57fff700 1 -- 172.21.15.5:0/3384138154 learned_addr learned my addr 172.21.15.5:0/3384138154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca5d3c6700 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b1999b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca57fff700 1 -- 172.21.15.5:0/3384138154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 msgr2=0x5632b1999b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca57fff700 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b1999b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca57fff700 1 -- 172.21.15.5:0/3384138154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 msgr2=0x5632b1a24160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca57fff700 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 0x5632b1a24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.364+0000 7fca57fff700 1 -- 172.21.15.5:0/3384138154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b199a1d0 con 0x5632b1985e10 2024-05-14T15:33:07.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.366+0000 7fca57fff700 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b1a246a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca500090a0 tx=0x7fca5000cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf80f9fd5b37cd67 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:07.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.366+0000 7fca37fff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.367+0000 7fca37fff700 1 -- 172.21.15.5:0/3384138154 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500a5550 con 0x5632b1985e10 2024-05-14T15:33:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.367+0000 7fca37fff700 1 -- 172.21.15.5:0/3384138154 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7fca500a4bf0 con 0x5632b1985e10 2024-05-14T15:33:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.367+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca48003db0 con 0x5632b1985e10 2024-05-14T15:33:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.368+0000 7fca37fff700 1 -- 172.21.15.5:0/3384138154 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fca500a5720 con 0x5632b1985e10 2024-05-14T15:33:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.368+0000 7fca6d83db80 10 client.26951 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.368+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5632b199a1d0 con 0x5632b1985e10 2024-05-14T15:33:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.368+0000 7fca37fff700 1 -- 172.21.15.5:0/3384138154 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5000ce10 con 0x5632b1985e10 2024-05-14T15:33:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.368+0000 7fca37fff700 1 -- 172.21.15.5:0/3384138154 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fca500c9440 con 0x5632b1985e10 2024-05-14T15:33:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.369+0000 7fca6d83db80 10 client.26951 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.369+0000 7fca6d83db80 10 client.26951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.369+0000 7fca6d83db80 20 client.26951 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.369+0000 7fca6d83db80 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1a2baf0 0x5632b1a2df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.369+0000 7fca6d83db80 4 client.26951 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:33:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.369+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fca48003db0 con 0x5632b1a2baf0 2024-05-14T15:33:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.375+0000 7fca5cbc5700 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1a2baf0 0x5632b1a2df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.383+0000 7fca37fff700 10 client.26951 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.383+0000 7fca5cbc5700 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1a2baf0 0x5632b1a2df10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d483e00c69c89397 server_cookie=fe371282aa6839c4 in_seq=0 out_seq=0 2024-05-14T15:33:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.384+0000 7fca37fff700 1 -- 172.21.15.5:0/3384138154 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fca48003db0 con 0x5632b1a2baf0 2024-05-14T15:33:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.384+0000 7fca37fff700 10 client.26951 handle_command_reply: tid=0 2024-05-14T15:33:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.384+0000 7fca6d83db80 1 client.26951 shutdown 2024-05-14T15:33:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.386+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 msgr2=0x5632b1a246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.386+0000 7fca6d83db80 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b1a246a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca500090a0 tx=0x7fca5000cd60 comp rx=0 tx=0).stop 2024-05-14T15:33:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.387+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 shutdown_connections 2024-05-14T15:33:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.387+0000 7fca6d83db80 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5632b1a2baf0 0x5632b1a2df10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.387+0000 7fca6d83db80 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b198bd80 0x5632b1999b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.387+0000 7fca6d83db80 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632b1807e40 0x5632b1a24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.387+0000 7fca6d83db80 1 --2- 172.21.15.5:0/3384138154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632b1985e10 0x5632b1a246a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.387+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 >> 172.21.15.5:0/3384138154 conn(0x5632b184f630 msgr2=0x5632b197ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.388+0000 7fca37fff700 0 client.26951 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.391+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 shutdown_connections 2024-05-14T15:33:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.391+0000 7fca6d83db80 1 -- 172.21.15.5:0/3384138154 wait complete. 2024-05-14T15:33:07.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.391+0000 7fca6d83db80 20 client.26951 trim_cache size 0 max 16384 2024-05-14T15:33:07.395 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:33:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:33:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:33:07.395 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:33:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fca400b2e40 msgr2=0x7fca400b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fca400b2e40 0x7fca400b5350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca480098e0 tx=0x7fca48005100 comp rx=0 tx=0).stop 2024-05-14T15:33:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 msgr2=0x7fca581a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 0x7fca581a1940 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fca5000ef10 tx=0x7fca5000c3a0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 shutdown_connections 2024-05-14T15:33:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fca400b2e40 0x7fca400b5350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581086a0 0x7fca581a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca58103250 0x7fca581a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 --2- 172.21.15.5:0/2298663433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca58103ba0 0x7fca581a1940 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.395+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 >> 172.21.15.5:0/2298663433 conn(0x7fca580fedc0 msgr2=0x7fca581060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.397+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 shutdown_connections 2024-05-14T15:33:07.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:07.398+0000 7fca5ee29700 1 -- 172.21.15.5:0/2298663433 wait complete. 2024-05-14T15:33:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.036+0000 7f88440c8640 1 -- 172.21.15.5:0/2995051759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 msgr2=0x7f883c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.036+0000 7f88440c8640 1 --2- 172.21.15.5:0/2995051759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 0x7f883c103db0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8834009ca0 tx=0x7f883402a8f0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.039+0000 7f88440c8640 1 -- 172.21.15.5:0/2995051759 shutdown_connections 2024-05-14T15:33:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.039+0000 7f88440c8640 1 --2- 172.21.15.5:0/2995051759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f883c104420 0x7f883c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.039+0000 7f88440c8640 1 --2- 172.21.15.5:0/2995051759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 0x7f883c103db0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.039+0000 7f88440c8640 1 --2- 172.21.15.5:0/2995051759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.039+0000 7f88440c8640 1 -- 172.21.15.5:0/2995051759 >> 172.21.15.5:0/2995051759 conn(0x7f883c0793d0 msgr2=0x7f883c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.041+0000 7f88440c8640 1 -- 172.21.15.5:0/2995051759 shutdown_connections 2024-05-14T15:33:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.043+0000 7f88440c8640 1 -- 172.21.15.5:0/2995051759 wait complete. 2024-05-14T15:33:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.044+0000 7f88440c8640 1 Processor -- start 2024-05-14T15:33:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.045+0000 7f88440c8640 1 -- start start 2024-05-14T15:33:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.047+0000 7f88440c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f883c104420 0x7f883c1a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.047+0000 7f88440c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c1a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.047+0000 7f88440c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 0x7f883c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.047+0000 7f88440c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f883c116210 con 0x7f883c10a170 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.047+0000 7f88440c8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f883c116090 con 0x7f883c109cd0 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.047+0000 7f88440c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f883c116390 con 0x7f883c104420 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.049+0000 7f8841663640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c1a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.049+0000 7f8841663640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c1a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48406/0 (socket says 172.21.15.5:48406) 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.049+0000 7f8841663640 1 -- 172.21.15.5:0/655312015 learned_addr learned my addr 172.21.15.5:0/655312015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.049+0000 7f8842665640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 0x7f883c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.050+0000 7f8841663640 1 -- 172.21.15.5:0/655312015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f883c104420 msgr2=0x7f883c1a28a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.050+0000 7f8841663640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f883c104420 0x7f883c1a28a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.050+0000 7f8841663640 1 -- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 msgr2=0x7f883c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.050+0000 7f8841663640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 0x7f883c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.050+0000 7f8841663640 1 -- 172.21.15.5:0/655312015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c19d150 con 0x7f883c109cd0 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.051+0000 7f8841663640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c1a2de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8834009bf0 tx=0x7f8834003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.052+0000 7f883affd640 1 -- 172.21.15.5:0/655312015 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834003ce0 con 0x7f883c109cd0 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.052+0000 7f883affd640 1 -- 172.21.15.5:0/655312015 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883402d070 con 0x7f883c109cd0 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.052+0000 7f883affd640 1 -- 172.21.15.5:0/655312015 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883402fa40 con 0x7f883c109cd0 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.052+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c19d3e0 con 0x7f883c109cd0 2024-05-14T15:33:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.052+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c19d8e0 con 0x7f883c109cd0 2024-05-14T15:33:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.056+0000 7f884af7c740 1 Processor -- start 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 -- start start 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b5cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab0479b4d0 0x55ab04b5d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab04ac25d0 con 0x55ab047bc1b0 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab04a7b890 con 0x55ab047932f0 2024-05-14T15:33:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.057+0000 7f884af7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab04abe590 con 0x55ab0479b4d0 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b5d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48412/0 (socket says 172.21.15.5:48412) 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 -- 172.21.15.5:0/940340815 learned_addr learned my addr 172.21.15.5:0/940340815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8842665640 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b5cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 -- 172.21.15.5:0/940340815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab0479b4d0 msgr2=0x55ab04b5d550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab0479b4d0 0x55ab04b5d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 -- 172.21.15.5:0/940340815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 msgr2=0x55ab04b5cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.058+0000 7f8841e64640 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b5cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.059+0000 7f8841e64640 1 -- 172.21.15.5:0/940340815 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab047d4210 con 0x55ab047932f0 2024-05-14T15:33:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.059+0000 7f8841e64640 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b5d010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88240096f0 tx=0x7f8824009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3a0c4f6914ec18 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.059+0000 7f8833fff640 1 -- 172.21.15.5:0/940340815 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824015020 con 0x55ab047932f0 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.060+0000 7f8833fff640 1 -- 172.21.15.5:0/940340815 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882400bd80 con 0x55ab047932f0 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.060+0000 7f8833fff640 1 -- 172.21.15.5:0/940340815 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824010380 con 0x55ab047932f0 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.062+0000 7f883affd640 1 -- 172.21.15.5:0/655312015 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f883402fbe0 con 0x7f883c109cd0 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.062+0000 7f884af7c740 1 -- 172.21.15.5:0/940340815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 msgr2=0x55ab04b5d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.062+0000 7f884af7c740 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b5d010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88240096f0 tx=0x7f8824009bb0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.063+0000 7f884af7c740 1 -- 172.21.15.5:0/940340815 shutdown_connections 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.063+0000 7f884af7c740 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab0479b4d0 0x55ab04b5d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.063+0000 7f884af7c740 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b5d010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.063+0000 7f884af7c740 1 --2- 172.21.15.5:0/940340815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b5cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.063+0000 7f884af7c740 1 -- 172.21.15.5:0/940340815 >> 172.21.15.5:0/940340815 conn(0x55ab047b0300 msgr2=0x55ab04b59340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.063+0000 7f884af7c740 1 -- 172.21.15.5:0/940340815 shutdown_connections 2024-05-14T15:33:08.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:07 smithi005 ceph-mon[31559]: pgmap v257: 129 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 98 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-05-14T15:33:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.064+0000 7f884af7c740 1 -- 172.21.15.5:0/940340815 wait complete. 2024-05-14T15:33:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.064+0000 7f883affd640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f88280b29b0 0x7f88280b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.064+0000 7f884af7c740 1 -- start start 2024-05-14T15:33:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.064+0000 7f883affd640 1 -- 172.21.15.5:0/655312015 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7f88341040d0 con 0x7f883c109cd0 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.064+0000 7f884af7c740 1 Processor -- start 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.065+0000 7f8841e64640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f88280b29b0 0x7f88280b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b6f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b6ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047bc1b0 0x55ab04b7ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab04b72520 con 0x55ab0479b4d0 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab04a7f370 con 0x55ab047bc1b0 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab04b726a0 con 0x55ab047932f0 2024-05-14T15:33:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f8841e64640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f88280b29b0 0x7f88280b4ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8824011880 tx=0x7f8824011b20 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f884af7c740 10 client.0 _ll_register_callbacks cb 0x55ab047d3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f8841e64640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b6f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f8841e64640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b6f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50266/0 (socket says 172.21.15.5:50266) 2024-05-14T15:33:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f8841e64640 1 -- 172.21.15.5:0/3727193695 learned_addr learned my addr 172.21.15.5:0/3727193695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f8841663640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b6ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.066+0000 7f8842665640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047bc1b0 0x55ab04b7ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f8841663640 1 -- 172.21.15.5:0/3727193695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 msgr2=0x55ab04b6f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f8841663640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b6f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f8841663640 1 -- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047bc1b0 msgr2=0x55ab04b7ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f8841663640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047bc1b0 0x55ab04b7ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f8841663640 1 -- 172.21.15.5:0/3727193695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab048a0620 con 0x55ab0479b4d0 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f88317fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.067+0000 7f8841663640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b6ff00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8834101460 tx=0x7f88340caa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d334fc83aa8bb1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.068+0000 7f88317fa640 1 -- 172.21.15.5:0/3727193695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88341017e0 con 0x55ab0479b4d0 2024-05-14T15:33:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.068+0000 7f88317fa640 1 -- 172.21.15.5:0/3727193695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7f88340c75e0 con 0x55ab0479b4d0 2024-05-14T15:33:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.068+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882400bd80 con 0x55ab0479b4d0 2024-05-14T15:33:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.068+0000 7f88317fa640 1 -- 172.21.15.5:0/3727193695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88340ea050 con 0x55ab0479b4d0 2024-05-14T15:33:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.071+0000 7f884af7c740 10 client.17343 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.071+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab04b7e8e0 con 0x55ab0479b4d0 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.073+0000 7f88317fa640 1 -- 172.21.15.5:0/3727193695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882400bd80 con 0x55ab0479b4d0 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.073+0000 7f88317fa640 1 -- 172.21.15.5:0/3727193695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f88340d3ac0 con 0x55ab0479b4d0 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.074+0000 7f884af7c740 10 client.17343 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.074+0000 7f884af7c740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.074+0000 7f884af7c740 20 client.17343 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.074+0000 7f884af7c740 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab0477ad80 0x55ab04b81de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.074+0000 7f884af7c740 4 client.17343 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.074+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab04ad01e0 con 0x55ab0477ad80 2024-05-14T15:33:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.076+0000 7f8841e64640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab0477ad80 0x55ab04b81de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.085+0000 7f8841e64640 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab0477ad80 0x55ab04b81de0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8217ab8dbabdd server_cookie=bc77c33fc38d0c4 in_seq=0 out_seq=0 2024-05-14T15:33:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.085+0000 7f88317fa640 10 client.17343 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.087+0000 7f88317fa640 1 -- 172.21.15.5:0/3727193695 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab04ad01e0 con 0x55ab0477ad80 2024-05-14T15:33:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.087+0000 7f88317fa640 10 client.17343 handle_command_reply: tid=0 2024-05-14T15:33:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.088+0000 7f884af7c740 1 client.17343 shutdown 2024-05-14T15:33:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.095+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 msgr2=0x55ab04b6ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.095+0000 7f884af7c740 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b6ff00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8834101460 tx=0x7f88340caa70 comp rx=0 tx=0).stop 2024-05-14T15:33:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.095+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 shutdown_connections 2024-05-14T15:33:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.095+0000 7f884af7c740 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab0477ad80 0x55ab04b81de0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.096+0000 7f884af7c740 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047bc1b0 0x55ab04b7ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.096+0000 7f884af7c740 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b6ff00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.096+0000 7f884af7c740 1 --2- 172.21.15.5:0/3727193695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b6f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.096+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 >> 172.21.15.5:0/3727193695 conn(0x55ab047b0300 msgr2=0x55ab04b5c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.096+0000 7f88317fa640 0 client.17343 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.098+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 shutdown_connections 2024-05-14T15:33:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.099+0000 7f884af7c740 1 -- 172.21.15.5:0/3727193695 wait complete. 2024-05-14T15:33:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.100+0000 7f884af7c740 20 client.17343 trim_cache size 0 max 16384 2024-05-14T15:33:08.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:33:08.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 Processor -- start 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 -- start start 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b5f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 0x55ab04b71600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f881c012a00 con 0x55ab047bc1b0 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0102d0 con 0x55ab0479b4d0 2024-05-14T15:33:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f884af7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f881c00e100 con 0x55ab047932f0 2024-05-14T15:33:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f8841663640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 0x55ab04b71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f8841663640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 0x55ab04b71600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48428/0 (socket says 172.21.15.5:48428) 2024-05-14T15:33:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.133+0000 7f8841663640 1 -- 172.21.15.5:0/1083478838 learned_addr learned my addr 172.21.15.5:0/1083478838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.134+0000 7f8842665640 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.135+0000 7f8841663640 1 -- 172.21.15.5:0/1083478838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 msgr2=0x55ab04b5f330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:33:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.135+0000 7f8841663640 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b5f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.135+0000 7f8841663640 1 -- 172.21.15.5:0/1083478838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 msgr2=0x55ab04b71b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.135+0000 7f8841663640 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b71b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.136+0000 7f8841663640 1 -- 172.21.15.5:0/1083478838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab04b5d8c0 con 0x55ab0479b4d0 2024-05-14T15:33:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.137+0000 7f8841663640 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 0x55ab04b71600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8834031a90 tx=0x7f88340ca940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e36e56e9382a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.137+0000 7f88337fe640 1 -- 172.21.15.5:0/1083478838 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340ea050 con 0x55ab0479b4d0 2024-05-14T15:33:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.138+0000 7f88337fe640 1 -- 172.21.15.5:0/1083478838 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8834036030 con 0x55ab0479b4d0 2024-05-14T15:33:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.138+0000 7f88337fe640 1 -- 172.21.15.5:0/1083478838 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340caff0 con 0x55ab0479b4d0 2024-05-14T15:33:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.139+0000 7f884af7c740 1 -- 172.21.15.5:0/1083478838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 msgr2=0x55ab04b71600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.139+0000 7f884af7c740 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 0x55ab04b71600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8834031a90 tx=0x7f88340ca940 comp rx=0 tx=0).stop 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.141+0000 7f884af7c740 1 -- 172.21.15.5:0/1083478838 shutdown_connections 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.141+0000 7f884af7c740 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab047bc1b0 0x55ab04b71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.141+0000 7f884af7c740 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab0479b4d0 0x55ab04b71600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.141+0000 7f884af7c740 1 --2- 172.21.15.5:0/1083478838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047932f0 0x55ab04b5f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.141+0000 7f884af7c740 1 -- 172.21.15.5:0/1083478838 >> 172.21.15.5:0/1083478838 conn(0x55ab047b0300 msgr2=0x55ab04b70ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:08.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.142+0000 7f884af7c740 1 -- 172.21.15.5:0/1083478838 shutdown_connections 2024-05-14T15:33:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.143+0000 7f884af7c740 1 -- 172.21.15.5:0/1083478838 wait complete. 2024-05-14T15:33:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.144+0000 7f884af7c740 1 -- start start 2024-05-14T15:33:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.144+0000 7f884af7c740 1 Processor -- start 2024-05-14T15:33:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.145+0000 7f884af7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b77eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.145+0000 7f884af7c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b75270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.145+0000 7f884af7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047bc1b0 0x55ab04b757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.146+0000 7f884af7c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f881c00e790 con 0x55ab0479b4d0 2024-05-14T15:33:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.146+0000 7f884af7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f881c00fc00 con 0x55ab047932f0 2024-05-14T15:33:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.146+0000 7f884af7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f881c00ee60 con 0x55ab047bc1b0 2024-05-14T15:33:08.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 779015 10.16 MB/sec execute 269 sec latency 90.970 ms 2024-05-14T15:33:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.146+0000 7f884af7c740 10 client.0 _ll_register_callbacks cb 0x7f8834009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b77eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b77eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48436/0 (socket says 172.21.15.5:48436) 2024-05-14T15:33:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 -- 172.21.15.5:0/454566982 learned_addr learned my addr 172.21.15.5:0/454566982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841663640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b75270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 -- 172.21.15.5:0/454566982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047bc1b0 msgr2=0x55ab04b757b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:33:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047bc1b0 0x55ab04b757b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 -- 172.21.15.5:0/454566982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 msgr2=0x55ab04b75270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b75270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841e64640 1 -- 172.21.15.5:0/454566982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab04b75e50 con 0x55ab047932f0 2024-05-14T15:33:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.147+0000 7f8841663640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b75270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.151+0000 7f8841e64640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b77eb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8824009640 tx=0x7f88240090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e15df34038bda32 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.153+0000 7f88317fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.153+0000 7f88317fa640 1 -- 172.21.15.5:0/454566982 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824015040 con 0x55ab047932f0 2024-05-14T15:33:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.153+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8834036030 con 0x55ab047932f0 2024-05-14T15:33:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.153+0000 7f88317fa640 1 -- 172.21.15.5:0/454566982 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7f882400bca0 con 0x55ab047932f0 2024-05-14T15:33:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.157+0000 7f88317fa640 1 -- 172.21.15.5:0/454566982 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8824009150 con 0x55ab047932f0 2024-05-14T15:33:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.157+0000 7f884af7c740 10 client.26966 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.157+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab04be9fe0 con 0x55ab047932f0 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f88317fa640 1 -- 172.21.15.5:0/454566982 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8834036030 con 0x55ab047932f0 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f88317fa640 1 -- 172.21.15.5:0/454566982 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8824003b40 con 0x55ab047932f0 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f884af7c740 10 client.26966 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f884af7c740 10 client.26966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f884af7c740 20 client.26966 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f884af7c740 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab047501c0 0x55ab04bec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f884af7c740 4 client.26966 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:33:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.158+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab04aca050 con 0x55ab047501c0 2024-05-14T15:33:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.160+0000 7f8841663640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab047501c0 0x55ab04bec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.174+0000 7f8841663640 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab047501c0 0x55ab04bec710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb2fd64596b80de server_cookie=85f992bef03e7d2e in_seq=0 out_seq=0 2024-05-14T15:33:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.174+0000 7f88317fa640 10 client.26966 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:07 smithi144 ceph-mon[35650]: pgmap v257: 129 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 98 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-05-14T15:33:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:07 smithi193 ceph-mon[36001]: pgmap v257: 129 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 98 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-05-14T15:33:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.568+0000 7f88317fa640 1 -- 172.21.15.5:0/454566982 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab04aca050 con 0x55ab047501c0 2024-05-14T15:33:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.568+0000 7f88317fa640 10 client.26966 handle_command_reply: tid=0 2024-05-14T15:33:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.568+0000 7f884af7c740 1 client.26966 shutdown 2024-05-14T15:33:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 msgr2=0x55ab04b77eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b77eb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8824009640 tx=0x7f88240090e0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 shutdown_connections 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ab047501c0 0x55ab04bec710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab047bc1b0 0x55ab04b757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab0479b4d0 0x55ab04b75270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 --2- 172.21.15.5:0/454566982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab047932f0 0x55ab04b77eb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 >> 172.21.15.5:0/454566982 conn(0x55ab047b0300 msgr2=0x55ab04b71890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.571+0000 7f88317fa640 0 client.26966 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.572+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 shutdown_connections 2024-05-14T15:33:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.573+0000 7f884af7c740 1 -- 172.21.15.5:0/454566982 wait complete. 2024-05-14T15:33:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.573+0000 7f884af7c740 20 client.26966 trim_cache size 0 max 16384 2024-05-14T15:33:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:33:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:33:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:33:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:33:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.575+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f88280b29b0 msgr2=0x7f88280b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.575+0000 7f88440c8640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f88280b29b0 0x7f88280b4ec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8824011880 tx=0x7f8824011b20 comp rx=0 tx=0).stop 2024-05-14T15:33:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.575+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 msgr2=0x7f883c1a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.575+0000 7f88440c8640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c1a2de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8834009bf0 tx=0x7f8834003970 comp rx=0 tx=0).stop 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 shutdown_connections 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f88280b29b0 0x7f88280b4ec0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c10a170 0x7f883c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c109cd0 0x7f883c1a2de0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 --2- 172.21.15.5:0/655312015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f883c104420 0x7f883c1a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 >> 172.21.15.5:0/655312015 conn(0x7f883c0793d0 msgr2=0x7f883c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 shutdown_connections 2024-05-14T15:33:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:08.576+0000 7f88440c8640 1 -- 172.21.15.5:0/655312015 wait complete. 2024-05-14T15:33:08.640 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:09.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 779249 10.13 MB/sec execute 270 sec latency 288.032 ms 2024-05-14T15:33:10.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:09 smithi005 ceph-mon[31559]: pgmap v258: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 89 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-14T15:33:10.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 780386 10.11 MB/sec execute 271 sec latency 49.598 ms 2024-05-14T15:33:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:09 smithi144 ceph-mon[35650]: pgmap v258: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 89 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-14T15:33:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:09 smithi193 ceph-mon[36001]: pgmap v258: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 89 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-14T15:33:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.875+0000 7f3fa0e54700 1 -- 172.21.15.5:0/2951583384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106570 msgr2=0x7f3f9c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.875+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/2951583384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106570 0x7f3f9c106980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f900098f0 tx=0x7f3f90009c00 comp rx=0 tx=0).stop 2024-05-14T15:33:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.876+0000 7f3fa0e54700 1 -- 172.21.15.5:0/2951583384 shutdown_connections 2024-05-14T15:33:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.876+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/2951583384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c106ec0 0x7f3f9c113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.876+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/2951583384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c10c5b0 0x7f3f9c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.876+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/2951583384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106570 0x7f3f9c106980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.876+0000 7f3fa0e54700 1 -- 172.21.15.5:0/2951583384 >> 172.21.15.5:0/2951583384 conn(0x7f3f9c102050 msgr2=0x7f3f9c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:10.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.878+0000 7f3fa0e54700 1 -- 172.21.15.5:0/2951583384 shutdown_connections 2024-05-14T15:33:10.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.879+0000 7f3fa0e54700 1 -- 172.21.15.5:0/2951583384 wait complete. 2024-05-14T15:33:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.881+0000 7f3fa0e54700 1 Processor -- start 2024-05-14T15:33:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.881+0000 7f3fa0e54700 1 -- start start 2024-05-14T15:33:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.882+0000 7f3fa0e54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 0x7f3f9c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.882+0000 7f3fa0e54700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106ec0 0x7f3f9c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.882+0000 7f3fa0e54700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c10c5b0 0x7f3f9c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.882+0000 7f3fa0e54700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1166e0 con 0x7f3f9c10c5b0 2024-05-14T15:33:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.882+0000 7f3fa0e54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c116560 con 0x7f3f9c106570 2024-05-14T15:33:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.882+0000 7f3fa0e54700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c116860 con 0x7f3f9c106ec0 2024-05-14T15:33:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.884+0000 7f3f99d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106ec0 0x7f3f9c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.884+0000 7f3f9a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 0x7f3f9c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.884+0000 7f3f9a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 0x7f3f9c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48462/0 (socket says 172.21.15.5:48462) 2024-05-14T15:33:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.884+0000 7f3f9a59c700 1 -- 172.21.15.5:0/637315722 learned_addr learned my addr 172.21.15.5:0/637315722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.885+0000 7f3f9a59c700 1 -- 172.21.15.5:0/637315722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106ec0 msgr2=0x7f3f9c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.885+0000 7f3f9a59c700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106ec0 0x7f3f9c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.885+0000 7f3f9a59c700 1 -- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c10c5b0 msgr2=0x7f3f9c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.885+0000 7f3f9a59c700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c10c5b0 0x7f3f9c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.885+0000 7f3f9a59c700 1 -- 172.21.15.5:0/637315722 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c1a1d80 con 0x7f3f9c106570 2024-05-14T15:33:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.887+0000 7f3f9a59c700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 0x7f3f9c19ceb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ea30 tx=0x7f3f8c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.888+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00cbd0 con 0x7f3f9c106570 2024-05-14T15:33:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.888+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c0044d0 con 0x7f3f9c106570 2024-05-14T15:33:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.888+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c010430 con 0x7f3f9c106570 2024-05-14T15:33:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.888+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c1a2070 con 0x7f3f9c106570 2024-05-14T15:33:10.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.888+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c1a9990 con 0x7f3f9c106570 2024-05-14T15:33:10.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.890+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c005330 con 0x7f3f9c106570 2024-05-14T15:33:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.894+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3f8c0479e0 con 0x7f3f9c106570 2024-05-14T15:33:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.896+0000 7f3f8b7fe700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3f840b2bc0 0x7f3f840b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.896+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9650+0+0 (secure 0 0 0) 0x7f3f8c0e2bb0 con 0x7f3f9c106570 2024-05-14T15:33:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.897+0000 7f3f99d9b700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3f840b2bc0 0x7f3f840b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.898+0000 7f3f99d9b700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3f840b2bc0 0x7f3f840b50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f900098c0 tx=0x7f3f90019040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:10.899+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f8c0aa140 con 0x7f3f9c106570 2024-05-14T15:33:11.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781364 10.10 MB/sec execute 272 sec latency 92.057 ms 2024-05-14T15:33:11.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.418+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f7c005620 con 0x7f3f9c106570 2024-05-14T15:33:11.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.420+0000 7f3f8b7fe700 1 -- 172.21.15.5:0/637315722 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7f3f8c0a9a40 con 0x7f3f9c106570 2024-05-14T15:33:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:33:11.423 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:33:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.426+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3f840b2bc0 msgr2=0x7f3f840b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.426+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3f840b2bc0 0x7f3f840b50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f900098c0 tx=0x7f3f90019040 comp rx=0 tx=0).stop 2024-05-14T15:33:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 msgr2=0x7f3f9c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 0x7f3f9c19ceb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ea30 tx=0x7f3f8c00ed40 comp rx=0 tx=0).stop 2024-05-14T15:33:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 shutdown_connections 2024-05-14T15:33:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3f840b2bc0 0x7f3f840b50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c10c5b0 0x7f3f9c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f9c106570 0x7f3f9c19ceb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 --2- 172.21.15.5:0/637315722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f9c106ec0 0x7f3f9c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.427+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 >> 172.21.15.5:0/637315722 conn(0x7f3f9c102050 msgr2=0x7f3f9c10d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.429+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 shutdown_connections 2024-05-14T15:33:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:11.429+0000 7f3fa0e54700 1 -- 172.21.15.5:0/637315722 wait complete. 2024-05-14T15:33:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:33:12.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:11 smithi005 ceph-mon[31559]: pgmap v259: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 89 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-14T15:33:12.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:11 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/637315722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:33:12.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 782293 10.07 MB/sec execute 273 sec latency 29.859 ms 2024-05-14T15:33:12.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:33:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:11 smithi144 ceph-mon[35650]: pgmap v259: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 89 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-14T15:33:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:11 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/637315722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:33:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:11 smithi193 ceph-mon[36001]: pgmap v259: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 89 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2024-05-14T15:33:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:11 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/637315722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:33:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:13.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 784777 10.10 MB/sec execute 274 sec latency 20.338 ms 2024-05-14T15:33:13.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:12 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:33:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:12 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:33:13.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:12 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:33:14.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 786200 10.08 MB/sec execute 275 sec latency 19.436 ms 2024-05-14T15:33:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:13 smithi144 ceph-mon[35650]: pgmap v260: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 294 op/s 2024-05-14T15:33:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:13 smithi144 ceph-mon[35650]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T15:33:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:13 smithi193 ceph-mon[36001]: pgmap v260: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 294 op/s 2024-05-14T15:33:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:13 smithi193 ceph-mon[36001]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T15:33:14.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:13 smithi005 ceph-mon[31559]: pgmap v260: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 294 op/s 2024-05-14T15:33:14.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:13 smithi005 ceph-mon[31559]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T15:33:15.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 788426 10.10 MB/sec execute 276 sec latency 46.795 ms 2024-05-14T15:33:15.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:14 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:33:15.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:14 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:33:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:14 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:33:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:14 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:33:15.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:14 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:33:15.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:14 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:33:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 -- 172.21.15.5:0/2570123363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 msgr2=0x7fc02810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2570123363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02810e3a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc018009990 tx=0x7fc018009ca0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 -- 172.21.15.5:0/2570123363 shutdown_connections 2024-05-14T15:33:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2570123363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc02810e930 0x7fc028110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2570123363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc028105500 0x7fc028105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2570123363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02810e3a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 -- 172.21.15.5:0/2570123363 >> 172.21.15.5:0/2570123363 conn(0x7fc0280fedf0 msgr2=0x7fc028101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 -- 172.21.15.5:0/2570123363 shutdown_connections 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.639+0000 7fc02fa97700 1 -- 172.21.15.5:0/2570123363 wait complete. 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 Processor -- start 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 -- start start 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc028105500 0x7fc02819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02810e930 0x7fc02819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc028114450 con 0x7fc028105e50 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281142d0 con 0x7fc02810e930 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02fa97700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281145d0 con 0x7fc028105500 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02d032700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02d032700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50768/0 (socket says 172.21.15.5:50768) 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02d032700 1 -- 172.21.15.5:0/2725987177 learned_addr learned my addr 172.21.15.5:0/2725987177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:15.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02d833700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc028105500 0x7fc02819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.640+0000 7fc02e034700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02810e930 0x7fc02819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02d032700 1 -- 172.21.15.5:0/2725987177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc028105500 msgr2=0x7fc02819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02d032700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc028105500 0x7fc02819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02d032700 1 -- 172.21.15.5:0/2725987177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02810e930 msgr2=0x7fc02819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02d032700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02810e930 0x7fc02819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02d032700 1 -- 172.21.15.5:0/2725987177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02819fcd0 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02d032700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02819b3a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc01800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc01effd700 1 -- 172.21.15.5:0/2725987177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018027070 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc01effd700 1 -- 172.21.15.5:0/2725987177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018022470 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02819ff60 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc01effd700 1 -- 172.21.15.5:0/2725987177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018017650 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.641+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0281a0440 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.644+0000 7fc01effd700 1 -- 172.21.15.5:0/2725987177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc01802a070 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.645+0000 7fc01effd700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc0140b29f0 0x7fc0140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.645+0000 7fc01effd700 1 -- 172.21.15.5:0/2725987177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc0180e3710 con 0x7fc028105e50 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.645+0000 7fc02d833700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc0140b29f0 0x7fc0140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.645+0000 7fc03e4abb80 1 Processor -- start 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.645+0000 7fc02d833700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc0140b29f0 0x7fc0140b4f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc02806d150 tx=0x7fc02400b040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.645+0000 7fc03e4abb80 1 -- start start 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ede720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7edec60 0x5560c7ee2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7ee3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560c7e00860 con 0x5560c7ee3470 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560c7d9f7c0 con 0x5560c7edec60 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560c7dfc820 con 0x5560c7d60e40 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7ee3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02d833700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ede720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7ee3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50770/0 (socket says 172.21.15.5:50770) 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 -- 172.21.15.5:0/2586298438 learned_addr learned my addr 172.21.15.5:0/2586298438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02d032700 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7edec60 0x5560c7ee2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 -- 172.21.15.5:0/2586298438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 msgr2=0x5560c7ede720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ede720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 -- 172.21.15.5:0/2586298438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7edec60 msgr2=0x5560c7ee2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7edec60 0x5560c7ee2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 -- 172.21.15.5:0/2586298438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7d4d8d0 con 0x5560c7ee3470 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02d833700 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ede720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc02e034700 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7ee3880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc020009cb0 tx=0x7fc020011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ee2030a57bce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc013fff700 1 -- 172.21.15.5:0/2586298438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020011d80 con 0x5560c7ee3470 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc013fff700 1 -- 172.21.15.5:0/2586298438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0200044e0 con 0x5560c7ee3470 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.646+0000 7fc013fff700 1 -- 172.21.15.5:0/2586298438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020019660 con 0x5560c7ee3470 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 -- 172.21.15.5:0/2586298438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 msgr2=0x5560c7ee3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7ee3880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc020009cb0 tx=0x7fc020011440 comp rx=0 tx=0).stop 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 -- 172.21.15.5:0/2586298438 shutdown_connections 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7ee3880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ede720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/2586298438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7edec60 0x5560c7ee2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.647+0000 7fc03e4abb80 1 -- 172.21.15.5:0/2586298438 >> 172.21.15.5:0/2586298438 conn(0x5560c7da8600 msgr2=0x5560c7ed94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 -- 172.21.15.5:0/2586298438 shutdown_connections 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 -- 172.21.15.5:0/2586298438 wait complete. 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 -- start start 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 Processor -- start 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ee80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.648+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7ee3470 0x5560c7ee8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560c7eeb420 con 0x5560c7edec60 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560c7d66c30 con 0x5560c7ee3470 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560c7eeb5a0 con 0x5560c7d60e40 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc03e4abb80 10 client.0 _ll_register_callbacks cb 0x5560c7d98920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d032700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d032700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50784/0 (socket says 172.21.15.5:50784) 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d032700 1 -- 172.21.15.5:0/4212050343 learned_addr learned my addr 172.21.15.5:0/4212050343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ee80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02e034700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7ee3470 0x5560c7ee8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 -- 172.21.15.5:0/4212050343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7ee3470 msgr2=0x5560c7ee8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7ee3470 0x5560c7ee8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 -- 172.21.15.5:0/4212050343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 msgr2=0x5560c7ee85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 -- 172.21.15.5:0/4212050343 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c7d9c690 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d032700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc02d833700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ee80a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc024009fd0 tx=0x7fc024004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489a4743ef11b6ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.649+0000 7fc0117fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.650+0000 7fc0117fa700 1 -- 172.21.15.5:0/4212050343 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024027030 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.650+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0200044e0 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.650+0000 7fc0117fa700 1 -- 172.21.15.5:0/4212050343 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc024015d60 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.650+0000 7fc0117fa700 1 -- 172.21.15.5:0/4212050343 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc024033050 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.650+0000 7fc03e4abb80 10 client.26953 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.650+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560c7efa400 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc0117fa700 1 -- 172.21.15.5:0/4212050343 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0200044e0 con 0x5560c7d60e40 2024-05-14T15:33:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc0117fa700 1 -- 172.21.15.5:0/4212050343 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc024020050 con 0x5560c7d60e40 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc03e4abb80 10 client.26953 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc03e4abb80 10 client.26953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc03e4abb80 20 client.26953 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7efc990 0x5560c7efedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc03e4abb80 4 client.26953 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c7e0e240 con 0x5560c7efc990 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc02d032700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7efc990 0x5560c7efedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc02d032700 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7efc990 0x5560c7efedb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99aea27749a0863a server_cookie=1fc687b046d2ecc4 in_seq=0 out_seq=0 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.651+0000 7fc0117fa700 10 client.26953 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc0117fa700 1 -- 172.21.15.5:0/4212050343 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560c7e0e240 con 0x5560c7efc990 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc0117fa700 10 client.26953 handle_command_reply: tid=0 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 client.26953 shutdown 2024-05-14T15:33:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 msgr2=0x5560c7ee80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ee80a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc024009fd0 tx=0x7fc024004ad0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 shutdown_connections 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7efc990 0x5560c7efedb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7ee3470 0x5560c7ee8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7d60e40 0x5560c7ee80a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc0117fa700 0 client.26953 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/4212050343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.652+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 >> 172.21.15.5:0/4212050343 conn(0x5560c7da8600 msgr2=0x5560c7edd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.653+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 shutdown_connections 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.653+0000 7fc03e4abb80 1 -- 172.21.15.5:0/4212050343 wait complete. 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.653+0000 7fc03e4abb80 20 client.26953 trim_cache size 0 max 16384 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.668+0000 7fc03e4abb80 1 Processor -- start 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.668+0000 7fc03e4abb80 1 -- start start 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.668+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ee2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.668+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7ee3470 0x5560c7ef8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281b0d70 con 0x5560c7edec60 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281ae600 con 0x5560c7d60e40 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281ac470 con 0x5560c7ee3470 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50798/0 (socket says 172.21.15.5:50798) 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 -- 172.21.15.5:0/3080629861 learned_addr learned my addr 172.21.15.5:0/3080629861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d833700 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ee2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02e034700 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7ee3470 0x5560c7ef8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 -- 172.21.15.5:0/3080629861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7ee3470 msgr2=0x5560c7ef8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7ee3470 0x5560c7ef8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 -- 172.21.15.5:0/3080629861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 msgr2=0x5560c7ee2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ee2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 -- 172.21.15.5:0/3080629861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7ee8ea0 con 0x5560c7edec60 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc02d032700 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee8960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b6c0 tx=0x7fc01801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b156dc9e360362c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc0137fe700 1 -- 172.21.15.5:0/3080629861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180c3070 con 0x5560c7edec60 2024-05-14T15:33:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc0137fe700 1 -- 172.21.15.5:0/3080629861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0180aea20 con 0x5560c7edec60 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.669+0000 7fc0137fe700 1 -- 172.21.15.5:0/3080629861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018022b20 con 0x5560c7edec60 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 -- 172.21.15.5:0/3080629861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 msgr2=0x5560c7ee8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee8960 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc01800b6c0 tx=0x7fc01801a040 comp rx=0 tx=0).stop 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 -- 172.21.15.5:0/3080629861 shutdown_connections 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7ee3470 0x5560c7ef8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ee2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/3080629861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7edec60 0x5560c7ee8960 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 -- 172.21.15.5:0/3080629861 >> 172.21.15.5:0/3080629861 conn(0x5560c7da8600 msgr2=0x5560c7ee26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 -- 172.21.15.5:0/3080629861 shutdown_connections 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.670+0000 7fc03e4abb80 1 -- 172.21.15.5:0/3080629861 wait complete. 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 -- start start 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 Processor -- start 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ef2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7edec60 0x5560c7ef27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7f5f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281acac0 con 0x5560c7ee3470 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281adf30 con 0x5560c7d60e40 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281ad190 con 0x5560c7edec60 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc03e4abb80 10 client.0 _ll_register_callbacks cb 0x5560c7d98920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc02d833700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ef2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc02e034700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7f5f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc02d833700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ef2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48500/0 (socket says 172.21.15.5:48500) 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.671+0000 7fc02d833700 1 -- 172.21.15.5:0/800768482 learned_addr learned my addr 172.21.15.5:0/800768482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d032700 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7edec60 0x5560c7ef27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d833700 1 -- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7edec60 msgr2=0x5560c7ef27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d833700 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7edec60 0x5560c7ef27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d833700 1 -- 172.21.15.5:0/800768482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 msgr2=0x5560c7f5f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d833700 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7f5f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d833700 1 -- 172.21.15.5:0/800768482 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c7f5f6f0 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc02d833700 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ef2290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0240047f0 tx=0x7fc024004df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03a18e02b89994c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc0117fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc0117fa700 1 -- 172.21.15.5:0/800768482 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024027040 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0180aea20 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc0117fa700 1 -- 172.21.15.5:0/800768482 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc024023750 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc0117fa700 1 -- 172.21.15.5:0/800768482 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc024042050 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc03e4abb80 10 client.26990 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.672+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560c7f669f0 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc0117fa700 1 -- 172.21.15.5:0/800768482 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0180aea20 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc0117fa700 1 -- 172.21.15.5:0/800768482 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fc024020040 con 0x5560c7d60e40 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc03e4abb80 10 client.26990 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc03e4abb80 10 client.26990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc03e4abb80 20 client.26990 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7f67180 0x5560c7f695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc03e4abb80 4 client.26990 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560c7e082e0 con 0x5560c7f67180 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.673+0000 7fc02d032700 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7f67180 0x5560c7f695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc0117fa700 10 client.26990 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc02d032700 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7f67180 0x5560c7f695a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5ed202ace04c9d server_cookie=c9fd6f8613572e6a in_seq=0 out_seq=0 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc0117fa700 1 -- 172.21.15.5:0/800768482 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560c7e082e0 con 0x5560c7f67180 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc0117fa700 10 client.26990 handle_command_reply: tid=0 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 client.26990 shutdown 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 msgr2=0x5560c7ef2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ef2290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0240047f0 tx=0x7fc024004df0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 shutdown_connections 2024-05-14T15:33:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560c7ee3470 0x5560c7f5f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5560c7f67180 0x5560c7f695a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560c7d60e40 0x5560c7ef2290 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 --2- 172.21.15.5:0/800768482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560c7edec60 0x5560c7ef27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc0117fa700 0 client.26990 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 >> 172.21.15.5:0/800768482 conn(0x5560c7da8600 msgr2=0x5560c7ed8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 shutdown_connections 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 1 -- 172.21.15.5:0/800768482 wait complete. 2024-05-14T15:33:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.674+0000 7fc03e4abb80 20 client.26990 trim_cache size 0 max 16384 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc0140b29f0 msgr2=0x7fc0140b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc0140b29f0 0x7fc0140b4f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc02806d150 tx=0x7fc02400b040 comp rx=0 tx=0).stop 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 msgr2=0x7fc02819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02819b3a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc01800bfd0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 shutdown_connections 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc0140b29f0 0x7fc0140b4f00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02810e930 0x7fc02819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc028105500 0x7fc02819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 --2- 172.21.15.5:0/2725987177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc028105e50 0x7fc02819b3a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 >> 172.21.15.5:0/2725987177 conn(0x7fc0280fedf0 msgr2=0x7fc0280ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 shutdown_connections 2024-05-14T15:33:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:14.675+0000 7fc02fa97700 1 -- 172.21.15.5:0/2725987177 wait complete. 2024-05-14T15:33:16.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789461 10.07 MB/sec execute 277 sec latency 32.974 ms 2024-05-14T15:33:16.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:15 smithi005 ceph-mon[31559]: pgmap v262: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-05-14T15:33:16.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:15 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:33:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:15 smithi144 ceph-mon[35650]: pgmap v262: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-05-14T15:33:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:15 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:33:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:15 smithi193 ceph-mon[36001]: pgmap v262: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 68 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-05-14T15:33:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:15 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:33:17.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 791604 10.08 MB/sec execute 278 sec latency 34.123 ms 2024-05-14T15:33:17.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:33:17.568 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:18.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 792893 10.06 MB/sec execute 279 sec latency 31.932 ms 2024-05-14T15:33:18.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:17 smithi005 ceph-mon[31559]: pgmap v263: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 70 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-05-14T15:33:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:17 smithi144 ceph-mon[35650]: pgmap v263: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 70 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-05-14T15:33:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:17 smithi193 ceph-mon[36001]: pgmap v263: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 70 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-05-14T15:33:19.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 795281 10.08 MB/sec execute 280 sec latency 56.111 ms 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 -- 172.21.15.5:0/3109005703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0108740 msgr2=0x7f08c01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 --2- 172.21.15.5:0/3109005703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0108740 0x7f08c01100a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b0d0 tx=0x7f08b800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 -- 172.21.15.5:0/3109005703 shutdown_connections 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 --2- 172.21.15.5:0/3109005703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0108740 0x7f08c01100a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 --2- 172.21.15.5:0/3109005703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c01034c0 0x7f08c01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 --2- 172.21.15.5:0/3109005703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08c0107df0 0x7f08c0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 -- 172.21.15.5:0/3109005703 >> 172.21.15.5:0/3109005703 conn(0x7f08c00fedc0 msgr2=0x7f08c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 -- 172.21.15.5:0/3109005703 shutdown_connections 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.588+0000 7f08c8439700 1 -- 172.21.15.5:0/3109005703 wait complete. 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 Processor -- start 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 -- start start 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 0x7f08c019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0107df0 0x7f08c019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08c0108740 0x7f08c0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0113370 con 0x7f08c01034c0 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01131f0 con 0x7f08c0108740 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c8439700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01134f0 con 0x7f08c0107df0 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 0x7f08c019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 0x7f08c019f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58666/0 (socket says 172.21.15.5:58666) 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c61d5700 1 -- 172.21.15.5:0/3505280712 learned_addr learned my addr 172.21.15.5:0/3505280712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c69d6700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08c0108740 0x7f08c0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.589+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0107df0 0x7f08c019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c61d5700 1 -- 172.21.15.5:0/3505280712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0107df0 msgr2=0x7f08c019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0107df0 0x7f08c019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c61d5700 1 -- 172.21.15.5:0/3505280712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08c0108740 msgr2=0x7f08c0199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08c0108740 0x7f08c0199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c61d5700 1 -- 172.21.15.5:0/3505280712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c019a1b0 con 0x7f08c01034c0 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 0x7f08c019f890 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009f60 tx=0x7f08bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08b77fe700 1 -- 172.21.15.5:0/3505280712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00cbd0 con 0x7f08c01034c0 2024-05-14T15:33:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c019a4a0 con 0x7f08c01034c0 2024-05-14T15:33:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08b77fe700 1 -- 172.21.15.5:0/3505280712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08bc0044d0 con 0x7f08c01034c0 2024-05-14T15:33:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08b77fe700 1 -- 172.21.15.5:0/3505280712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0115d0 con 0x7f08c01034c0 2024-05-14T15:33:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.590+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c006b610 con 0x7f08c01034c0 2024-05-14T15:33:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.593+0000 7f08b77fe700 1 -- 172.21.15.5:0/3505280712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f08bc020020 con 0x7f08c01034c0 2024-05-14T15:33:19.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.594+0000 7f08b77fe700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f08ac0b29f0 0x7f08ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.594+0000 7f08d6e4db80 1 Processor -- start 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.594+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f08ac0b29f0 0x7f08ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08b77fe700 1 -- 172.21.15.5:0/3505280712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f08bc0e4830 con 0x7f08c01034c0 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 -- start start 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f08ac0b29f0 0x7f08ac0b4f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08c0106b70 tx=0x7f08b0009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b69720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5b6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573a5a8b860 con 0x5573a59ebe40 2024-05-14T15:33:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5573a5a2a7c0 con 0x5573a5b69c60 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5573a5a87820 con 0x5573a5b6e470 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b69720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c69d6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5b6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b69720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58678/0 (socket says 172.21.15.5:58678) 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c61d5700 1 -- 172.21.15.5:0/513193197 learned_addr learned my addr 172.21.15.5:0/513193197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c59d4700 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c69d6700 1 -- 172.21.15.5:0/513193197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 msgr2=0x5573a5b6df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c69d6700 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b6df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c69d6700 1 -- 172.21.15.5:0/513193197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 msgr2=0x5573a5b69720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.595+0000 7f08c69d6700 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b69720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08c69d6700 1 -- 172.21.15.5:0/513193197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a59d88d0 con 0x5573a5b6e470 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08c61d5700 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b69720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08c59d4700 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b6df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08c69d6700 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5b6e880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b770 tx=0x7f08b80098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eede6ebcdf2034 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08b4d47700 1 -- 172.21.15.5:0/513193197 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80092e0 con 0x5573a5b6e470 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08b4d47700 1 -- 172.21.15.5:0/513193197 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b8020070 con 0x5573a5b6e470 2024-05-14T15:33:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.596+0000 7f08b4d47700 1 -- 172.21.15.5:0/513193197 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80052d0 con 0x5573a5b6e470 2024-05-14T15:33:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 -- 172.21.15.5:0/513193197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 msgr2=0x5573a5b6e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5b6e880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b770 tx=0x7f08b80098c0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 -- 172.21.15.5:0/513193197 shutdown_connections 2024-05-14T15:33:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5b6e880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b69720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/513193197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b6df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 -- 172.21.15.5:0/513193197 >> 172.21.15.5:0/513193197 conn(0x5573a5a33600 msgr2=0x5573a5b644b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 -- 172.21.15.5:0/513193197 shutdown_connections 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.597+0000 7f08d6e4db80 1 -- 172.21.15.5:0/513193197 wait complete. 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 -- start start 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 Processor -- start 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b73150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573a5b76300 con 0x5573a5b6e470 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5573a59f1c30 con 0x5573a5b69c60 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5573a5b76480 con 0x5573a59ebe40 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08d6e4db80 10 client.0 _ll_register_callbacks cb 0x5573a5a23920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b73bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58684/0 (socket says 172.21.15.5:58684) 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c59d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 -- 172.21.15.5:0/2554340644 learned_addr learned my addr 172.21.15.5:0/2554340644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c61d5700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b73150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 -- 172.21.15.5:0/2554340644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 msgr2=0x5573a5b73150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b73150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 -- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 msgr2=0x5573a5b73690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b73690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.598+0000 7f08c69d6700 1 -- 172.21.15.5:0/2554340644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a5a27690 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08c61d5700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b73150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08c69d6700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b73bd0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b420 tx=0x7f08b8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7026ec68d63bcca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08a9ffb700 1 -- 172.21.15.5:0/2554340644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80050c0 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08a9ffb700 1 -- 172.21.15.5:0/2554340644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f08b8005260 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b8020070 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08a9ffb700 1 -- 172.21.15.5:0/2554340644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08b8036050 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08d6e4db80 10 client.17382 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573a5b80f30 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.599+0000 7f08a9ffb700 1 -- 172.21.15.5:0/2554340644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b8020070 con 0x5573a5b6e470 2024-05-14T15:33:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08a9ffb700 1 -- 172.21.15.5:0/2554340644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f08b8011070 con 0x5573a5b6e470 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08d6e4db80 10 client.17382 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08d6e4db80 10 client.17382 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08d6e4db80 20 client.17382 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5b83400 0x5573a5b85820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08d6e4db80 4 client.17382 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a5a99240 con 0x5573a5b83400 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08c61d5700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5b83400 0x5573a5b85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08c61d5700 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5b83400 0x5573a5b85820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=380eb400ba5d4817 server_cookie=5345f5a4deedccb6 in_seq=0 out_seq=0 2024-05-14T15:33:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.600+0000 7f08a9ffb700 10 client.17382 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08a9ffb700 1 -- 172.21.15.5:0/2554340644 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5573a5a99240 con 0x5573a5b83400 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08a9ffb700 10 client.17382 handle_command_reply: tid=0 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 client.17382 shutdown 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 msgr2=0x5573a5b73bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b73bd0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b420 tx=0x7f08b8009590 comp rx=0 tx=0).stop 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 shutdown_connections 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5b83400 0x5573a5b85820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b73bd0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b73150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/2554340644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b73690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 >> 172.21.15.5:0/2554340644 conn(0x5573a5a33600 msgr2=0x5573a5b68450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08a9ffb700 0 client.17382 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:33:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 shutdown_connections 2024-05-14T15:33:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 1 -- 172.21.15.5:0/2554340644 wait complete. 2024-05-14T15:33:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.601+0000 7f08d6e4db80 20 client.17382 trim_cache size 0 max 16384 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 Processor -- start 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 -- start start 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b72a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b88430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01b0df0 con 0x5573a5b6e470 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ae680 con 0x5573a5b69c60 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.617+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ac4f0 con 0x5573a59ebe40 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b88430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b72a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b88430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58690/0 (socket says 172.21.15.5:58690) 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 -- 172.21.15.5:0/3756884013 learned_addr learned my addr 172.21.15.5:0/3756884013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b72a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57042/0 (socket says 172.21.15.5:57042) 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 -- 172.21.15.5:0/3756884013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 msgr2=0x5573a5b72a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b72a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 -- 172.21.15.5:0/3756884013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 msgr2=0x5573a5b642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 -- 172.21.15.5:0/3756884013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a5b60ae0 con 0x5573a5b6e470 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b72a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08c69d6700 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b88430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b770 tx=0x7f08b80097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cfc4351d778a13 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08abfff700 1 -- 172.21.15.5:0/3756884013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8005e60 con 0x5573a5b6e470 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08abfff700 1 -- 172.21.15.5:0/3756884013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b8020070 con 0x5573a5b6e470 2024-05-14T15:33:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.618+0000 7f08abfff700 1 -- 172.21.15.5:0/3756884013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8005410 con 0x5573a5b6e470 2024-05-14T15:33:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3756884013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 msgr2=0x5573a5b88430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b88430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b770 tx=0x7f08b80097f0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3756884013 shutdown_connections 2024-05-14T15:33:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a5b6e470 0x5573a5b88430 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a59ebe40 0x5573a5b72a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3756884013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3756884013 >> 172.21.15.5:0/3756884013 conn(0x5573a5a33600 msgr2=0x5573a5b63be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3756884013 shutdown_connections 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3756884013 wait complete. 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.619+0000 7f08d6e4db80 1 -- start start 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 Processor -- start 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b87870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b87db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5be5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01acb40 con 0x5573a59ebe40 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01adfb0 con 0x5573a5b69c60 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b87870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ad210 con 0x5573a5b6e470 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b87870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58706/0 (socket says 172.21.15.5:58706) 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 -- 172.21.15.5:0/3920056243 learned_addr learned my addr 172.21.15.5:0/3920056243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08d6e4db80 10 client.0 _ll_register_callbacks cb 0x5573a5a23920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b87db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c69d6700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5be5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 -- 172.21.15.5:0/3920056243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 msgr2=0x5573a5be5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5be5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 -- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 msgr2=0x5573a5b87db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b87db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.620+0000 7f08c61d5700 1 -- 172.21.15.5:0/3920056243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a5be5e90 con 0x5573a59ebe40 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08c61d5700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b87870 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0a7890 tx=0x7f08bc093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980184f3377b705a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08a9ffb700 1 -- 172.21.15.5:0/3920056243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc020520 con 0x5573a59ebe40 2024-05-14T15:33:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08a9ffb700 1 -- 172.21.15.5:0/3920056243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f08bc0aba70 con 0x5573a59ebe40 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b8020070 con 0x5573a59ebe40 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08a9ffb700 1 -- 172.21.15.5:0/3920056243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08bc0abf10 con 0x5573a59ebe40 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 10 client.17388 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573a5bed190 con 0x5573a59ebe40 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08a9ffb700 1 -- 172.21.15.5:0/3920056243 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08bc0c8020 con 0x5573a59ebe40 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08a9ffb700 1 -- 172.21.15.5:0/3920056243 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f08bc0b41e0 con 0x5573a59ebe40 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 10 client.17388 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 10 client.17388 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 20 client.17388 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.621+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5bed870 0x5573a5befc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 4 client.17388 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573a5a932e0 con 0x5573a5bed870 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5bed870 0x5573a5befc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08c59d4700 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5bed870 0x5573a5befc90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=433e1d6c3905177 server_cookie=50a6654275eba60c in_seq=0 out_seq=0 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08a9ffb700 10 client.17388 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08a9ffb700 1 -- 172.21.15.5:0/3920056243 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573a5a932e0 con 0x5573a5bed870 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08a9ffb700 10 client.17388 handle_command_reply: tid=0 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 client.17388 shutdown 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 msgr2=0x5573a5b87870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b87870 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0a7890 tx=0x7f08bc093040 comp rx=0 tx=0).stop 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 shutdown_connections 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5573a5bed870 0x5573a5befc90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5573a5b6e470 0x5573a5be5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573a59ebe40 0x5573a5b87870 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 --2- 172.21.15.5:0/3920056243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a5b69c60 0x5573a5b87db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 >> 172.21.15.5:0/3920056243 conn(0x5573a5a33600 msgr2=0x5573a5b68450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.622+0000 7f08a9ffb700 0 client.17388 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:33:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 shutdown_connections 2024-05-14T15:33:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08d6e4db80 1 -- 172.21.15.5:0/3920056243 wait complete. 2024-05-14T15:33:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08d6e4db80 20 client.17388 trim_cache size 0 max 16384 2024-05-14T15:33:19.630 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f08ac0b29f0 msgr2=0x7f08ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f08ac0b29f0 0x7f08ac0b4f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08c0106b70 tx=0x7f08b0009260 comp rx=0 tx=0).stop 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 msgr2=0x7f08c019f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 0x7f08c019f890 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009f60 tx=0x7f08bc009f90 comp rx=0 tx=0).stop 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 shutdown_connections 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f08ac0b29f0 0x7f08ac0b4f00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08c0108740 0x7f08c0199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c01034c0 0x7f08c019f890 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 --2- 172.21.15.5:0/3505280712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0107df0 0x7f08c019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 >> 172.21.15.5:0/3505280712 conn(0x7f08c00fedc0 msgr2=0x7f08c0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.623+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 shutdown_connections 2024-05-14T15:33:19.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:19.624+0000 7f08c8439700 1 -- 172.21.15.5:0/3505280712 wait complete. 2024-05-14T15:33:20.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 796731 10.07 MB/sec execute 281 sec latency 40.775 ms 2024-05-14T15:33:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:19 smithi144 ceph-mon[35650]: pgmap v264: 129 pgs: 129 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-05-14T15:33:20.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:19 smithi193 ceph-mon[36001]: pgmap v264: 129 pgs: 129 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-05-14T15:33:20.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:19 smithi005 ceph-mon[31559]: pgmap v264: 129 pgs: 129 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-05-14T15:33:20.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:33:20.670 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:21.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 798689 10.07 MB/sec execute 282 sec latency 67.569 ms 2024-05-14T15:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 -- 172.21.15.5:0/1045299818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 msgr2=0x7ff1c810f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 --2- 172.21.15.5:0/1045299818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c810f6e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009900 tx=0x7ff1c002a410 comp rx=0 tx=0).stop 2024-05-14T15:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 -- 172.21.15.5:0/1045299818 shutdown_connections 2024-05-14T15:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 --2- 172.21.15.5:0/1045299818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c810f6e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 --2- 172.21.15.5:0/1045299818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c8109060 0x7ff1c8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 --2- 172.21.15.5:0/1045299818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1c8108bc0 0x7ff1c8102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 -- 172.21.15.5:0/1045299818 >> 172.21.15.5:0/1045299818 conn(0x7ff1c80fe390 msgr2=0x7ff1c81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 -- 172.21.15.5:0/1045299818 shutdown_connections 2024-05-14T15:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.698+0000 7ff1d037d640 1 -- 172.21.15.5:0/1045299818 wait complete. 2024-05-14T15:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 Processor -- start 2024-05-14T15:33:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 -- start start 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c80fe870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1c8108bc0 0x7ff1c80fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c8109060 0x7ff1c80ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8112e20 con 0x7ff1c8103310 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8112ca0 con 0x7ff1c8109060 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1d037d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8112fa0 con 0x7ff1c8108bc0 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c80fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c80fe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58726/0 (socket says 172.21.15.5:58726) 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 -- 172.21.15.5:0/3180557165 learned_addr learned my addr 172.21.15.5:0/3180557165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce91a640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c8109060 0x7ff1c80ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1cd918640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1c8108bc0 0x7ff1c80fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 -- 172.21.15.5:0/3180557165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1c8108bc0 msgr2=0x7ff1c80fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1c8108bc0 0x7ff1c80fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 -- 172.21.15.5:0/3180557165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c8109060 msgr2=0x7ff1c80ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c8109060 0x7ff1c80ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.699+0000 7ff1ce119640 1 -- 172.21.15.5:0/3180557165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c81a8bf0 con 0x7ff1c8103310 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.700+0000 7ff1ce119640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c80fe870 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400ebc0 tx=0x7ff1c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.700+0000 7ff1bf7fe640 1 -- 172.21.15.5:0/3180557165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004d20 con 0x7ff1c8103310 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.700+0000 7ff1bf7fe640 1 -- 172.21.15.5:0/3180557165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c40054d0 con 0x7ff1c8103310 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.700+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c81a8ee0 con 0x7ff1c8103310 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.700+0000 7ff1bf7fe640 1 -- 172.21.15.5:0/3180557165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4010720 con 0x7ff1c8103310 2024-05-14T15:33:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.700+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c81a9340 con 0x7ff1c8103310 2024-05-14T15:33:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.703+0000 7ff1bf7fe640 1 -- 172.21.15.5:0/3180557165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff1c4005020 con 0x7ff1c8103310 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.703+0000 7ff1d7235740 1 Processor -- start 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.703+0000 7ff1d7235740 1 -- start start 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1d7235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f4a8125d0 con 0x557f4a4e32f0 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1d7235740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557f4a7cb890 con 0x557f4a50c1b0 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1d7235740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f4a80e590 con 0x557f4a4eb4d0 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1ce91a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1ce91a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44478/0 (socket says 172.21.15.5:44478) 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1ce91a640 1 -- 172.21.15.5:0/250332767 learned_addr learned my addr 172.21.15.5:0/250332767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1bf7fe640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff1b40b2950 0x7ff1b40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1ce91a640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff1b40b2950 0x7ff1b40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 -- 172.21.15.5:0/250332767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 msgr2=0x557f4a8acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 -- 172.21.15.5:0/250332767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 msgr2=0x557f4a8ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 -- 172.21.15.5:0/250332767 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f4a524210 con 0x557f4a4eb4d0 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1bf7fe640 1 -- 172.21.15.5:0/3180557165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff1c40e33e0 con 0x7ff1c8103310 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.704+0000 7ff1cd918640 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8ad660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8109490 tx=0x7ff1b8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66edbe3b69132186 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.705+0000 7ff1bcbfe640 1 -- 172.21.15.5:0/250332767 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8011d70 con 0x557f4a4eb4d0 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.705+0000 7ff1ce91a640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff1b40b2950 0x7ff1b40b4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0031440 tx=0x7ff1c002a410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.705+0000 7ff1bcbfe640 1 -- 172.21.15.5:0/250332767 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b8016070 con 0x557f4a4eb4d0 2024-05-14T15:33:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.705+0000 7ff1bcbfe640 1 -- 172.21.15.5:0/250332767 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8004a00 con 0x557f4a4eb4d0 2024-05-14T15:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 -- 172.21.15.5:0/250332767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 msgr2=0x557f4a8ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8ad660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8109490 tx=0x7ff1b8009f50 comp rx=0 tx=0).stop 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 -- 172.21.15.5:0/250332767 shutdown_connections 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8ad660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 --2- 172.21.15.5:0/250332767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 -- 172.21.15.5:0/250332767 >> 172.21.15.5:0/250332767 conn(0x557f4a500300 msgr2=0x557f4a8a9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 -- 172.21.15.5:0/250332767 shutdown_connections 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 -- 172.21.15.5:0/250332767 wait complete. 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 -- start start 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.706+0000 7ff1d7235740 1 Processor -- start 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 0x557f4a8b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f4a8b9d10 con 0x557f4a4eb4d0 2024-05-14T15:33:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557f4a7cf370 con 0x557f4a50c1b0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f4a8b9e90 con 0x557f4a4e32f0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1d7235740 10 client.0 _ll_register_callbacks cb 0x557f4a523a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 0x557f4a8b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58742/0 (socket says 172.21.15.5:58742) 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 0x557f4a8b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57082/0 (socket says 172.21.15.5:57082) 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 -- 172.21.15.5:0/1141123953 learned_addr learned my addr 172.21.15.5:0/1141123953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1ce91a640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 -- 172.21.15.5:0/1141123953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 msgr2=0x557f4a8b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 0x557f4a8b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 -- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 msgr2=0x557f4a8cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 -- 172.21.15.5:0/1141123953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f4a5f0620 con 0x557f4a4eb4d0 2024-05-14T15:33:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1ce119640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 0x557f4a8b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b7710 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8004280 tx=0x7ff1b80042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e8d8aefe338801 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.707+0000 7ff1b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1141123953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8029060 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1141123953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff1b8003a00 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8016070 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1141123953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1b8030050 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 10 client.17400 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557f4a8ce860 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1141123953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b8016070 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1141123953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff1b8023070 con 0x557f4a4eb4d0 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 10 client.17400 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 20 client.17400 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4cad80 0x557f4a8d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 4 client.17400 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.708+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f4a8201e0 con 0x557f4a4cad80 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.709+0000 7ff1ce119640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4cad80 0x557f4a8d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.709+0000 7ff1b1ffb640 10 client.17400 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.709+0000 7ff1ce119640 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4cad80 0x557f4a8d1cd0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377f5280b2069884 server_cookie=7d6d02a809e08576 in_seq=0 out_seq=0 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1141123953 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f4a8201e0 con 0x557f4a4cad80 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1b1ffb640 10 client.17400 handle_command_reply: tid=0 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 client.17400 shutdown 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 msgr2=0x557f4a8b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b7710 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8004280 tx=0x7ff1b80042b0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 shutdown_connections 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4cad80 0x557f4a8d1cd0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1b1ffb640 0 client.17400 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b7710 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1141123953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4e32f0 0x557f4a8b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 >> 172.21.15.5:0/1141123953 conn(0x557f4a500300 msgr2=0x557f4a8ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 shutdown_connections 2024-05-14T15:33:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 1 -- 172.21.15.5:0/1141123953 wait complete. 2024-05-14T15:33:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.710+0000 7ff1d7235740 20 client.17400 trim_cache size 0 max 16384 2024-05-14T15:33:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.727+0000 7ff1d7235740 1 Processor -- start 2024-05-14T15:33:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 -- start start 2024-05-14T15:33:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4e32f0 0x557f4a8b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81b1ac0 con 0x557f4a50c1b0 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81af390 con 0x557f4a4e32f0 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1d7235740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81ad200 con 0x557f4a4eb4d0 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1ce91a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1ce91a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58750/0 (socket says 172.21.15.5:58750) 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57098/0 (socket says 172.21.15.5:57098) 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1ce91a640 1 -- 172.21.15.5:0/3564445576 learned_addr learned my addr 172.21.15.5:0/3564445576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4e32f0 0x557f4a8b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 -- 172.21.15.5:0/3564445576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4e32f0 msgr2=0x557f4a8b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4e32f0 0x557f4a8b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 -- 172.21.15.5:0/3564445576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 msgr2=0x557f4a8cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1cd918640 1 -- 172.21.15.5:0/3564445576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f4a8b5510 con 0x557f4a4eb4d0 2024-05-14T15:33:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.728+0000 7ff1ce91a640 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.729+0000 7ff1cd918640 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b8df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1b801a440 tx=0x7ff1b8012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e8760ef79bdeae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.729+0000 7ff1b3fff640 1 -- 172.21.15.5:0/3564445576 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8029070 con 0x557f4a4eb4d0 2024-05-14T15:33:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.729+0000 7ff1b3fff640 1 -- 172.21.15.5:0/3564445576 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b8016070 con 0x557f4a4eb4d0 2024-05-14T15:33:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.729+0000 7ff1b3fff640 1 -- 172.21.15.5:0/3564445576 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8003c60 con 0x557f4a4eb4d0 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- 172.21.15.5:0/3564445576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 msgr2=0x557f4a8b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b8df0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff1b801a440 tx=0x7ff1b8012c60 comp rx=0 tx=0).stop 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- 172.21.15.5:0/3564445576 shutdown_connections 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a4eb4d0 0x557f4a8b8df0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a50c1b0 0x557f4a8cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- 172.21.15.5:0/3564445576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4e32f0 0x557f4a8b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- 172.21.15.5:0/3564445576 >> 172.21.15.5:0/3564445576 conn(0x557f4a500300 msgr2=0x557f4a8b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- 172.21.15.5:0/3564445576 shutdown_connections 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- 172.21.15.5:0/3564445576 wait complete. 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- start start 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 Processor -- start 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4eb4d0 0x557f4a8d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a50c1b0 0x557f4a937df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.730+0000 7ff1d7235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81ad850 con 0x557f4a4e32f0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1d7235740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81aecc0 con 0x557f4a4eb4d0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1d7235740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81adf20 con 0x557f4a50c1b0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1d7235740 10 client.0 _ll_register_callbacks cb 0x7ff1c40a6980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8d2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58766/0 (socket says 172.21.15.5:58766) 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 -- 172.21.15.5:0/1740440246 learned_addr learned my addr 172.21.15.5:0/1740440246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4eb4d0 0x557f4a8d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce91a640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a50c1b0 0x557f4a937df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 -- 172.21.15.5:0/1740440246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a50c1b0 msgr2=0x557f4a937df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a50c1b0 0x557f4a937df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 -- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4eb4d0 msgr2=0x557f4a8d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4eb4d0 0x557f4a8d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 -- 172.21.15.5:0/1740440246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f4a938330 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4eb4d0 0x557f4a8d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1ce119640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8d2050 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40aa240 tx=0x7ff1c400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187fe4d887134add server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1740440246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4005980 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1740440246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff1c40e6d00 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.731+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8016070 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1740440246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1c40a9dc0 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 10 client.17406 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557f4a9395b0 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1740440246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b8016070 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1740440246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff1c40a6540 con 0x557f4a4e32f0 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 10 client.17406 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 20 client.17406 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4a01c0 0x557f4a93bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 4 client.17406 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:33:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f4a81a050 con 0x557f4a4a01c0 2024-05-14T15:33:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.732+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4a01c0 0x557f4a93bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.733+0000 7ff1b1ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:21.733+0000 7ff1cd918640 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4a01c0 0x557f4a93bd60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232c3571ca212eaf server_cookie=edb85d050ee8386c in_seq=0 out_seq=0 2024-05-14T15:33:22.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:21 smithi005 ceph-mon[31559]: pgmap v265: 129 pgs: 129 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-05-14T15:33:22.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 799548 10.05 MB/sec execute 283 sec latency 175.567 ms 2024-05-14T15:33:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1b1ffb640 1 -- 172.21.15.5:0/1740440246 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f4a81a050 con 0x557f4a4a01c0 2024-05-14T15:33:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1b1ffb640 10 client.17406 handle_command_reply: tid=0 2024-05-14T15:33:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 client.17406 shutdown 2024-05-14T15:33:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 msgr2=0x557f4a8d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8d2050 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40aa240 tx=0x7ff1c400eec0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 shutdown_connections 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557f4a4a01c0 0x557f4a93bd60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f4a50c1b0 0x557f4a937df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f4a4eb4d0 0x557f4a8d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 --2- 172.21.15.5:0/1740440246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f4a4e32f0 0x557f4a8d2050 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 >> 172.21.15.5:0/1740440246 conn(0x557f4a500300 msgr2=0x557f4a8b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1b1ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 shutdown_connections 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.164+0000 7ff1d7235740 1 -- 172.21.15.5:0/1740440246 wait complete. 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d7235740 20 client.17406 trim_cache size 0 max 16384 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:33:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff1b40b2950 msgr2=0x7ff1b40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff1b40b2950 0x7ff1b40b4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0031440 tx=0x7ff1c002a410 comp rx=0 tx=0).stop 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 msgr2=0x7ff1c80fe870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c80fe870 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400ebc0 tx=0x7ff1c400c6a0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 shutdown_connections 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff1b40b2950 0x7ff1b40b4e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1c8108bc0 0x7ff1c80fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c8109060 0x7ff1c80ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 --2- 172.21.15.5:0/3180557165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1c8103310 0x7ff1c80fe870 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 >> 172.21.15.5:0/3180557165 conn(0x7ff1c80fe390 msgr2=0x7ff1c8075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 shutdown_connections 2024-05-14T15:33:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:22.165+0000 7ff1d037d640 1 -- 172.21.15.5:0/3180557165 wait complete. 2024-05-14T15:33:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:21 smithi144 ceph-mon[35650]: pgmap v265: 129 pgs: 129 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-05-14T15:33:22.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:21 smithi193 ceph-mon[36001]: pgmap v265: 129 pgs: 129 active+clean; 77 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 -- 172.21.15.5:0/4198056788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7948109860 msgr2=0x7f79481111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 --2- 172.21.15.5:0/4198056788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7948109860 0x7f79481111c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f794000b0d0 tx=0x7f794000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 -- 172.21.15.5:0/4198056788 shutdown_connections 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 --2- 172.21.15.5:0/4198056788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7948109860 0x7f79481111c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 --2- 172.21.15.5:0/4198056788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79481045e0 0x7f79481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 --2- 172.21.15.5:0/4198056788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7948108f10 0x7f7948109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 -- 172.21.15.5:0/4198056788 >> 172.21.15.5:0/4198056788 conn(0x7f79480ffe70 msgr2=0x7f79481022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 -- 172.21.15.5:0/4198056788 shutdown_connections 2024-05-14T15:33:22.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.686+0000 7f794ff92700 1 -- 172.21.15.5:0/4198056788 wait complete. 2024-05-14T15:33:22.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 Processor -- start 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 -- start start 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 0x7f794819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7948108f10 0x7f794819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7948109860 0x7f794819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7948114490 con 0x7f79481045e0 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7948114310 con 0x7f7948108f10 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794ff92700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7948114610 con 0x7f7948109860 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 0x7f794819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 0x7f794819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58778/0 (socket says 172.21.15.5:58778) 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794dd2e700 1 -- 172.21.15.5:0/1814502839 learned_addr learned my addr 172.21.15.5:0/1814502839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794d52d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7948108f10 0x7f794819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.687+0000 7f794e52f700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7948109860 0x7f794819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794dd2e700 1 -- 172.21.15.5:0/1814502839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7948109860 msgr2=0x7f794819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794dd2e700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7948109860 0x7f794819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794dd2e700 1 -- 172.21.15.5:0/1814502839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7948108f10 msgr2=0x7f794819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794dd2e700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7948108f10 0x7f794819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794dd2e700 1 -- 172.21.15.5:0/1814502839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794819fc70 con 0x7f79481045e0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794d52d700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7948108f10 0x7f794819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794dd2e700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 0x7f794819ada0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9a0 tx=0x7f794400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f793effd700 1 -- 172.21.15.5:0/1814502839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794400ca10 con 0x7f79481045e0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794819ff60 con 0x7f79481045e0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f793effd700 1 -- 172.21.15.5:0/1814502839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794400cbb0 con 0x7f79481045e0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f793effd700 1 -- 172.21.15.5:0/1814502839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944018680 con 0x7f79481045e0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.688+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79481a7880 con 0x7f79481045e0 2024-05-14T15:33:22.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.691+0000 7f793effd700 1 -- 172.21.15.5:0/1814502839 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7944010430 con 0x7f79481045e0 2024-05-14T15:33:22.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.692+0000 7f793effd700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f79340b29f0 0x7f79340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.692+0000 7f794d52d700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f79340b29f0 0x7f79340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.692+0000 7f793effd700 1 -- 172.21.15.5:0/1814502839 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f79440e2be0 con 0x7f79481045e0 2024-05-14T15:33:22.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.692+0000 7f794d52d700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f79340b29f0 0x7f79340b4f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f79481037e0 tx=0x7f7938009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:22.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.692+0000 7f795e9a6b80 1 Processor -- start 2024-05-14T15:33:22.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.692+0000 7f795e9a6b80 1 -- start start 2024-05-14T15:33:22.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34adde40 0x55fe34c5b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c5bc60 0x55fe34c5ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c60880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe34b7d860 con 0x55fe34c60470 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fe34b1c7c0 con 0x55fe34adde40 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe34b79820 con 0x55fe34c5bc60 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794e52f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c60880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794d52d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c5bc60 0x55fe34c5ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794e52f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c60880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58790/0 (socket says 172.21.15.5:58790) 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794e52f700 1 -- 172.21.15.5:0/1200690499 learned_addr learned my addr 172.21.15.5:0/1200690499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794d52d700 1 -- 172.21.15.5:0/1200690499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34adde40 msgr2=0x55fe34c5b720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794d52d700 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34adde40 0x55fe34c5b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794d52d700 1 -- 172.21.15.5:0/1200690499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 msgr2=0x55fe34c60880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794d52d700 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c60880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794d52d700 1 -- 172.21.15.5:0/1200690499 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe34aca8d0 con 0x55fe34c5bc60 2024-05-14T15:33:22.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.693+0000 7f794e52f700 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c60880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:22.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.694+0000 7f794d52d700 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c5bc60 0x55fe34c5ff30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f793800e630 tx=0x7f7938014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=699686a20c698561 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:22.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.694+0000 7f7933fff700 1 -- 172.21.15.5:0/1200690499 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938022070 con 0x55fe34c5bc60 2024-05-14T15:33:22.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.694+0000 7f7933fff700 1 -- 172.21.15.5:0/1200690499 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7938003680 con 0x55fe34c5bc60 2024-05-14T15:33:22.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.694+0000 7f7933fff700 1 -- 172.21.15.5:0/1200690499 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793801e650 con 0x55fe34c5bc60 2024-05-14T15:33:22.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 -- 172.21.15.5:0/1200690499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c5bc60 msgr2=0x55fe34c5ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c5bc60 0x55fe34c5ff30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f793800e630 tx=0x7f7938014000 comp rx=0 tx=0).stop 2024-05-14T15:33:22.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 -- 172.21.15.5:0/1200690499 shutdown_connections 2024-05-14T15:33:22.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c60880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34adde40 0x55fe34c5b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/1200690499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c5bc60 0x55fe34c5ff30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 -- 172.21.15.5:0/1200690499 >> 172.21.15.5:0/1200690499 conn(0x55fe34b25600 msgr2=0x55fe34c564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 -- 172.21.15.5:0/1200690499 shutdown_connections 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 -- 172.21.15.5:0/1200690499 wait complete. 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.695+0000 7f795e9a6b80 1 -- start start 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 Processor -- start 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c65240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c65780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c65cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe34c68420 con 0x55fe34adde40 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fe34ae3c30 con 0x55fe34c5bc60 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe34c685a0 con 0x55fe34c60470 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c65240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c65240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58792/0 (socket says 172.21.15.5:58792) 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f794dd2e700 1 -- 172.21.15.5:0/2333443394 learned_addr learned my addr 172.21.15.5:0/2333443394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f794d52d700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c65780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f795e9a6b80 10 client.0 _ll_register_callbacks cb 0x55fe34b15920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.696+0000 7f794e52f700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c65cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794e52f700 1 -- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 msgr2=0x55fe34c65780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794e52f700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c65780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794e52f700 1 -- 172.21.15.5:0/2333443394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 msgr2=0x55fe34c65240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794e52f700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c65240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794e52f700 1 -- 172.21.15.5:0/2333443394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe34b19690 con 0x55fe34c60470 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794dd2e700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c65240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f794e52f700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c65cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7940009900 tx=0x7f794000bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4530fb5a10c90396 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f79317fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f79317fa700 1 -- 172.21.15.5:0/2333443394 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940003e10 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7938003680 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.697+0000 7f79317fa700 1 -- 172.21.15.5:0/2333443394 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f79400075c0 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f79317fa700 1 -- 172.21.15.5:0/2333443394 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f794002d050 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 10 client.27007 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fe34c730f0 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f79317fa700 1 -- 172.21.15.5:0/2333443394 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7938003680 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f79317fa700 1 -- 172.21.15.5:0/2333443394 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7940011070 con 0x55fe34c60470 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 10 client.27007 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 10 client.27007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 20 client.27007 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34c77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 4 client.27007 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe34b8b240 con 0x55fe34c75a10 2024-05-14T15:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.698+0000 7f794dd2e700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34c77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.699+0000 7f79317fa700 10 client.27007 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:22.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.699+0000 7f794dd2e700 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34c77e60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2100364ea611228 server_cookie=85ead0435e74d146 in_seq=0 out_seq=0 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f79317fa700 1 -- 172.21.15.5:0/2333443394 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe34b8b240 con 0x55fe34c75a10 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f79317fa700 10 client.27007 handle_command_reply: tid=0 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 client.27007 shutdown 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 msgr2=0x55fe34c65cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c65cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7940009900 tx=0x7f794000bcd0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 shutdown_connections 2024-05-14T15:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34c77e60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c65cc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f79317fa700 0 client.27007 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c65240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/2333443394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c65780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 >> 172.21.15.5:0/2333443394 conn(0x55fe34b25600 msgr2=0x55fe34c5a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 shutdown_connections 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 1 -- 172.21.15.5:0/2333443394 wait complete. 2024-05-14T15:33:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.700+0000 7f795e9a6b80 20 client.27007 trim_cache size 0 max 16384 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 Processor -- start 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 -- start start 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 0x55fe34c5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c7db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c71840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79481b0d10 con 0x55fe34c60470 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ae5a0 con 0x55fe34c5bc60 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f795e9a6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ac430 con 0x55fe34adde40 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 0x55fe34c5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 0x55fe34c5fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57142/0 (socket says 172.21.15.5:57142) 2024-05-14T15:33:22.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f794dd2e700 1 -- 172.21.15.5:0/77376459 learned_addr learned my addr 172.21.15.5:0/77376459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.717+0000 7f794d52d700 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c7db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794e52f700 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c71840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794dd2e700 1 -- 172.21.15.5:0/77376459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 msgr2=0x55fe34c7db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794dd2e700 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c7db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794dd2e700 1 -- 172.21.15.5:0/77376459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 msgr2=0x55fe34c71840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794dd2e700 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c71840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794dd2e700 1 -- 172.21.15.5:0/77376459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe34c64560 con 0x55fe34adde40 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794e52f700 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c71840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f794dd2e700 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 0x55fe34c5fec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7944009230 tx=0x7f7944010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691ebda01ded6452 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f79337fe700 1 -- 172.21.15.5:0/77376459 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440a5040 con 0x55fe34adde40 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f79337fe700 1 -- 172.21.15.5:0/77376459 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79440b2040 con 0x55fe34adde40 2024-05-14T15:33:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.718+0000 7f79337fe700 1 -- 172.21.15.5:0/77376459 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794400cd60 con 0x55fe34adde40 2024-05-14T15:33:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 -- 172.21.15.5:0/77376459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 msgr2=0x55fe34c5fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 0x55fe34c5fec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7944009230 tx=0x7f7944010c10 comp rx=0 tx=0).stop 2024-05-14T15:33:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 -- 172.21.15.5:0/77376459 shutdown_connections 2024-05-14T15:33:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34c60470 0x55fe34c71840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34adde40 0x55fe34c5fec0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/77376459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c7db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 -- 172.21.15.5:0/77376459 >> 172.21.15.5:0/77376459 conn(0x55fe34b25600 msgr2=0x55fe34c5f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 -- 172.21.15.5:0/77376459 shutdown_connections 2024-05-14T15:33:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.719+0000 7f795e9a6b80 1 -- 172.21.15.5:0/77376459 wait complete. 2024-05-14T15:33:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.720+0000 7f795e9a6b80 1 -- start start 2024-05-14T15:33:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.720+0000 7f795e9a6b80 1 Processor -- start 2024-05-14T15:33:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.721+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.721+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f794dd2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c77e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58806/0 (socket says 172.21.15.5:58806) 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f795e9a6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f794dd2e700 1 -- 172.21.15.5:0/3518286438 learned_addr learned my addr 172.21.15.5:0/3518286438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79481aca80 con 0x55fe34adde40 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f79481aded0 con 0x55fe34c5bc60 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ad130 con 0x55fe34c60470 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f794d52d700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.722+0000 7f794e52f700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794dd2e700 1 -- 172.21.15.5:0/3518286438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 msgr2=0x55fe34c788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794dd2e700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794dd2e700 1 -- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 msgr2=0x55fe34c783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794dd2e700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794dd2e700 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe34cde0f0 con 0x55fe34adde40 2024-05-14T15:33:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f795e9a6b80 10 client.0 _ll_register_callbacks cb 0x55fe34b15920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794d52d700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:33:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79440b2040 con 0x55fe34adde40 2024-05-14T15:33:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f794dd2e700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c77e60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f79440dff20 tx=0x7f79440a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53e28c1d2dc0d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f79317fa700 10 client.17421 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f79317fa700 1 -- 172.21.15.5:0/3518286438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794406d030 con 0x55fe34adde40 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.723+0000 7f79317fa700 1 -- 172.21.15.5:0/3518286438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f794408f070 con 0x55fe34adde40 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f79317fa700 1 -- 172.21.15.5:0/3518286438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79440ade00 con 0x55fe34adde40 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 10 client.17421 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55fe34cdf0b0 con 0x55fe34adde40 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f79317fa700 1 -- 172.21.15.5:0/3518286438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f794406f2c0 con 0x55fe34adde40 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 10 client.17421 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 20 client.17421 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34ce1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 4 client.17421 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe34b852e0 con 0x55fe34c75a10 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.724+0000 7f794d52d700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34ce1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f79317fa700 10 client.17421 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f794d52d700 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34ce1610 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfcadc3f7f5e711 server_cookie=bf8fb07ee6dcd365 in_seq=0 out_seq=0 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f79317fa700 1 -- 172.21.15.5:0/3518286438 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe34b852e0 con 0x55fe34c75a10 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f79317fa700 10 client.17421 handle_command_reply: tid=0 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f795e9a6b80 1 client.17421 shutdown 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 msgr2=0x55fe34c77e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.725+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c77e60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f79440dff20 tx=0x7f79440a5040 comp rx=0 tx=0).stop 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 shutdown_connections 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fe34c75a10 0x55fe34ce1610 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe34c60470 0x55fe34c788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe34adde40 0x55fe34c77e60 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f79317fa700 0 client.17421 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 --2- 172.21.15.5:0/3518286438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe34c5bc60 0x55fe34c783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 >> 172.21.15.5:0/3518286438 conn(0x55fe34b25600 msgr2=0x55fe34c55be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 shutdown_connections 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 1 -- 172.21.15.5:0/3518286438 wait complete. 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.726+0000 7f795e9a6b80 20 client.17421 trim_cache size 0 max 16384 2024-05-14T15:33:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f79340b29f0 msgr2=0x7f79340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f79340b29f0 0x7f79340b4f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f79481037e0 tx=0x7f7938009260 comp rx=0 tx=0).stop 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 msgr2=0x7f794819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 0x7f794819ada0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9a0 tx=0x7f794400ecb0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 shutdown_connections 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f79340b29f0 0x7f79340b4f00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7948109860 0x7f794819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79481045e0 0x7f794819ada0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 --2- 172.21.15.5:0/1814502839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7948108f10 0x7f794819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 >> 172.21.15.5:0/1814502839 conn(0x7f79480ffe70 msgr2=0x7f7948108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 shutdown_connections 2024-05-14T15:33:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:22.727+0000 7f794ff92700 1 -- 172.21.15.5:0/1814502839 wait complete. 2024-05-14T15:33:23.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 801621 10.05 MB/sec execute 284 sec latency 21.247 ms 2024-05-14T15:33:24.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 803338 10.05 MB/sec execute 285 sec latency 15.263 ms 2024-05-14T15:33:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:23 smithi144 ceph-mon[35650]: pgmap v266: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-05-14T15:33:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:23 smithi144 ceph-mon[35650]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T15:33:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:23 smithi193 ceph-mon[36001]: pgmap v266: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-05-14T15:33:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:23 smithi193 ceph-mon[36001]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T15:33:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:23 smithi005 ceph-mon[31559]: pgmap v266: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-05-14T15:33:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:23 smithi005 ceph-mon[31559]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T15:33:24.650 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:33:24.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:33:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:25.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 805748 10.06 MB/sec execute 286 sec latency 38.161 ms 2024-05-14T15:33:26.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 806878 10.05 MB/sec execute 287 sec latency 48.681 ms 2024-05-14T15:33:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:25 smithi144 ceph-mon[35650]: pgmap v268: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-05-14T15:33:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:25 smithi193 ceph-mon[36001]: pgmap v268: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-05-14T15:33:26.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:25 smithi005 ceph-mon[31559]: pgmap v268: 129 pgs: 129 active+clean; 83 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 241 op/s 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 -- 172.21.15.5:0/3909277468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 msgr2=0x7f49b01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 --2- 172.21.15.5:0/3909277468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b01100a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b0d0 tx=0x7f49a800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 -- 172.21.15.5:0/3909277468 shutdown_connections 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 --2- 172.21.15.5:0/3909277468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b01100a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 --2- 172.21.15.5:0/3909277468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49b01034c0 0x7f49b01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 --2- 172.21.15.5:0/3909277468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 0x7f49b0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 -- 172.21.15.5:0/3909277468 >> 172.21.15.5:0/3909277468 conn(0x7f49b00fedc0 msgr2=0x7f49b01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 -- 172.21.15.5:0/3909277468 shutdown_connections 2024-05-14T15:33:26.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.911+0000 7f49b7679700 1 -- 172.21.15.5:0/3909277468 wait complete. 2024-05-14T15:33:26.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 Processor -- start 2024-05-14T15:33:26.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 -- start start 2024-05-14T15:33:26.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49b01034c0 0x7f49b00ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 0x7f49b00ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b00ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0113370 con 0x7f49b0108740 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01131f0 con 0x7f49b0107df0 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b7679700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01134f0 con 0x7f49b01034c0 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b5c16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b00ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b4c14700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 0x7f49b00ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b5415700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49b01034c0 0x7f49b00ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b5c16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b00ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48772/0 (socket says 172.21.15.5:48772) 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.912+0000 7f49b5c16700 1 -- 172.21.15.5:0/1579479407 learned_addr learned my addr 172.21.15.5:0/1579479407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b5c16700 1 -- 172.21.15.5:0/1579479407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49b01034c0 msgr2=0x7f49b00ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b5c16700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49b01034c0 0x7f49b00ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b5c16700 1 -- 172.21.15.5:0/1579479407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 msgr2=0x7f49b00ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b5c16700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 0x7f49b00ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b5c16700 1 -- 172.21.15.5:0/1579479407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b01a9150 con 0x7f49b0108740 2024-05-14T15:33:26.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b4c14700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 0x7f49b00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b5c16700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b00ffd20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009850 tx=0x7f49a8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49a67fc700 1 -- 172.21.15.5:0/1579479407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a801f070 con 0x7f49b0108740 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49a67fc700 1 -- 172.21.15.5:0/1579479407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a8004c50 con 0x7f49b0108740 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b01a93e0 con 0x7f49b0108740 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49a67fc700 1 -- 172.21.15.5:0/1579479407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a801b690 con 0x7f49b0108740 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.913+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01a9830 con 0x7f49b0108740 2024-05-14T15:33:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.916+0000 7f49a67fc700 1 -- 172.21.15.5:0/1579479407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f49a8022070 con 0x7f49b0108740 2024-05-14T15:33:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.917+0000 7f49a67fc700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f499c0b29f0 0x7f499c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.917+0000 7f49b5415700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f499c0b29f0 0x7f499c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.917+0000 7f49a67fc700 1 -- 172.21.15.5:0/1579479407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f49a800f020 con 0x7f49b0108740 2024-05-14T15:33:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 Processor -- start 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 -- start start 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f499c0b29f0 0x7f499c0b4f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005950 tx=0x7f49ac00b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 0x55975b1f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55975b1108e0 con 0x55975b1eece0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55975b0af840 con 0x55975b070ec0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49c608db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55975b10c8a0 con 0x55975b1f34f0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b4c14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 0x55975b1f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b4c14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 0x55975b1f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48784/0 (socket says 172.21.15.5:48784) 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b4c14700 1 -- 172.21.15.5:0/76662133 learned_addr learned my addr 172.21.15.5:0/76662133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5c16700 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 -- 172.21.15.5:0/76662133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 msgr2=0x55975b1f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 -- 172.21.15.5:0/76662133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 msgr2=0x55975b1f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 0x55975b1f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b5415700 1 -- 172.21.15.5:0/76662133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975b05d950 con 0x55975b070ec0 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.918+0000 7f49b4c14700 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 0x55975b1f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.919+0000 7f49b5415700 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1ee7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005d90 tx=0x7f49ac013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e91c0dd6c05bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.919+0000 7f499b7fe700 1 -- 172.21.15.5:0/76662133 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac011b80 con 0x55975b070ec0 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.919+0000 7f499b7fe700 1 -- 172.21.15.5:0/76662133 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ac011d20 con 0x55975b070ec0 2024-05-14T15:33:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.919+0000 7f499b7fe700 1 -- 172.21.15.5:0/76662133 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01e420 con 0x55975b070ec0 2024-05-14T15:33:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 -- 172.21.15.5:0/76662133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 msgr2=0x55975b1ee7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1ee7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005d90 tx=0x7f49ac013000 comp rx=0 tx=0).stop 2024-05-14T15:33:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 -- 172.21.15.5:0/76662133 shutdown_connections 2024-05-14T15:33:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1ee7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 --2- 172.21.15.5:0/76662133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1eece0 0x55975b1f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 -- 172.21.15.5:0/76662133 >> 172.21.15.5:0/76662133 conn(0x55975b0b8680 msgr2=0x55975b1e9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 -- 172.21.15.5:0/76662133 shutdown_connections 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 -- 172.21.15.5:0/76662133 wait complete. 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 -- start start 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.920+0000 7f49c608db80 1 Processor -- start 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1eece0 0x55975b1f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b1f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55975b1fb4a0 con 0x55975b1f34f0 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55975b076cb0 con 0x55975b070ec0 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55975b1fb620 con 0x55975b1eece0 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49c608db80 10 client.0 _ll_register_callbacks cb 0x55975b0a89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49b5415700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49b5415700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51156/0 (socket says 172.21.15.5:51156) 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49b5c16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b1f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49b5415700 1 -- 172.21.15.5:0/273050357 learned_addr learned my addr 172.21.15.5:0/273050357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49b4c14700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1eece0 0x55975b1f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.921+0000 7f49b5415700 1 -- 172.21.15.5:0/273050357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1eece0 msgr2=0x55975b1f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49b5415700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1eece0 0x55975b1f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49b5415700 1 -- 172.21.15.5:0/273050357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 msgr2=0x55975b1f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49b5415700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b1f8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49b5415700 1 -- 172.21.15.5:0/273050357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975b0ac710 con 0x55975b070ec0 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49b4c14700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1eece0 0x55975b1f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49b5415700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1f8160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005160 tx=0x7f49ac011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ea0ff4f9d35707 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f4998ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f4998ff9700 1 -- 172.21.15.5:0/273050357 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac035060 con 0x55975b070ec0 2024-05-14T15:33:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f4998ff9700 1 -- 172.21.15.5:0/273050357 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f49ac005630 con 0x55975b070ec0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ac011d20 con 0x55975b070ec0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f4998ff9700 1 -- 172.21.15.5:0/273050357 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49ac03c050 con 0x55975b070ec0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49c608db80 10 client.27050 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55975b206040 con 0x55975b070ec0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.922+0000 7f4998ff9700 1 -- 172.21.15.5:0/273050357 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ac011d20 con 0x55975b070ec0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f4998ff9700 1 -- 172.21.15.5:0/273050357 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f49ac0269b0 con 0x55975b070ec0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49c608db80 10 client.27050 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49c608db80 10 client.27050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49c608db80 20 client.27050 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49c608db80 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b208620 0x55975b20aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49c608db80 4 client.27050 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55975b11e2c0 con 0x55975b208620 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49b4c14700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b208620 0x55975b20aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f49b4c14700 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b208620 0x55975b20aa40 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb968ae382d6456 server_cookie=a2b7a62c83218276 in_seq=0 out_seq=0 2024-05-14T15:33:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.923+0000 7f4998ff9700 10 client.27050 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.924+0000 7f4998ff9700 1 -- 172.21.15.5:0/273050357 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55975b11e2c0 con 0x55975b208620 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f4998ff9700 10 client.27050 handle_command_reply: tid=0 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 client.27050 shutdown 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 msgr2=0x55975b1f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1f8160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005160 tx=0x7f49ac011bf0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 shutdown_connections 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b208620 0x55975b20aa40 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b1f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b070ec0 0x55975b1f8160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 --2- 172.21.15.5:0/273050357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1eece0 0x55975b1f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 >> 172.21.15.5:0/273050357 conn(0x55975b0b8680 msgr2=0x55975b1ed4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f4998ff9700 0 client.27050 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 shutdown_connections 2024-05-14T15:33:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 1 -- 172.21.15.5:0/273050357 wait complete. 2024-05-14T15:33:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.925+0000 7f49c608db80 20 client.27050 trim_cache size 0 max 16384 2024-05-14T15:33:26.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 Processor -- start 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 -- start start 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 0x55975b2048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b1e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b4f80 con 0x55975b070ec0 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b2810 con 0x55975b1eece0 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49c608db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b0680 con 0x55975b1f34f0 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49b5415700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 0x55975b2048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49b5415700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 0x55975b2048f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48790/0 (socket says 172.21.15.5:48790) 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49b4c14700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b1e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49b5415700 1 -- 172.21.15.5:0/426119253 learned_addr learned my addr 172.21.15.5:0/426119253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:26.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.947+0000 7f49b5c16700 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b4c14700 1 -- 172.21.15.5:0/426119253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 msgr2=0x55975b1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b4c14700 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b4c14700 1 -- 172.21.15.5:0/426119253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 msgr2=0x55975b2048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b4c14700 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 0x55975b2048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b4c14700 1 -- 172.21.15.5:0/426119253 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975b1ef480 con 0x55975b1eece0 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b5415700 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 0x55975b2048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b5c16700 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f49b4c14700 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b1e9690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55975b1fb6a0 tx=0x7f49a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c939e398cae2c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f499affd700 1 -- 172.21.15.5:0/426119253 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0003680 con 0x55975b1eece0 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f499affd700 1 -- 172.21.15.5:0/426119253 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a0024030 con 0x55975b1eece0 2024-05-14T15:33:26.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.948+0000 7f499affd700 1 -- 172.21.15.5:0/426119253 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00208f0 con 0x55975b1eece0 2024-05-14T15:33:26.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 -- 172.21.15.5:0/426119253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 msgr2=0x55975b1e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b1e9690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55975b1fb6a0 tx=0x7f49a0009260 comp rx=0 tx=0).stop 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 -- 172.21.15.5:0/426119253 shutdown_connections 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b1f34f0 0x55975b1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b070ec0 0x55975b2048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 --2- 172.21.15.5:0/426119253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b1e9690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 -- 172.21.15.5:0/426119253 >> 172.21.15.5:0/426119253 conn(0x55975b0b8680 msgr2=0x55975b1f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 -- 172.21.15.5:0/426119253 shutdown_connections 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.949+0000 7f49c608db80 1 -- 172.21.15.5:0/426119253 wait complete. 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 -- start start 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 Processor -- start 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 0x55975b20b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b20bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b271b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b0cd0 con 0x55975b1f34f0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b2140 con 0x55975b1eece0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01b13a0 con 0x55975b070ec0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49c608db80 10 client.0 _ll_register_callbacks cb 0x55975b0a89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49b5415700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 0x55975b20b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.950+0000 7f49b5c16700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b271b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 0x55975b20b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37022/0 (socket says 172.21.15.5:37022) 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 -- 172.21.15.5:0/4081332749 learned_addr learned my addr 172.21.15.5:0/4081332749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b4c14700 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b20bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 -- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 msgr2=0x55975b20bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b20bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 -- 172.21.15.5:0/4081332749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 msgr2=0x55975b271b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b271b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 -- 172.21.15.5:0/4081332749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975b2720b0 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49b5415700 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 0x55975b20b880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005160 tx=0x7f49ac010fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e25580572d6f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f4998ff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f4998ff9700 1 -- 172.21.15.5:0/4081332749 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac035040 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f4998ff9700 1 -- 172.21.15.5:0/4081332749 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f49ac01e420 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.951+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a0024030 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f4998ff9700 1 -- 172.21.15.5:0/4081332749 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49ac04e050 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 10 client.27031 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55975b2731e0 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f4998ff9700 1 -- 172.21.15.5:0/4081332749 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a0024030 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f4998ff9700 1 -- 172.21.15.5:0/4081332749 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f49ac004960 con 0x55975b070ec0 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 10 client.27031 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 10 client.27031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 20 client.27031 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b2738b0 0x55975b275cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 4 client.27031 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:33:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55975b118360 con 0x55975b2738b0 2024-05-14T15:33:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.952+0000 7f49b4c14700 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b2738b0 0x55975b275cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.953+0000 7f4998ff9700 10 client.27031 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.953+0000 7f49b4c14700 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b2738b0 0x55975b275cd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5613d0f405a8a16c server_cookie=48f3300987e210ac in_seq=0 out_seq=0 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f4998ff9700 1 -- 172.21.15.5:0/4081332749 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55975b118360 con 0x55975b2738b0 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f4998ff9700 10 client.27031 handle_command_reply: tid=0 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 client.27031 shutdown 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 msgr2=0x55975b20b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 0x55975b20b880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005160 tx=0x7f49ac010fa0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 shutdown_connections 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55975b2738b0 0x55975b275cd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55975b1f34f0 0x55975b271b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975b070ec0 0x55975b20b880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 --2- 172.21.15.5:0/4081332749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55975b1eece0 0x55975b20bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 >> 172.21.15.5:0/4081332749 conn(0x55975b0b8680 msgr2=0x55975b1e93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.954+0000 7f4998ff9700 0 client.27031 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 shutdown_connections 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49c608db80 1 -- 172.21.15.5:0/4081332749 wait complete. 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49c608db80 20 client.27031 trim_cache size 0 max 16384 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5f508b55-34b3-48ca-9645-a83409937d37", 2024-05-14T15:33:26.960 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:33:26.961 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:33:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f499c0b29f0 msgr2=0x7f499c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f499c0b29f0 0x7f499c0b4f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005950 tx=0x7f49ac00b410 comp rx=0 tx=0).stop 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 msgr2=0x7f49b00ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b00ffd20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009850 tx=0x7f49a8007a20 comp rx=0 tx=0).stop 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 shutdown_connections 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f499c0b29f0 0x7f499c0b4f00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49b0108740 0x7f49b00ffd20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49b01034c0 0x7f49b00ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 --2- 172.21.15.5:0/1579479407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49b0107df0 0x7f49b00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.955+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 >> 172.21.15.5:0/1579479407 conn(0x7f49b00fedc0 msgr2=0x7f49b006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.956+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 shutdown_connections 2024-05-14T15:33:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:26.956+0000 7f49b7679700 1 -- 172.21.15.5:0/1579479407 wait complete. 2024-05-14T15:33:27.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 808736 10.05 MB/sec execute 288 sec latency 39.090 ms 2024-05-14T15:33:27.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:33:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:33:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:27 smithi005 ceph-mon[31559]: pgmap v269: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-05-14T15:33:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:27 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:33:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:27 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:33:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:27 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:33:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:27 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:33:28.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 810314 10.04 MB/sec execute 289 sec latency 21.283 ms 2024-05-14T15:33:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:27 smithi144 ceph-mon[35650]: pgmap v269: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-05-14T15:33:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:27 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:33:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:27 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:33:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:27 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:33:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:27 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:33:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:27 smithi193 ceph-mon[36001]: pgmap v269: 129 pgs: 129 active+clean; 81 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2024-05-14T15:33:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:27 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:33:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:27 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:33:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:27 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:33:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:27 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:33:29.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 812724 10.05 MB/sec execute 290 sec latency 26.038 ms 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 -- 172.21.15.5:0/3855265290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 msgr2=0x7f3da01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/3855265290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da01038d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3d900099c0 tx=0x7f3d90009cd0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 -- 172.21.15.5:0/3855265290 shutdown_connections 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/3855265290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da0108740 0x7f3da01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/3855265290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da01038d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/3855265290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 0x7f3da0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 -- 172.21.15.5:0/3855265290 >> 172.21.15.5:0/3855265290 conn(0x7f3da00fedc0 msgr2=0x7f3da01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 -- 172.21.15.5:0/3855265290 shutdown_connections 2024-05-14T15:33:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.931+0000 7f3da4ee3700 1 -- 172.21.15.5:0/3855265290 wait complete. 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 Processor -- start 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 -- start start 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 0x7f3da019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da0108740 0x7f3da019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0113370 con 0x7f3da01034c0 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01131f0 con 0x7f3da0108740 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3da4ee3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01134f0 con 0x7f3da0107df0 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3d9e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3d9e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48824/0 (socket says 172.21.15.5:48824) 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3d9e59c700 1 -- 172.21.15.5:0/1139270890 learned_addr learned my addr 172.21.15.5:0/1139270890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da0108740 0x7f3da019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.932+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 0x7f3da019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9e59c700 1 -- 172.21.15.5:0/1139270890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 msgr2=0x7f3da019c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9e59c700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 0x7f3da019c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9e59c700 1 -- 172.21.15.5:0/1139270890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da0108740 msgr2=0x7f3da019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9e59c700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da0108740 0x7f3da019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9e59c700 1 -- 172.21.15.5:0/1139270890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da01a6f80 con 0x7f3da01034c0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 0x7f3da019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d9e59c700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da019bc30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006010 tx=0x7f3d90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d8f7fe700 1 -- 172.21.15.5:0/1139270890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90026070 con 0x7f3da01034c0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d8f7fe700 1 -- 172.21.15.5:0/1139270890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d90004630 con 0x7f3da01034c0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da01a7210 con 0x7f3da01034c0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3d8f7fe700 1 -- 172.21.15.5:0/1139270890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000fb20 con 0x7f3da01034c0 2024-05-14T15:33:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.933+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da01a7660 con 0x7f3da01034c0 2024-05-14T15:33:29.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.937+0000 7f3d8f7fe700 1 -- 172.21.15.5:0/1139270890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3d90029070 con 0x7f3da01034c0 2024-05-14T15:33:29.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.937+0000 7f3db38f7b80 1 Processor -- start 2024-05-14T15:33:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.937+0000 7f3db38f7b80 1 -- start start 2024-05-14T15:33:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74be9f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d8f7fe700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3d880b29f0 0x7f3d880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74be998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74be9a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d74bdbb940 con 0x55d74bd1be40 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d74bd20da0 con 0x55d74be99e10 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d74bdb7900 con 0x55d74be9fd80 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74be9f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74be998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74be998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37048/0 (socket says 172.21.15.5:37048) 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3621136225 learned_addr learned my addr 172.21.15.5:0/3621136225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74be9f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48826/0 (socket says 172.21.15.5:48826) 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d8f7fe700 1 -- 172.21.15.5:0/1139270890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f3d900e3120 con 0x7f3da01034c0 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9e59c700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3d880b29f0 0x7f3d880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74be9a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 -- 172.21.15.5:0/3621136225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 msgr2=0x55d74be998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74be998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 -- 172.21.15.5:0/3621136225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 msgr2=0x55d74be9a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74be9a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.938+0000 7f3d9ed9d700 1 -- 172.21.15.5:0/3621136225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74bd088d0 con 0x55d74bd1be40 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.939+0000 7f3d9e59c700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3d880b29f0 0x7f3d880b4f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b580 tx=0x7f3d900769e0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.939+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74be9f840 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3da006c6f0 tx=0x7f3d98009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948957fe37bd0ef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.939+0000 7f3d8cff9700 1 -- 172.21.15.5:0/3621136225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98008e70 con 0x55d74bd1be40 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.939+0000 7f3d8cff9700 1 -- 172.21.15.5:0/3621136225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d98018850 con 0x55d74bd1be40 2024-05-14T15:33:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.939+0000 7f3d8cff9700 1 -- 172.21.15.5:0/3621136225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004280 con 0x55d74bd1be40 2024-05-14T15:33:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3621136225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 msgr2=0x55d74be9f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74be9f840 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3da006c6f0 tx=0x7f3d98009f90 comp rx=0 tx=0).stop 2024-05-14T15:33:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3621136225 shutdown_connections 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74be9a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74be9f840 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3621136225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74be998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3621136225 >> 172.21.15.5:0/3621136225 conn(0x55d74bd63630 msgr2=0x55d74be94a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3621136225 shutdown_connections 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3621136225 wait complete. 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 -- start start 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.940+0000 7f3db38f7b80 1 Processor -- start 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74bd1be40 0x55d74bea3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bea3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d74beaac30 con 0x55d74be9fd80 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d74bd74ed0 con 0x55d74be99e10 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d74beaadb0 con 0x55d74bd1be40 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3db38f7b80 10 client.0 _ll_register_callbacks cb 0x55d74bd53950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:29.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74bd1be40 0x55d74bea3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bea3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74bd1be40 0x55d74bea3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37050/0 (socket says 172.21.15.5:37050) 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9ed9d700 1 -- 172.21.15.5:0/3728923609 learned_addr learned my addr 172.21.15.5:0/3728923609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3728923609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74bd1be40 msgr2=0x55d74bea3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74bd1be40 0x55d74bea3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3728923609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 msgr2=0x55d74bea3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.941+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3728923609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74bd72760 con 0x55d74be99e10 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bea3ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55d74be961f0 tx=0x7f3d94009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ddadf34017e7e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3728923609 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94004030 con 0x55d74be99e10 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3728923609 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f3d940041d0 con 0x55d74be99e10 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d98018850 con 0x55d74be99e10 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3728923609 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d940043d0 con 0x55d74be99e10 2024-05-14T15:33:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3db38f7b80 10 client.27068 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d74beb58f0 con 0x55d74be99e10 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3728923609 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d98018850 con 0x55d74be99e10 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3728923609 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3d94003a20 con 0x55d74be99e10 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3db38f7b80 10 client.27068 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.942+0000 7f3db38f7b80 10 client.27068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3db38f7b80 20 client.27068 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74beb7f30 0x55d74beba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3db38f7b80 4 client.27068 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d74bdc9320 con 0x55d74beb7f30 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74beb7f30 0x55d74beba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3d85ffb700 10 client.27068 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.943+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74beb7f30 0x55d74beba350 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f03d88c4f26407 server_cookie=d9c305f2be6e811a in_seq=0 out_seq=0 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3728923609 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d74bdc9320 con 0x55d74beb7f30 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3d85ffb700 10 client.27068 handle_command_reply: tid=0 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 client.27068 shutdown 2024-05-14T15:33:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 msgr2=0x55d74bea3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bea3ab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55d74be961f0 tx=0x7f3d94009e20 comp rx=0 tx=0).stop 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 shutdown_connections 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74beb7f30 0x55d74beba350 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74bd1be40 0x55d74bea3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3d85ffb700 0 client.27068 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3728923609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bea3ab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 >> 172.21.15.5:0/3728923609 conn(0x55d74bd63630 msgr2=0x55d74be988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 shutdown_connections 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.944+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3728923609 wait complete. 2024-05-14T15:33:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.945+0000 7f3db38f7b80 20 client.27068 trim_cache size 0 max 16384 2024-05-14T15:33:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 Processor -- start 2024-05-14T15:33:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 -- start start 2024-05-14T15:33:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74bd1be40 0x55d74be998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 0x55d74bea11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01b0c30 con 0x55d74be9fd80 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ae4c0 con 0x55d74bd1be40 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.967+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ac330 con 0x55d74be99e10 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 0x55d74bea11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74bd1be40 0x55d74be998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 0x55d74bea11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37066/0 (socket says 172.21.15.5:37066) 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/1615541514 learned_addr learned my addr 172.21.15.5:0/1615541514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/1615541514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74bd1be40 msgr2=0x55d74be998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74bd1be40 0x55d74be998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/1615541514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 msgr2=0x55d74bea3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/1615541514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74bea1ae0 con 0x55d74be99e10 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 0x55d74bea11a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400a010 tx=0x7f3d94010690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21701fe80abd96a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d87fff700 1 -- 172.21.15.5:0/1615541514 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9402f070 con 0x55d74be99e10 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d87fff700 1 -- 172.21.15.5:0/1615541514 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d94003b30 con 0x55d74be99e10 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.968+0000 7f3d87fff700 1 -- 172.21.15.5:0/1615541514 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400fd70 con 0x55d74be99e10 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 -- 172.21.15.5:0/1615541514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 msgr2=0x55d74bea11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 0x55d74bea11a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400a010 tx=0x7f3d94010690 comp rx=0 tx=0).stop 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 -- 172.21.15.5:0/1615541514 shutdown_connections 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74be9fd80 0x55d74bea3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74bd1be40 0x55d74be998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/1615541514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be99e10 0x55d74bea11a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 -- 172.21.15.5:0/1615541514 >> 172.21.15.5:0/1615541514 conn(0x55d74bd63630 msgr2=0x55d74be9edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 -- 172.21.15.5:0/1615541514 shutdown_connections 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.969+0000 7f3db38f7b80 1 -- 172.21.15.5:0/1615541514 wait complete. 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 -- start start 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 Processor -- start 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74bebf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bebf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74bebfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ac980 con 0x55d74bd1be40 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01addf0 con 0x55d74be99e10 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ad050 con 0x55d74be9fd80 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3db38f7b80 10 client.0 _ll_register_callbacks cb 0x55d74bd53950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bebf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3d9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74bebf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3d9dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bebf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51200/0 (socket says 172.21.15.5:51200) 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.970+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3541992028 learned_addr learned my addr 172.21.15.5:0/3541992028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74bebfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3541992028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 msgr2=0x55d74bebfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74bebfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3541992028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 msgr2=0x55d74bebf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74bebf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9dd9b700 1 -- 172.21.15.5:0/3541992028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74bf21710 con 0x55d74be99e10 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74bebf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d9dd9b700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bebf930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d94000c00 tx=0x7f3d94005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8da96508a519f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3541992028 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94039060 con 0x55d74be99e10 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d94003b30 con 0x55d74be99e10 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3541992028 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f3d94003680 con 0x55d74be99e10 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.971+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3541992028 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d94040050 con 0x55d74be99e10 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 10 client.27074 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d74bf22980 con 0x55d74be99e10 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3541992028 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d94003b30 con 0x55d74be99e10 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3541992028 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3d94010b20 con 0x55d74be99e10 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 10 client.27074 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 20 client.27074 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74bf22f40 0x55d74bf25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 4 client.27074 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d74bdc33c0 con 0x55d74bf22f40 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.972+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74bf22f40 0x55d74bf25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.973+0000 7f3d85ffb700 10 client.27074 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.973+0000 7f3d9ed9d700 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74bf22f40 0x55d74bf25360 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5aaabc2b93ce4d8 server_cookie=7256e94b8683d68 in_seq=0 out_seq=0 2024-05-14T15:33:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3d85ffb700 1 -- 172.21.15.5:0/3541992028 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d74bdc33c0 con 0x55d74bf22f40 2024-05-14T15:33:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3d85ffb700 10 client.27074 handle_command_reply: tid=0 2024-05-14T15:33:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 client.27074 shutdown 2024-05-14T15:33:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 msgr2=0x55d74bebf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bebf930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d94000c00 tx=0x7f3d94005a00 comp rx=0 tx=0).stop 2024-05-14T15:33:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 shutdown_connections 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55d74bf22f40 0x55d74bf25360 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d74be9fd80 0x55d74bebfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74bd1be40 0x55d74bebf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3d85ffb700 0 client.27074 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 --2- 172.21.15.5:0/3541992028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d74be99e10 0x55d74bebf930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 >> 172.21.15.5:0/3541992028 conn(0x55d74bd63630 msgr2=0x55d74be9d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 shutdown_connections 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 1 -- 172.21.15.5:0/3541992028 wait complete. 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3db38f7b80 20 client.27074 trim_cache size 0 max 16384 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:33:29.979 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "5f508b55-34b3-48ca-9645-a83409937d37": { 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5f508b55-34b3-48ca-9645-a83409937d37", 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:33:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3d880b29f0 msgr2=0x7f3d880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.976+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3d880b29f0 0x7f3d880b4f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b580 tx=0x7f3d900769e0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 msgr2=0x7f3da019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da019bc30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006010 tx=0x7f3d90004270 comp rx=0 tx=0).stop 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 shutdown_connections 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3d880b29f0 0x7f3d880b4f00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da0108740 0x7f3da019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da01034c0 0x7f3da019bc30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 --2- 172.21.15.5:0/1139270890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3da0107df0 0x7f3da019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 >> 172.21.15.5:0/1139270890 conn(0x7f3da00fedc0 msgr2=0x7f3da0101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 shutdown_connections 2024-05-14T15:33:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:33:29.977+0000 7f3da4ee3700 1 -- 172.21.15.5:0/1139270890 wait complete. 2024-05-14T15:33:30.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 814296 10.04 MB/sec execute 291 sec latency 20.371 ms 2024-05-14T15:33:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:29 smithi144 ceph-mon[35650]: pgmap v270: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-05-14T15:33:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:29 smithi193 ceph-mon[36001]: pgmap v270: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-05-14T15:33:30.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:29 smithi005 ceph-mon[31559]: pgmap v270: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-05-14T15:33:30.749 INFO:tasks.cephfs.filesystem:scrub status for tag:5f508b55-34b3-48ca-9645-a83409937d37 - {'path': '/', 'tag': '5f508b55-34b3-48ca-9645-a83409937d37', 'options': 'recursive,force'} 2024-05-14T15:33:31.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 816550 10.06 MB/sec execute 292 sec latency 91.482 ms 2024-05-14T15:33:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:30 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:33:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:30 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:33:31.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:30 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:33:32.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 817401 10.03 MB/sec execute 293 sec latency 87.631 ms 2024-05-14T15:33:32.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:31 smithi144 ceph-mon[35650]: pgmap v271: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-05-14T15:33:32.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:31 smithi193 ceph-mon[36001]: pgmap v271: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-05-14T15:33:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:31 smithi005 ceph-mon[31559]: pgmap v271: 129 pgs: 129 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 357 op/s 2024-05-14T15:33:33.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 819590 10.04 MB/sec execute 294 sec latency 29.136 ms 2024-05-14T15:33:34.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 821271 10.04 MB/sec execute 295 sec latency 21.398 ms 2024-05-14T15:33:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:33 smithi144 ceph-mon[35650]: pgmap v272: 129 pgs: 129 active+clean; 80 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 361 op/s 2024-05-14T15:33:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:33 smithi193 ceph-mon[36001]: pgmap v272: 129 pgs: 129 active+clean; 80 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 361 op/s 2024-05-14T15:33:34.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:33 smithi005 ceph-mon[31559]: pgmap v272: 129 pgs: 129 active+clean; 80 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 361 op/s 2024-05-14T15:33:35.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 823693 10.05 MB/sec execute 296 sec latency 32.019 ms 2024-05-14T15:33:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 -- 172.21.15.5:0/2211683378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac10e6d0 msgr2=0x7f6cac110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/2211683378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac10e6d0 0x7f6cac110af0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009ca0 tx=0x7f6ca402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 -- 172.21.15.5:0/2211683378 shutdown_connections 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/2211683378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac1021d0 0x7f6cac1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/2211683378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac10e6d0 0x7f6cac110af0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/2211683378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 0x7f6cac101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 -- 172.21.15.5:0/2211683378 >> 172.21.15.5:0/2211683378 conn(0x7f6cac0779e0 msgr2=0x7f6cac077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 -- 172.21.15.5:0/2211683378 shutdown_connections 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.274+0000 7f6cb30bf640 1 -- 172.21.15.5:0/2211683378 wait complete. 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 Processor -- start 2024-05-14T15:33:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 -- start start 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 0x7f6cac1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 0x7f6cac1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac10e6d0 0x7f6cac19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac113e90 con 0x7f6cac1021d0 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac113d10 con 0x7f6cac109d20 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.275+0000 7f6cb30bf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac114010 con 0x7f6cac10e6d0 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 0x7f6cac1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 0x7f6cac1a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48874/0 (socket says 172.21.15.5:48874) 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 -- 172.21.15.5:0/1870536088 learned_addr learned my addr 172.21.15.5:0/1870536088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cabfff640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 0x7f6cac1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb165c640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac10e6d0 0x7f6cac19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 -- 172.21.15.5:0/1870536088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac10e6d0 msgr2=0x7f6cac19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac10e6d0 0x7f6cac19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 -- 172.21.15.5:0/1870536088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 msgr2=0x7f6cac1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 0x7f6cac1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 -- 172.21.15.5:0/1870536088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac19ade0 con 0x7f6cac1021d0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cabfff640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 0x7f6cac1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb0e5b640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 0x7f6cac1a0530 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e980 tx=0x7f6ca000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6ca9ffb640 1 -- 172.21.15.5:0/1870536088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000cd80 con 0x7f6cac1021d0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6ca9ffb640 1 -- 172.21.15.5:0/1870536088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca00108e0 con 0x7f6cac1021d0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac19b070 con 0x7f6cac1021d0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.276+0000 7f6ca9ffb640 1 -- 172.21.15.5:0/1870536088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0018990 con 0x7f6cac1021d0 2024-05-14T15:33:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.277+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac1a72d0 con 0x7f6cac1021d0 2024-05-14T15:33:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.279+0000 7f6cb9f70740 1 Processor -- start 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.279+0000 7f6cb9f70740 1 -- start start 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.279+0000 7f6ca9ffb640 1 -- 172.21.15.5:0/1870536088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6ca001f020 con 0x7f6cac1021d0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e82fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e830120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac5e7955d0 con 0x55ac5e4662f0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac5e74e890 con 0x55ac5e46e4d0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ac5e791590 con 0x55ac5e48f1b0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e82fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e82fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37086/0 (socket says 172.21.15.5:37086) 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 -- 172.21.15.5:0/1479660698 learned_addr learned my addr 172.21.15.5:0/1479660698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb0e5b640 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e830120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cabfff640 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 -- 172.21.15.5:0/1479660698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 msgr2=0x55ac5e830660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 -- 172.21.15.5:0/1479660698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 msgr2=0x55ac5e830120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e830120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.280+0000 7f6cb165c640 1 -- 172.21.15.5:0/1479660698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5e4a7210 con 0x55ac5e48f1b0 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6cb165c640 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e82fbe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4035000 tx=0x7f6ca402f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a45514d40a433709 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6c96ffd640 1 -- 172.21.15.5:0/1479660698 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4031cf0 con 0x55ac5e48f1b0 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6ca9ffb640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c9c0b2950 0x7f6c9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6c96ffd640 1 -- 172.21.15.5:0/1479660698 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca4036030 con 0x55ac5e48f1b0 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6c96ffd640 1 -- 172.21.15.5:0/1479660698 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4004040 con 0x55ac5e48f1b0 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6cb0e5b640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c9c0b2950 0x7f6c9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.281+0000 7f6ca9ffb640 1 -- 172.21.15.5:0/1870536088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f6ca00e2ba0 con 0x7f6cac1021d0 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 -- 172.21.15.5:0/1479660698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 msgr2=0x55ac5e82fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb0e5b640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c9c0b2950 0x7f6c9c0b4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00dffc0 tx=0x7f6ca00e66f0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e82fbe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4035000 tx=0x7f6ca402f410 comp rx=0 tx=0).stop 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 -- 172.21.15.5:0/1479660698 shutdown_connections 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e830120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/1479660698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e82fbe0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 -- 172.21.15.5:0/1479660698 >> 172.21.15.5:0/1479660698 conn(0x55ac5e483300 msgr2=0x55ac5e82c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 -- 172.21.15.5:0/1479660698 shutdown_connections 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 -- 172.21.15.5:0/1479660698 wait complete. 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 -- start start 2024-05-14T15:33:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.282+0000 7f6cb9f70740 1 Processor -- start 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e83a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e845fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac5e83cb30 con 0x55ac5e4662f0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac5e752370 con 0x55ac5e46e4d0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ac5e83ccb0 con 0x55ac5e48f1b0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb9f70740 10 client.0 _ll_register_callbacks cb 0x55ac5e4a6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e83a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e83a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48884/0 (socket says 172.21.15.5:48884) 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 -- 172.21.15.5:0/4070693853 learned_addr learned my addr 172.21.15.5:0/4070693853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cabfff640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e845fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 -- 172.21.15.5:0/4070693853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 msgr2=0x55ac5e845fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e845fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 -- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 msgr2=0x55ac5e83a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 -- 172.21.15.5:0/4070693853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac5e573620 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6cb165c640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e83a1b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6ca402f480 tx=0x7f6ca4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eed7d5a17cd040b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.283+0000 7f6c8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6c8bfff640 1 -- 172.21.15.5:0/4070693853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4058060 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6c8bfff640 1 -- 172.21.15.5:0/4070693853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f6ca4031860 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca4036030 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6c8bfff640 1 -- 172.21.15.5:0/4070693853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ca405f050 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 10 client.17469 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac5e83cb30 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6c8bfff640 1 -- 172.21.15.5:0/4070693853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca4036030 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6c8bfff640 1 -- 172.21.15.5:0/4070693853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6ca4043070 con 0x55ac5e4662f0 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 10 client.17469 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 20 client.17469 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e44dd80 0x55ac5e84c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 4 client.17469 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.284+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac5e7a31e0 con 0x55ac5e44dd80 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.285+0000 7f6cabfff640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e44dd80 0x55ac5e84c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.285+0000 7f6c8bfff640 10 client.17469 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.285+0000 7f6cabfff640 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e44dd80 0x55ac5e84c040 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b785bea57d3a95c7 server_cookie=a56253f71f66613b in_seq=0 out_seq=0 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6c8bfff640 1 -- 172.21.15.5:0/4070693853 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac5e7a31e0 con 0x55ac5e44dd80 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6c8bfff640 10 client.17469 handle_command_reply: tid=0 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 client.17469 shutdown 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 msgr2=0x55ac5e83a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e83a1b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6ca402f480 tx=0x7f6ca4003b90 comp rx=0 tx=0).stop 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 shutdown_connections 2024-05-14T15:33:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e44dd80 0x55ac5e84c040 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e845fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/4070693853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e83a1b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6c8bfff640 0 client.17469 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 >> 172.21.15.5:0/4070693853 conn(0x55ac5e483300 msgr2=0x55ac5e82fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 shutdown_connections 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.287+0000 7f6cb9f70740 1 -- 172.21.15.5:0/4070693853 wait complete. 2024-05-14T15:33:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.288+0000 7f6cb9f70740 20 client.17469 trim_cache size 0 max 16384 2024-05-14T15:33:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.304+0000 7f6cb9f70740 1 Processor -- start 2024-05-14T15:33:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.304+0000 7f6cb9f70740 1 -- start start 2024-05-14T15:33:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.304+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 0x55ac5e83c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.304+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.304+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e48f1b0 0x55ac5e851790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.304+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1b3c40 con 0x55ac5e48f1b0 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1b1510 con 0x55ac5e4662f0 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1af380 con 0x55ac5e46e4d0 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e48f1b0 0x55ac5e851790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 0x55ac5e83c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 0x55ac5e83c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51248/0 (socket says 172.21.15.5:51248) 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:37114/0 (socket says 172.21.15.5:37114) 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb165c640 1 -- 172.21.15.5:0/614486047 learned_addr learned my addr 172.21.15.5:0/614486047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 -- 172.21.15.5:0/614486047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 msgr2=0x55ac5e83c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 0x55ac5e83c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 -- 172.21.15.5:0/614486047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e48f1b0 msgr2=0x55ac5e851790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e48f1b0 0x55ac5e851790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 -- 172.21.15.5:0/614486047 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac5e83c840 con 0x55ac5e46e4d0 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cb165c640 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 0x55ac5e83c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.305+0000 7f6cabfff640 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55ac5e83ca40 tx=0x7f6c98005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ce16266ae93bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.306+0000 7f6c967fc640 1 -- 172.21.15.5:0/614486047 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005220 con 0x55ac5e46e4d0 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.306+0000 7f6c967fc640 1 -- 172.21.15.5:0/614486047 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c98004c50 con 0x55ac5e46e4d0 2024-05-14T15:33:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.306+0000 7f6c967fc640 1 -- 172.21.15.5:0/614486047 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98021530 con 0x55ac5e46e4d0 2024-05-14T15:33:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.306+0000 7f6cb9f70740 1 -- 172.21.15.5:0/614486047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 msgr2=0x55ac5e830080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.306+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55ac5e83ca40 tx=0x7f6c98005f90 comp rx=0 tx=0).stop 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- 172.21.15.5:0/614486047 shutdown_connections 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e46e4d0 0x55ac5e830080 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e48f1b0 0x55ac5e851790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/614486047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e4662f0 0x55ac5e83c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- 172.21.15.5:0/614486047 >> 172.21.15.5:0/614486047 conn(0x55ac5e483300 msgr2=0x55ac5e82f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- 172.21.15.5:0/614486047 shutdown_connections 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- 172.21.15.5:0/614486047 wait complete. 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- start start 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 Processor -- start 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e842290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e83db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1af9d0 con 0x55ac5e4662f0 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1b0e40 con 0x55ac5e46e4d0 2024-05-14T15:33:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.307+0000 7f6cb9f70740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac1b00a0 con 0x55ac5e48f1b0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb9f70740 10 client.0 _ll_register_callbacks cb 0x7f6ca00e65c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e842290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e842290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48904/0 (socket says 172.21.15.5:48904) 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 -- 172.21.15.5:0/464133048 learned_addr learned my addr 172.21.15.5:0/464133048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cabfff640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e83db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 -- 172.21.15.5:0/464133048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 msgr2=0x55ac5e83db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e83db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 -- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 msgr2=0x55ac5e83d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 -- 172.21.15.5:0/464133048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac5e83e060 con 0x55ac5e4662f0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb165c640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e842290 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009ca0 tx=0x7f6ca4003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a86cb47a50153c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6c8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6c8bfff640 1 -- 172.21.15.5:0/464133048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4068060 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6c8bfff640 1 -- 172.21.15.5:0/464133048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f6ca4031870 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c98004c50 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6c8bfff640 1 -- 172.21.15.5:0/464133048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ca406f050 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.308+0000 7f6cb9f70740 10 client.17475 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac5e8b4000 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6c8bfff640 1 -- 172.21.15.5:0/464133048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c98004c50 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6c8bfff640 1 -- 172.21.15.5:0/464133048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6ca4043040 con 0x55ac5e4662f0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 10 client.17475 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 20 client.17475 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e4231c0 0x55ac5e8b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 4 client.17475 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac5e79d050 con 0x55ac5e4231c0 2024-05-14T15:33:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.309+0000 7f6cabfff640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e4231c0 0x55ac5e8b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.310+0000 7f6c8bfff640 10 client.17475 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.310+0000 7f6cabfff640 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e4231c0 0x55ac5e8b6670 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a079431c7077d57 server_cookie=e629e4a1781e7969 in_seq=0 out_seq=0 2024-05-14T15:33:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6c8bfff640 1 -- 172.21.15.5:0/464133048 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac5e79d050 con 0x55ac5e4231c0 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6c8bfff640 10 client.17475 handle_command_reply: tid=0 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 client.17475 shutdown 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 msgr2=0x55ac5e842290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e842290 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009ca0 tx=0x7f6ca4003af0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 shutdown_connections 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ac5e4231c0 0x55ac5e8b6670 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac5e48f1b0 0x55ac5e83db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6c8bfff640 0 client.17475 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac5e46e4d0 0x55ac5e83d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 --2- 172.21.15.5:0/464133048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac5e4662f0 0x55ac5e842290 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 >> 172.21.15.5:0/464133048 conn(0x55ac5e483300 msgr2=0x55ac5e82b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 shutdown_connections 2024-05-14T15:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.660+0000 7f6cb9f70740 1 -- 172.21.15.5:0/464133048 wait complete. 2024-05-14T15:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.661+0000 7f6cb9f70740 20 client.17475 trim_cache size 0 max 16384 2024-05-14T15:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:33:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.661+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c9c0b2950 msgr2=0x7f6c9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.661+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c9c0b2950 0x7f6c9c0b4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00dffc0 tx=0x7f6ca00e66f0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.661+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 msgr2=0x7f6cac1a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 0x7f6cac1a0530 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e980 tx=0x7f6ca000ee40 comp rx=0 tx=0).stop 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 shutdown_connections 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6c9c0b2950 0x7f6c9c0b4e60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cac10e6d0 0x7f6cac19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac109d20 0x7f6cac1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 --2- 172.21.15.5:0/1870536088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cac1021d0 0x7f6cac1a0530 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 >> 172.21.15.5:0/1870536088 conn(0x7f6cac0779e0 msgr2=0x7f6cac10a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 shutdown_connections 2024-05-14T15:33:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:35.662+0000 7f6cb30bf640 1 -- 172.21.15.5:0/1870536088 wait complete. 2024-05-14T15:33:36.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 824394 10.03 MB/sec execute 297 sec latency 216.103 ms 2024-05-14T15:33:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:35 smithi144 ceph-mon[35650]: pgmap v273: 129 pgs: 129 active+clean; 80 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-05-14T15:33:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:35 smithi193 ceph-mon[36001]: pgmap v273: 129 pgs: 129 active+clean; 80 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-05-14T15:33:36.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:35 smithi005 ceph-mon[31559]: pgmap v273: 129 pgs: 129 active+clean; 80 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-05-14T15:33:37.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 826022 10.02 MB/sec execute 298 sec latency 53.805 ms 2024-05-14T15:33:38.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 827847 10.02 MB/sec execute 299 sec latency 10.494 ms 2024-05-14T15:33:38.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:37 smithi144 ceph-mon[35650]: pgmap v274: 129 pgs: 129 active+clean; 71 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 412 op/s 2024-05-14T15:33:38.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:37 smithi193 ceph-mon[36001]: pgmap v274: 129 pgs: 129 active+clean; 71 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 412 op/s 2024-05-14T15:33:38.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:37 smithi005 ceph-mon[31559]: pgmap v274: 129 pgs: 129 active+clean; 71 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 412 op/s 2024-05-14T15:33:39.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 829679 10.01 MB/sec execute 300 sec latency 19.856 ms 2024-05-14T15:33:40.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 831515 10.02 MB/sec execute 301 sec latency 16.783 ms 2024-05-14T15:33:40.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:39 smithi144 ceph-mon[35650]: pgmap v275: 129 pgs: 129 active+clean; 70 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-14T15:33:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:39 smithi193 ceph-mon[36001]: pgmap v275: 129 pgs: 129 active+clean; 70 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-14T15:33:40.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:39 smithi005 ceph-mon[31559]: pgmap v275: 129 pgs: 129 active+clean; 70 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2024-05-14T15:33:41.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 833675 10.03 MB/sec execute 302 sec latency 40.255 ms 2024-05-14T15:33:42.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 835124 10.02 MB/sec execute 303 sec latency 52.769 ms 2024-05-14T15:33:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:41 smithi144 ceph-mon[35650]: pgmap v276: 129 pgs: 129 active+clean; 70 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-05-14T15:33:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:41 smithi193 ceph-mon[36001]: pgmap v276: 129 pgs: 129 active+clean; 70 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-05-14T15:33:42.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:41 smithi005 ceph-mon[31559]: pgmap v276: 129 pgs: 129 active+clean; 70 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-05-14T15:33:43.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 837272 10.03 MB/sec execute 304 sec latency 61.789 ms 2024-05-14T15:33:44.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 838896 10.02 MB/sec execute 305 sec latency 22.103 ms 2024-05-14T15:33:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:43 smithi144 ceph-mon[35650]: pgmap v277: 129 pgs: 129 active+clean; 77 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.5 MiB/s wr, 412 op/s 2024-05-14T15:33:44.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:43 smithi193 ceph-mon[36001]: pgmap v277: 129 pgs: 129 active+clean; 77 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.5 MiB/s wr, 412 op/s 2024-05-14T15:33:44.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:43 smithi005 ceph-mon[31559]: pgmap v277: 129 pgs: 129 active+clean; 77 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.5 MiB/s wr, 412 op/s 2024-05-14T15:33:45.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 841414 10.04 MB/sec execute 306 sec latency 23.758 ms 2024-05-14T15:33:46.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 842858 10.03 MB/sec execute 307 sec latency 15.650 ms 2024-05-14T15:33:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:45 smithi144 ceph-mon[35650]: pgmap v278: 129 pgs: 129 active+clean; 77 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 311 op/s 2024-05-14T15:33:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:45 smithi193 ceph-mon[36001]: pgmap v278: 129 pgs: 129 active+clean; 77 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 311 op/s 2024-05-14T15:33:46.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:45 smithi005 ceph-mon[31559]: pgmap v278: 129 pgs: 129 active+clean; 77 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 311 op/s 2024-05-14T15:33:47.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 844641 10.02 MB/sec execute 308 sec latency 69.886 ms 2024-05-14T15:33:48.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 846141 10.02 MB/sec execute 309 sec latency 13.967 ms 2024-05-14T15:33:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:47 smithi144 ceph-mon[35650]: pgmap v279: 129 pgs: 129 active+clean; 83 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 373 op/s 2024-05-14T15:33:48.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:47 smithi193 ceph-mon[36001]: pgmap v279: 129 pgs: 129 active+clean; 83 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 373 op/s 2024-05-14T15:33:48.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:47 smithi005 ceph-mon[31559]: pgmap v279: 129 pgs: 129 active+clean; 83 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 373 op/s 2024-05-14T15:33:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 -- 172.21.15.5:0/3878127994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 msgr2=0x7f66c8102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 --2- 172.21.15.5:0/3878127994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c8102740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009860 tx=0x7f66b802a340 comp rx=0 tx=0).stop 2024-05-14T15:33:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 -- 172.21.15.5:0/3878127994 shutdown_connections 2024-05-14T15:33:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 --2- 172.21.15.5:0/3878127994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 0x7f66c810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 --2- 172.21.15.5:0/3878127994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66c8109040 0x7f66c8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 --2- 172.21.15.5:0/3878127994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c8102740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 -- 172.21.15.5:0/3878127994 >> 172.21.15.5:0/3878127994 conn(0x7f66c80782e0 msgr2=0x7f66c8078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 -- 172.21.15.5:0/3878127994 shutdown_connections 2024-05-14T15:33:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.769+0000 7f66cd015640 1 -- 172.21.15.5:0/3878127994 wait complete. 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.770+0000 7f66cd015640 1 Processor -- start 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 -- start start 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 0x7f66c80ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c80ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66c8109040 0x7f66c80ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8112d90 con 0x7f66c8109040 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8112c10 con 0x7f66c81032f0 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66cd015640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66c8112f10 con 0x7f66c8108ba0 2024-05-14T15:33:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66c659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c80ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 0x7f66c80ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66c659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c80ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57932/0 (socket says 172.21.15.5:57932) 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66c659c640 1 -- 172.21.15.5:0/927798131 learned_addr learned my addr 172.21.15.5:0/927798131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.771+0000 7f66c759e640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66c8109040 0x7f66c80ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c6d9d640 1 -- 172.21.15.5:0/927798131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 msgr2=0x7f66c80ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c80ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c6d9d640 1 -- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66c8109040 msgr2=0x7f66c80ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66c8109040 0x7f66c80ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c6d9d640 1 -- 172.21.15.5:0/927798131 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c81a8c60 con 0x7f66c81032f0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c659c640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c80ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 0x7f66c80ff380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66b8035040 tx=0x7f66b8032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66affff640 1 -- 172.21.15.5:0/927798131 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8040020 con 0x7f66c81032f0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66affff640 1 -- 172.21.15.5:0/927798131 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b8030e10 con 0x7f66c81032f0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c81a8ef0 con 0x7f66c81032f0 2024-05-14T15:33:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66affff640 1 -- 172.21.15.5:0/927798131 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8037040 con 0x7f66c81032f0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.772+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c81a9350 con 0x7f66c81032f0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 Processor -- start 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 -- start start 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 0x55ca1edc7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edc8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca1ed2d5d0 con 0x55ca1ea064d0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca1ece6890 con 0x55ca1ea271b0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ca1ed29590 con 0x55ca1e9fe2f0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66c659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 0x55ca1edc7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 0x55ca1edc7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:56986/0 (socket says 172.21.15.5:56986) 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66c759e640 1 -- 172.21.15.5:0/3176262957 learned_addr learned my addr 172.21.15.5:0/3176262957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.775+0000 7f66affff640 1 -- 172.21.15.5:0/927798131 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f66b8043020 con 0x7f66c81032f0 2024-05-14T15:33:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c659c640 1 -- 172.21.15.5:0/3176262957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 msgr2=0x55ca1edc8120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c659c640 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edc8120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c659c640 1 -- 172.21.15.5:0/3176262957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 msgr2=0x55ca1edc7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c659c640 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 0x55ca1edc7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c659c640 1 -- 172.21.15.5:0/3176262957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1ea3f210 con 0x55ca1ea064d0 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c759e640 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 0x55ca1edc7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c659c640 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edc8660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005c80 tx=0x7f66bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a947953a9010cd3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66ad7fa640 1 -- 172.21.15.5:0/3176262957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004020 con 0x55ca1ea064d0 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66ad7fa640 1 -- 172.21.15.5:0/3176262957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66bc0054d0 con 0x55ca1ea064d0 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66ad7fa640 1 -- 172.21.15.5:0/3176262957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004df0 con 0x55ca1ea064d0 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66affff640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f66b00b2950 0x7f66b00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.776+0000 7f66c759e640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f66b00b2950 0x7f66b00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66affff640 1 -- 172.21.15.5:0/927798131 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f66b81027f0 con 0x7f66c81032f0 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 -- 172.21.15.5:0/3176262957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 msgr2=0x55ca1edc8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edc8660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f66bc005c80 tx=0x7f66bc0118d0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66c759e640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f66b00b2950 0x7f66b00b4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ca1edc4d00 tx=0x7f66c0009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 -- 172.21.15.5:0/3176262957 shutdown_connections 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edc8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edc8660 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/3176262957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea271b0 0x55ca1edc7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 -- 172.21.15.5:0/3176262957 >> 172.21.15.5:0/3176262957 conn(0x55ca1ea1b300 msgr2=0x55ca1edc4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 -- 172.21.15.5:0/3176262957 shutdown_connections 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.777+0000 7f66d3fa7740 1 -- 172.21.15.5:0/3176262957 wait complete. 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 -- start start 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 Processor -- start 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1edd26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 0x55ca1edde1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca1edd4d10 con 0x55ca1ea271b0 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca1ecea370 con 0x55ca1ea064d0 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ca1edd4e90 con 0x55ca1e9fe2f0 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66d3fa7740 10 client.0 _ll_register_callbacks cb 0x55ca1ea3ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 0x55ca1edde1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 0x55ca1edde1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54058/0 (socket says 172.21.15.5:54058) 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 -- 172.21.15.5:0/2003323293 learned_addr learned my addr 172.21.15.5:0/2003323293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c659c640 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1edd26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 -- 172.21.15.5:0/2003323293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 msgr2=0x55ca1edd21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 -- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 msgr2=0x55ca1edd26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1edd26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.778+0000 7f66c759e640 1 -- 172.21.15.5:0/2003323293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1eb0b620 con 0x55ca1ea271b0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66c759e640 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 0x55ca1edde1c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f66c0003bc0 tx=0x7f66c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dceb34ecb816069b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66aaff5640 1 -- 172.21.15.5:0/2003323293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0004310 con 0x55ca1ea271b0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66aaff5640 1 -- 172.21.15.5:0/2003323293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f66c00044b0 con 0x55ca1ea271b0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66bc0054d0 con 0x55ca1ea271b0 2024-05-14T15:33:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66aaff5640 1 -- 172.21.15.5:0/2003323293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66c00046b0 con 0x55ca1ea271b0 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66d3fa7740 10 client.17487 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ca1ede1070 con 0x55ca1ea271b0 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66aaff5640 1 -- 172.21.15.5:0/2003323293 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66bc0054d0 con 0x55ca1ea271b0 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.779+0000 7f66aaff5640 1 -- 172.21.15.5:0/2003323293 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f66c0026070 con 0x55ca1ea271b0 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66d3fa7740 10 client.17487 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66d3fa7740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66d3fa7740 20 client.17487 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1e9e5d80 0x55ca1ede4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66d3fa7740 4 client.17487 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca1ed3b1e0 con 0x55ca1e9e5d80 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1e9e5d80 0x55ca1ede4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1e9e5d80 0x55ca1ede4590 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e4f7ea270fd7b server_cookie=806db7bcc4bee144 in_seq=0 out_seq=0 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.780+0000 7f66aaff5640 10 client.17487 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.782+0000 7f66aaff5640 1 -- 172.21.15.5:0/2003323293 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca1ed3b1e0 con 0x55ca1e9e5d80 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.782+0000 7f66aaff5640 10 client.17487 handle_command_reply: tid=0 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 client.17487 shutdown 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 msgr2=0x55ca1edde1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 0x55ca1edde1c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f66c0003bc0 tx=0x7f66c0004ab0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 shutdown_connections 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1e9e5d80 0x55ca1ede4590 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea271b0 0x55ca1edde1c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1edd26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66aaff5640 0 client.17487 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2003323293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 >> 172.21.15.5:0/2003323293 conn(0x55ca1ea1b300 msgr2=0x55ca1edc9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 shutdown_connections 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2003323293 wait complete. 2024-05-14T15:33:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.783+0000 7f66d3fa7740 20 client.17487 trim_cache size 0 max 16384 2024-05-14T15:33:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.798+0000 7f66d3fa7740 1 Processor -- start 2024-05-14T15:33:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 -- start start 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 0x55ca1edc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1ede9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81b5ec0 con 0x55ca1e9e5d80 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81b3790 con 0x55ca1ea064d0 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81b1600 con 0x55ca1e9fe2f0 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 0x55ca1edc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 0x55ca1edc8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54064/0 (socket says 172.21.15.5:54064) 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 -- 172.21.15.5:0/2714395 learned_addr learned my addr 172.21.15.5:0/2714395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c759e640 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1ede9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 -- 172.21.15.5:0/2714395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 msgr2=0x55ca1edd3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 -- 172.21.15.5:0/2714395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 msgr2=0x55ca1ede9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1ede9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 -- 172.21.15.5:0/2714395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1edc4600 con 0x55ca1e9e5d80 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c759e640 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1ede9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c659c640 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.799+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 0x55ca1edc8980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f66b80ffd20 tx=0x7f66b80c5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e49c2129fa34a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.800+0000 7f66acff9640 1 -- 172.21.15.5:0/2714395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b80c9310 con 0x55ca1e9e5d80 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.800+0000 7f66acff9640 1 -- 172.21.15.5:0/2714395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b80c9f00 con 0x55ca1e9e5d80 2024-05-14T15:33:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.800+0000 7f66acff9640 1 -- 172.21.15.5:0/2714395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b80ec160 con 0x55ca1e9e5d80 2024-05-14T15:33:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.800+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2714395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 msgr2=0x55ca1edc8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 0x55ca1edc8980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f66b80ffd20 tx=0x7f66b80c5040 comp rx=0 tx=0).stop 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2714395 shutdown_connections 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1ea064d0 0x55ca1ede9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/2714395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1e9e5d80 0x55ca1edc8980 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2714395 >> 172.21.15.5:0/2714395 conn(0x55ca1ea1b300 msgr2=0x55ca1edd36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2714395 shutdown_connections 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- 172.21.15.5:0/2714395 wait complete. 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- start start 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 Processor -- start 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1e9e5d80 0x55ca1edda4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edd5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81b1c50 con 0x55ca1ea064d0 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81b30c0 con 0x55ca1e9e5d80 2024-05-14T15:33:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81b2320 con 0x55ca1e9fe2f0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.801+0000 7f66d3fa7740 10 client.0 _ll_register_callbacks cb 0x7f66c0011e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edd5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1e9e5d80 0x55ca1edda4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edd5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54068/0 (socket says 172.21.15.5:54068) 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 -- 172.21.15.5:0/1139267952 learned_addr learned my addr 172.21.15.5:0/1139267952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1e9e5d80 0x55ca1edda4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57008/0 (socket says 172.21.15.5:57008) 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c659c640 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 -- 172.21.15.5:0/1139267952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 msgr2=0x55ca1edd5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 -- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1e9e5d80 msgr2=0x55ca1edda4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1e9e5d80 0x55ca1edda4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 -- 172.21.15.5:0/1139267952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1edd62a0 con 0x55ca1ea064d0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66c759e640 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edd5d60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f66c0000f80 tx=0x7f66c0003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e2b93f165827c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:33:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66aaff5640 1 -- 172.21.15.5:0/1139267952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c003b060 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66aaff5640 1 -- 172.21.15.5:0/1139267952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f66c001ed30 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b80c9f00 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66aaff5640 1 -- 172.21.15.5:0/1139267952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66c0042050 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66d3fa7740 10 client.17493 fetch_fsmap learned FSMap version 32 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.802+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ca1ee4c640 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66aaff5640 1 -- 172.21.15.5:0/1139267952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b80c9f00 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66aaff5640 1 -- 172.21.15.5:0/1139267952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f66c0026040 con 0x55ca1ea064d0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66d3fa7740 10 client.17493 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66d3fa7740 10 client.17493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66d3fa7740 20 client.17493 populate_metadata read hostname 'smithi005' 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1ea271b0 0x55ca1ee4ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66d3fa7740 4 client.17493 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca1ed35050 con 0x55ca1ea271b0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.803+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1ea271b0 0x55ca1ee4ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:33:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.804+0000 7f66aaff5640 10 client.17493 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:48.804+0000 7f66c6d9d640 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1ea271b0 0x55ca1ee4ed10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9898ccc64d31a4d server_cookie=cfc055d3f987b0df in_seq=0 out_seq=0 2024-05-14T15:33:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.106+0000 7f66aaff5640 1 -- 172.21.15.5:0/1139267952 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca1ed35050 con 0x55ca1ea271b0 2024-05-14T15:33:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.106+0000 7f66aaff5640 10 client.17493 handle_command_reply: tid=0 2024-05-14T15:33:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.106+0000 7f66d3fa7740 1 client.17493 shutdown 2024-05-14T15:33:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 msgr2=0x55ca1edd5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edd5d60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f66c0000f80 tx=0x7f66c0003a00 comp rx=0 tx=0).stop 2024-05-14T15:33:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 shutdown_connections 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ca1ea271b0 0x55ca1ee4ed10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66aaff5640 0 client.17493 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1e9fe2f0 0x55ca1edd5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1ea064d0 0x55ca1edd5d60 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 --2- 172.21.15.5:0/1139267952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca1e9e5d80 0x55ca1edda4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 >> 172.21.15.5:0/1139267952 conn(0x55ca1ea1b300 msgr2=0x55ca1edd3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 shutdown_connections 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 1 -- 172.21.15.5:0/1139267952 wait complete. 2024-05-14T15:33:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.107+0000 7f66d3fa7740 20 client.17493 trim_cache size 0 max 16384 2024-05-14T15:33:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:33:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:33:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:33:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f66b00b2950 msgr2=0x7f66b00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f66b00b2950 0x7f66b00b4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ca1edc4d00 tx=0x7f66c0009500 comp rx=0 tx=0).stop 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 msgr2=0x7f66c80ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 0x7f66c80ff380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66b8035040 tx=0x7f66b8032d50 comp rx=0 tx=0).stop 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 shutdown_connections 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f66b00b2950 0x7f66b00b4e60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c8108ba0 0x7f66c80ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66c8109040 0x7f66c80ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 --2- 172.21.15.5:0/927798131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c81032f0 0x7f66c80ff380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:33:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 >> 172.21.15.5:0/927798131 conn(0x7f66c80782e0 msgr2=0x7f66c806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:33:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.108+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 shutdown_connections 2024-05-14T15:33:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:33:49.109+0000 7f66cd015640 1 -- 172.21.15.5:0/927798131 wait complete. 2024-05-14T15:33:49.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 847615 10.00 MB/sec execute 310 sec latency 131.274 ms 2024-05-14T15:33:50.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 849373 10.01 MB/sec execute 311 sec latency 14.533 ms 2024-05-14T15:33:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:49 smithi144 ceph-mon[35650]: pgmap v280: 129 pgs: 129 active+clean; 84 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-05-14T15:33:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:49 smithi193 ceph-mon[36001]: pgmap v280: 129 pgs: 129 active+clean; 84 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-05-14T15:33:50.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:49 smithi005 ceph-mon[31559]: pgmap v280: 129 pgs: 129 active+clean; 84 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-05-14T15:33:51.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 851584 10.01 MB/sec execute 312 sec latency 38.440 ms 2024-05-14T15:33:52.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 852916 10.00 MB/sec execute 313 sec latency 70.194 ms 2024-05-14T15:33:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:51 smithi144 ceph-mon[35650]: pgmap v281: 129 pgs: 129 active+clean; 84 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-14T15:33:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:51 smithi193 ceph-mon[36001]: pgmap v281: 129 pgs: 129 active+clean; 84 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-14T15:33:52.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:51 smithi005 ceph-mon[31559]: pgmap v281: 129 pgs: 129 active+clean; 84 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-14T15:33:53.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 854605 10.00 MB/sec execute 314 sec latency 50.922 ms 2024-05-14T15:33:54.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 856391 10.00 MB/sec execute 315 sec latency 17.179 ms 2024-05-14T15:33:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:53 smithi144 ceph-mon[35650]: pgmap v282: 129 pgs: 129 active+clean; 82 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-05-14T15:33:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:53 smithi193 ceph-mon[36001]: pgmap v282: 129 pgs: 129 active+clean; 82 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-05-14T15:33:54.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:53 smithi005 ceph-mon[31559]: pgmap v282: 129 pgs: 129 active+clean; 82 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-05-14T15:33:55.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 858269 9.99 MB/sec execute 316 sec latency 53.595 ms 2024-05-14T15:33:56.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859962 10.00 MB/sec execute 317 sec latency 16.642 ms 2024-05-14T15:33:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:55 smithi144 ceph-mon[35650]: pgmap v283: 129 pgs: 129 active+clean; 82 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-05-14T15:33:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:55 smithi193 ceph-mon[36001]: pgmap v283: 129 pgs: 129 active+clean; 82 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-05-14T15:33:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:55 smithi005 ceph-mon[31559]: pgmap v283: 129 pgs: 129 active+clean; 82 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-05-14T15:33:57.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 861598 9.99 MB/sec execute 318 sec latency 57.181 ms 2024-05-14T15:33:58.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 863254 9.99 MB/sec execute 319 sec latency 41.546 ms 2024-05-14T15:33:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:57 smithi144 ceph-mon[35650]: pgmap v284: 129 pgs: 129 active+clean; 91 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 345 op/s 2024-05-14T15:33:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:57 smithi193 ceph-mon[36001]: pgmap v284: 129 pgs: 129 active+clean; 91 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 345 op/s 2024-05-14T15:33:58.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:57 smithi005 ceph-mon[31559]: pgmap v284: 129 pgs: 129 active+clean; 91 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 345 op/s 2024-05-14T15:33:59.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 865079 9.98 MB/sec execute 320 sec latency 18.768 ms 2024-05-14T15:34:00.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866954 9.99 MB/sec execute 321 sec latency 13.003 ms 2024-05-14T15:34:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:33:59 smithi144 ceph-mon[35650]: pgmap v285: 129 pgs: 129 active+clean; 87 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-05-14T15:34:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:33:59 smithi193 ceph-mon[36001]: pgmap v285: 129 pgs: 129 active+clean; 87 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-05-14T15:34:00.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:33:59 smithi005 ceph-mon[31559]: pgmap v285: 129 pgs: 129 active+clean; 87 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-05-14T15:34:00.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:34:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:01.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 868558 9.98 MB/sec execute 322 sec latency 66.146 ms 2024-05-14T15:34:02.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 868639 9.95 MB/sec execute 323 sec latency 181.930 ms 2024-05-14T15:34:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.214+0000 7f2e50a9f640 1 -- 172.21.15.5:0/1574850222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10e210 msgr2=0x7f2e4c110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.214+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/1574850222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10e210 0x7f2e4c110700 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009ca0 tx=0x7f2e4402a9e0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 -- 172.21.15.5:0/1574850222 shutdown_connections 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/1574850222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e4c10b860 0x7f2e4c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/1574850222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10e210 0x7f2e4c110700 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/1574850222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c104e00 0x7f2e4c105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 -- 172.21.15.5:0/1574850222 >> 172.21.15.5:0/1574850222 conn(0x7f2e4c0782e0 msgr2=0x7f2e4c078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 -- 172.21.15.5:0/1574850222 shutdown_connections 2024-05-14T15:34:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 -- 172.21.15.5:0/1574850222 wait complete. 2024-05-14T15:34:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.215+0000 7f2e50a9f640 1 Processor -- start 2024-05-14T15:34:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 -- start start 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c104e00 0x7f2e4c1026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 0x7f2e4c102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e4c10e210 0x7f2e4c103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c113e70 con 0x7f2e4c10b860 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c113cf0 con 0x7f2e4c104e00 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e50a9f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c113ff0 con 0x7f2e4c10e210 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 0x7f2e4c102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 0x7f2e4c102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49874/0 (socket says 172.21.15.5:49874) 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e41d9b640 1 -- 172.21.15.5:0/196647361 learned_addr learned my addr 172.21.15.5:0/196647361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c104e00 0x7f2e4c1026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.216+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e4c10e210 0x7f2e4c103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e41d9b640 1 -- 172.21.15.5:0/196647361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e4c10e210 msgr2=0x7f2e4c103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e4c10e210 0x7f2e4c103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e41d9b640 1 -- 172.21.15.5:0/196647361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c104e00 msgr2=0x7f2e4c1026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c104e00 0x7f2e4c1026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e41d9b640 1 -- 172.21.15.5:0/196647361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c1a8b90 con 0x7f2e4c10b860 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 0x7f2e4c102c00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9f0 tx=0x7f2e3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e43fff640 1 -- 172.21.15.5:0/196647361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400cd80 con 0x7f2e4c10b860 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e43fff640 1 -- 172.21.15.5:0/196647361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e340054d0 con 0x7f2e4c10b860 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c1a8e80 con 0x7f2e4c10b860 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e43fff640 1 -- 172.21.15.5:0/196647361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34010690 con 0x7f2e4c10b860 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.217+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1a9290 con 0x7f2e4c10b860 2024-05-14T15:34:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.220+0000 7f2e43fff640 1 -- 172.21.15.5:0/196647361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2e34004e90 con 0x7f2e4c10b860 2024-05-14T15:34:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.220+0000 7f2e57a2c740 1 Processor -- start 2024-05-14T15:34:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.220+0000 7f2e57a2c740 1 -- start start 2024-05-14T15:34:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b705be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b706120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b706660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55771b66b5d0 con 0x55771b33c2f0 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55771b624890 con 0x55771b3444d0 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55771b667590 con 0x55771b3651b0 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b706120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b705be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b706660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b705be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:41030/0 (socket says 172.21.15.5:41030) 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b706120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49880/0 (socket says 172.21.15.5:49880) 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 -- 172.21.15.5:0/2760473066 learned_addr learned my addr 172.21.15.5:0/2760473066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 -- 172.21.15.5:0/2760473066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 msgr2=0x55771b706660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b706660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 -- 172.21.15.5:0/2760473066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 msgr2=0x55771b706120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b706120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e43fff640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2e380b2950 0x7f2e380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4ad9d640 1 -- 172.21.15.5:0/2760473066 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771b37d210 con 0x55771b3651b0 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.221+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b706120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.222+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2e380b2950 0x7f2e380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.222+0000 7f2e43fff640 1 -- 172.21.15.5:0/196647361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f2e340e2d50 con 0x7f2e4c10b860 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.222+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b705be0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009390 tx=0x7f2e440093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33492bcf44c8ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.222+0000 7f2e40d99640 1 -- 172.21.15.5:0/2760473066 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44033040 con 0x55771b3651b0 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.222+0000 7f2e40d99640 1 -- 172.21.15.5:0/2760473066 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4402fbd0 con 0x55771b3651b0 2024-05-14T15:34:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.222+0000 7f2e40d99640 1 -- 172.21.15.5:0/2760473066 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44044400 con 0x55771b3651b0 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2760473066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 msgr2=0x55771b705be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2e380b2950 0x7f2e380b4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2e4404b000 tx=0x7f2e4404e000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b705be0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009390 tx=0x7f2e440093c0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2760473066 shutdown_connections 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b706660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b706120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2760473066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b705be0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2760473066 >> 172.21.15.5:0/2760473066 conn(0x55771b359300 msgr2=0x55771b702340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2760473066 shutdown_connections 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.223+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2760473066 wait complete. 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 -- start start 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 Processor -- start 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b710230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b710770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b716190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55771b712d10 con 0x55771b33c2f0 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55771b628370 con 0x55771b3444d0 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55771b712e90 con 0x55771b3651b0 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e57a2c740 10 client.0 _ll_register_callbacks cb 0x55771b37ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b710230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b710230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49894/0 (socket says 172.21.15.5:49894) 2024-05-14T15:34:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 -- 172.21.15.5:0/4057859343 learned_addr learned my addr 172.21.15.5:0/4057859343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b710770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b716190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 -- 172.21.15.5:0/4057859343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 msgr2=0x55771b716190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b716190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 -- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 msgr2=0x55771b710770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b710770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.224+0000 7f2e4a59c640 1 -- 172.21.15.5:0/4057859343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771b449620 con 0x55771b33c2f0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b710230 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c012700 tx=0x7f2e3c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a15a8629f81710 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e31ffb640 1 -- 172.21.15.5:0/4057859343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00f920 con 0x55771b33c2f0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e31ffb640 1 -- 172.21.15.5:0/4057859343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f2e3c00fac0 con 0x55771b33c2f0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4402fbd0 con 0x55771b33c2f0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e31ffb640 1 -- 172.21.15.5:0/4057859343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c00fcc0 con 0x55771b33c2f0 2024-05-14T15:34:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e57a2c740 10 client.17505 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55771b71f120 con 0x55771b33c2f0 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e31ffb640 1 -- 172.21.15.5:0/4057859343 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4402fbd0 con 0x55771b33c2f0 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.225+0000 7f2e31ffb640 1 -- 172.21.15.5:0/4057859343 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2e3c01c420 con 0x55771b33c2f0 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e57a2c740 10 client.17505 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e57a2c740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e57a2c740 20 client.17505 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b323d80 0x55771b722680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e57a2c740 4 client.17505 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55771b6791e0 con 0x55771b323d80 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b323d80 0x55771b722680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e31ffb640 10 client.17505 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.226+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b323d80 0x55771b722680 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c39c36cd7dc380 server_cookie=6ecfeed6ef54c90f in_seq=0 out_seq=0 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e31ffb640 1 -- 172.21.15.5:0/4057859343 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55771b6791e0 con 0x55771b323d80 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e31ffb640 10 client.17505 handle_command_reply: tid=0 2024-05-14T15:34:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 client.17505 shutdown 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 msgr2=0x55771b710230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b710230 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c012700 tx=0x7f2e3c012bc0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 shutdown_connections 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b323d80 0x55771b722680 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b716190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b710770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e31ffb640 0 client.17505 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/4057859343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b710230 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 >> 172.21.15.5:0/4057859343 conn(0x55771b359300 msgr2=0x55771b705a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 shutdown_connections 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.227+0000 7f2e57a2c740 1 -- 172.21.15.5:0/4057859343 wait complete. 2024-05-14T15:34:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.228+0000 7f2e57a2c740 20 client.17505 trim_cache size 0 max 16384 2024-05-14T15:34:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 Processor -- start 2024-05-14T15:34:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 -- start start 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b33c2f0 0x55771b712210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 0x55771b7063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3651b0 0x55771b701080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1b5ea0 con 0x55771b3444d0 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1b3770 con 0x55771b3651b0 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1b15e0 con 0x55771b33c2f0 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 0x55771b7063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 0x55771b7063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49896/0 (socket says 172.21.15.5:49896) 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e41d9b640 1 -- 172.21.15.5:0/2625034596 learned_addr learned my addr 172.21.15.5:0/2625034596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3651b0 0x55771b701080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.243+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b33c2f0 0x55771b712210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e41d9b640 1 -- 172.21.15.5:0/2625034596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b33c2f0 msgr2=0x55771b712210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b33c2f0 0x55771b712210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e41d9b640 1 -- 172.21.15.5:0/2625034596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3651b0 msgr2=0x55771b701080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3651b0 0x55771b701080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e41d9b640 1 -- 172.21.15.5:0/2625034596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55771b70e570 con 0x55771b3444d0 2024-05-14T15:34:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 0x55771b7063a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c6a0 tx=0x7f2e340e68a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbe97139467fe88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e33fff640 1 -- 172.21.15.5:0/2625034596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34091040 con 0x55771b3444d0 2024-05-14T15:34:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e33fff640 1 -- 172.21.15.5:0/2625034596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e340b23f0 con 0x55771b3444d0 2024-05-14T15:34:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.244+0000 7f2e33fff640 1 -- 172.21.15.5:0/2625034596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34005680 con 0x55771b3444d0 2024-05-14T15:34:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2625034596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 msgr2=0x55771b7063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 0x55771b7063a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c6a0 tx=0x7f2e340e68a0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2625034596 shutdown_connections 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3651b0 0x55771b701080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b3444d0 0x55771b7063a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2625034596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b33c2f0 0x55771b712210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2625034596 >> 172.21.15.5:0/2625034596 conn(0x55771b359300 msgr2=0x55771b700970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2625034596 shutdown_connections 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2625034596 wait complete. 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 -- start start 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.245+0000 7f2e57a2c740 1 Processor -- start 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b71c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b717cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b7181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1b1c30 con 0x55771b33c2f0 2024-05-14T15:34:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1b30a0 con 0x55771b3444d0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c1b2300 con 0x55771b3651b0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e57a2c740 10 client.0 _ll_register_callbacks cb 0x7f2e3c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b71c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b71c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49902/0 (socket says 172.21.15.5:49902) 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 -- 172.21.15.5:0/2898263366 learned_addr learned my addr 172.21.15.5:0/2898263366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e41d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b717cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4ad9d640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b7181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 -- 172.21.15.5:0/2898263366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 msgr2=0x55771b7181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b7181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 -- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 msgr2=0x55771b717cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b717cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 -- 172.21.15.5:0/2898263366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55771b718730 con 0x55771b33c2f0 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b717cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e4a59c640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b71c960 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c012420 tx=0x7f2e3c01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536528dedf29499a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.246+0000 7f2e31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e31ffb640 1 -- 172.21.15.5:0/2898263366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c009df0 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55771b673050 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e31ffb640 1 -- 172.21.15.5:0/2898263366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f2e3c01cce0 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e31ffb640 1 -- 172.21.15.5:0/2898263366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c005020 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 10 client.17511 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55771b78a600 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e31ffb640 1 -- 172.21.15.5:0/2898263366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55771b673050 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e31ffb640 1 -- 172.21.15.5:0/2898263366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f2e3c02a950 con 0x55771b33c2f0 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 10 client.17511 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 20 client.17511 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b2f91c0 0x55771b78cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 4 client.17511 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:34:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.247+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55771b5ee5f0 con 0x55771b2f91c0 2024-05-14T15:34:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.248+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b2f91c0 0x55771b78cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.248+0000 7f2e41d9b640 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b2f91c0 0x55771b78cdc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f854e14b37d60ff server_cookie=21ffc64d36695f6e in_seq=0 out_seq=0 2024-05-14T15:34:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.248+0000 7f2e31ffb640 10 client.17511 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:02.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:01 smithi144 ceph-mon[35650]: pgmap v286: 129 pgs: 129 active+clean; 87 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-05-14T15:34:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:01 smithi193 ceph-mon[36001]: pgmap v286: 129 pgs: 129 active+clean; 87 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-05-14T15:34:02.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:01 smithi005 ceph-mon[31559]: pgmap v286: 129 pgs: 129 active+clean; 87 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 283 op/s 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e31ffb640 1 -- 172.21.15.5:0/2898263366 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2e340c2040 con 0x55771b2f91c0 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e31ffb640 10 client.17511 handle_command_reply: tid=0 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 client.17511 shutdown 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 msgr2=0x55771b71c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b71c960 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c012420 tx=0x7f2e3c01c650 comp rx=0 tx=0).stop 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 shutdown_connections 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55771b2f91c0 0x55771b78cdc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55771b3651b0 0x55771b7181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55771b3444d0 0x55771b717cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 --2- 172.21.15.5:0/2898263366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55771b33c2f0 0x55771b71c960 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 >> 172.21.15.5:0/2898263366 conn(0x55771b359300 msgr2=0x55771b706100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e31ffb640 0 client.17511 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 shutdown_connections 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.572+0000 7f2e57a2c740 1 -- 172.21.15.5:0/2898263366 wait complete. 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.573+0000 7f2e57a2c740 20 client.17511 trim_cache size 0 max 16384 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:34:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.573+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2e380b2950 msgr2=0x7f2e380b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.573+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2e380b2950 0x7f2e380b4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2e4404b000 tx=0x7f2e4404e000 comp rx=0 tx=0).stop 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.573+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 msgr2=0x7f2e4c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.573+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 0x7f2e4c102c00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9f0 tx=0x7f2e3400edf0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 shutdown_connections 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2e380b2950 0x7f2e380b4e60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e4c10e210 0x7f2e4c103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4c10b860 0x7f2e4c102c00 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 --2- 172.21.15.5:0/196647361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e4c104e00 0x7f2e4c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 >> 172.21.15.5:0/196647361 conn(0x7f2e4c0782e0 msgr2=0x7f2e4c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 shutdown_connections 2024-05-14T15:34:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:02.574+0000 7f2e50a9f640 1 -- 172.21.15.5:0/196647361 wait complete. 2024-05-14T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 -- 172.21.15.5:0/3637661257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e810e8f0 msgr2=0x7fe5e8110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 --2- 172.21.15.5:0/3637661257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8110dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b0a0 tx=0x7fe5e000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 -- 172.21.15.5:0/3637661257 shutdown_connections 2024-05-14T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 --2- 172.21.15.5:0/3637661257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8110dd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 --2- 172.21.15.5:0/3637661257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e8105480 0x7fe5e8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 --2- 172.21.15.5:0/3637661257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 0x7fe5e810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 -- 172.21.15.5:0/3637661257 >> 172.21.15.5:0/3637661257 conn(0x7fe5e80fef30 msgr2=0x7fe5e8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 -- 172.21.15.5:0/3637661257 shutdown_connections 2024-05-14T15:34:02.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.977+0000 7fe5edd52700 1 -- 172.21.15.5:0/3637661257 wait complete. 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.978+0000 7fe5edd52700 1 Processor -- start 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.978+0000 7fe5edd52700 1 -- start start 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5edd52700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 0x7fe5e8103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5edd52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 0x7fe5e81036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5edd52700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5edd52700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8114450 con 0x7fe5e8105480 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5edd52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81142d0 con 0x7fe5e8105dd0 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5edd52700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81145d0 con 0x7fe5e810e8f0 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 0x7fe5e8103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 0x7fe5e8103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49910/0 (socket says 172.21.15.5:49910) 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1857699211 learned_addr learned my addr 172.21.15.5:0/1857699211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 0x7fe5e81036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1857699211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e810e8f0 msgr2=0x7fe5e8103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1857699211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 msgr2=0x7fe5e81036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 0x7fe5e81036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1857699211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e81a9190 con 0x7fe5e8105480 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 0x7fe5e81036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.979+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.980+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 0x7fe5e8103170 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e9a0 tx=0x7fe5d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.980+0000 7fe5e4ff9700 1 -- 172.21.15.5:0/1857699211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800cbd0 con 0x7fe5e8105480 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.980+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e81a9480 con 0x7fe5e8105480 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.980+0000 7fe5e4ff9700 1 -- 172.21.15.5:0/1857699211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d800cd70 con 0x7fe5e8105480 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.980+0000 7fe5e4ff9700 1 -- 172.21.15.5:0/1857699211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8018680 con 0x7fe5e8105480 2024-05-14T15:34:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.980+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e81a99c0 con 0x7fe5e8105480 2024-05-14T15:34:02.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.983+0000 7fe5e4ff9700 1 -- 172.21.15.5:0/1857699211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe5d8010430 con 0x7fe5e8105480 2024-05-14T15:34:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.984+0000 7fe5e4ff9700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5d00b29f0 0x7fe5d00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.984+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5d00b29f0 0x7fe5d00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.984+0000 7fe5e4ff9700 1 -- 172.21.15.5:0/1857699211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7fe5d80e2c80 con 0x7fe5e8105480 2024-05-14T15:34:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.984+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5d00b29f0 0x7fe5d00b4f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8111040 tx=0x7fe5dc009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 Processor -- start 2024-05-14T15:34:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 -- start start 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf7a3e40 0x55ebbf921720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 0x55ebbf925f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 0x55ebbf926880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ebbf843860 con 0x55ebbf7a3e40 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ebbf7e27c0 con 0x55ebbf921c60 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ebbf83f820 con 0x55ebbf926470 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 0x55ebbf925f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 0x55ebbf926880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf7a3e40 0x55ebbf921720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 0x55ebbf925f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:46696/0 (socket says 172.21.15.5:46696) 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 0x55ebbf926880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:41090/0 (socket says 172.21.15.5:41090) 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf7a3e40 0x55ebbf921720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49918/0 (socket says 172.21.15.5:49918) 2024-05-14T15:34:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 -- 172.21.15.5:0/1313090440 learned_addr learned my addr 172.21.15.5:0/1313090440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 -- 172.21.15.5:0/1313090440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 msgr2=0x55ebbf926880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 0x55ebbf926880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 -- 172.21.15.5:0/1313090440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf7a3e40 msgr2=0x55ebbf921720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.985+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf7a3e40 0x55ebbf921720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.986+0000 7fe5e6ffd700 1 -- 172.21.15.5:0/1313090440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbf7908d0 con 0x55ebbf921c60 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.986+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 0x55ebbf926880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.986+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 0x55ebbf925f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e720 tx=0x7fe5dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff57078d580d069 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.986+0000 7fe5cdffb700 1 -- 172.21.15.5:0/1313090440 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc026600 con 0x55ebbf921c60 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.986+0000 7fe5cdffb700 1 -- 172.21.15.5:0/1313090440 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc026c50 con 0x55ebbf921c60 2024-05-14T15:34:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.986+0000 7fe5cdffb700 1 -- 172.21.15.5:0/1313090440 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc01ec60 con 0x55ebbf921c60 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1313090440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 msgr2=0x55ebbf925f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 0x55ebbf925f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e720 tx=0x7fe5dc014000 comp rx=0 tx=0).stop 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1313090440 shutdown_connections 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf926470 0x55ebbf926880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf7a3e40 0x55ebbf921720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1313090440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf921c60 0x55ebbf925f30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1313090440 >> 172.21.15.5:0/1313090440 conn(0x55ebbf7eb600 msgr2=0x55ebbf91c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1313090440 shutdown_connections 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.987+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1313090440 wait complete. 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 -- start start 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 Processor -- start 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf92b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf92bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ebbf92e420 con 0x55ebbf926470 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ebbf7a9c30 con 0x55ebbf7a3e40 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ebbf92e5a0 con 0x55ebbf921c60 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5fc766b80 10 client.0 _ll_register_callbacks cb 0x55ebbf7db920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:46708/0 (socket says 172.21.15.5:46708) 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf92bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.988+0000 7fe5e77fe700 1 -- 172.21.15.5:0/926348706 learned_addr learned my addr 172.21.15.5:0/926348706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf92b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e77fe700 1 -- 172.21.15.5:0/926348706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 msgr2=0x55ebbf92b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf92b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e77fe700 1 -- 172.21.15.5:0/926348706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 msgr2=0x55ebbf92bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf92bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e77fe700 1 -- 172.21.15.5:0/926348706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbf7df690 con 0x55ebbf7a3e40 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf92b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80e00a0 tx=0x7fe5d80aa0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f431d6a29740ff17 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/926348706 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80b2320 con 0x55ebbf7a3e40 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/926348706 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7fe5d8010930 con 0x55ebbf7a3e40 2024-05-14T15:34:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5dc026c50 con 0x55ebbf7a3e40 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/926348706 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5d80a9690 con 0x55ebbf7a3e40 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5fc766b80 10 client.27140 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.989+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ebbf939020 con 0x55ebbf7a3e40 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/926348706 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5dc026c50 con 0x55ebbf7a3e40 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/926348706 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe5d80c4080 con 0x55ebbf7a3e40 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5fc766b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5fc766b80 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5fc766b80 20 client.27140 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf93b660 0x55ebbf93da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5fc766b80 4 client.27140 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebbf851240 con 0x55ebbf93b660 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.990+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf93b660 0x55ebbf93da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.991+0000 7fe5c6ffd700 10 client.27140 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.991+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf93b660 0x55ebbf93da80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9e6964ffba7c69 server_cookie=867e1e6aea07e16e in_seq=0 out_seq=0 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/926348706 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ebbf851240 con 0x55ebbf93b660 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5c6ffd700 10 client.27140 handle_command_reply: tid=0 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 client.27140 shutdown 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 msgr2=0x55ebbf92b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80e00a0 tx=0x7fe5d80aa0e0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 shutdown_connections 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf93b660 0x55ebbf93da80 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf92bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5c6ffd700 0 client.27140 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/926348706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf92b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 >> 172.21.15.5:0/926348706 conn(0x55ebbf7eb600 msgr2=0x55ebbf920450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 shutdown_connections 2024-05-14T15:34:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 1 -- 172.21.15.5:0/926348706 wait complete. 2024-05-14T15:34:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:02.993+0000 7fe5fc766b80 20 client.27140 trim_cache size 0 max 16384 2024-05-14T15:34:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.016+0000 7fe5fc766b80 1 Processor -- start 2024-05-14T15:34:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.016+0000 7fe5fc766b80 1 -- start start 2024-05-14T15:34:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf91c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b5200 con 0x55ebbf926470 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b2a90 con 0x55ebbf7a3e40 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b0900 con 0x55ebbf921c60 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf91c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf91c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:41110/0 (socket says 172.21.15.5:41110) 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e6ffd700 1 -- 172.21.15.5:0/274882810 learned_addr learned my addr 172.21.15.5:0/274882810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e7fff700 1 -- 172.21.15.5:0/274882810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 msgr2=0x55ebbf91c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf91c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e7fff700 1 -- 172.21.15.5:0/274882810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 msgr2=0x55ebbf92b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e7fff700 1 -- 172.21.15.5:0/274882810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbf93de40 con 0x55ebbf926470 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.017+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf91c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.018+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9374b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009fd0 tx=0x7fe5e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bc542485816843 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.018+0000 7fe5cd7fa700 1 -- 172.21.15.5:0/274882810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000bcb0 con 0x55ebbf926470 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.018+0000 7fe5cd7fa700 1 -- 172.21.15.5:0/274882810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e002b070 con 0x55ebbf926470 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.018+0000 7fe5cd7fa700 1 -- 172.21.15.5:0/274882810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0007d20 con 0x55ebbf926470 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 -- 172.21.15.5:0/274882810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 msgr2=0x55ebbf9374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9374b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009fd0 tx=0x7fe5e0009780 comp rx=0 tx=0).stop 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 -- 172.21.15.5:0/274882810 shutdown_connections 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9374b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf92b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/274882810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf91c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 -- 172.21.15.5:0/274882810 >> 172.21.15.5:0/274882810 conn(0x55ebbf7eb600 msgr2=0x55ebbf91bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 -- 172.21.15.5:0/274882810 shutdown_connections 2024-05-14T15:34:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 -- 172.21.15.5:0/274882810 wait complete. 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 -- start start 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.019+0000 7fe5fc766b80 1 Processor -- start 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf931710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf931c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b0f50 con 0x55ebbf926470 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b23c0 con 0x55ebbf7a3e40 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b1620 con 0x55ebbf921c60 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5fc766b80 10 client.0 _ll_register_callbacks cb 0x55ebbf7db920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf931710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf931710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:46730/0 (socket says 172.21.15.5:46730) 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1368916874 learned_addr learned my addr 172.21.15.5:0/1368916874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf931c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1368916874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 msgr2=0x55ebbf931c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf931c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1368916874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 msgr2=0x55ebbf9a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e77fe700 1 -- 172.21.15.5:0/1368916874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbf932470 con 0x55ebbf7a3e40 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e7fff700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.020+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf931c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5e77fe700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf931710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8000c00 tx=0x7fe5d806f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224c093bd2b4c42d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/1368916874 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80a9950 con 0x55ebbf7a3e40 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5e002b070 con 0x55ebbf7a3e40 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/1368916874 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7fe5d80a6170 con 0x55ebbf7a3e40 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/1368916874 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5d80b2320 con 0x55ebbf7a3e40 2024-05-14T15:34:03.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5fc766b80 10 client.27146 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ebbf9a6300 con 0x55ebbf7a3e40 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/1368916874 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e002b070 con 0x55ebbf7a3e40 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/1368916874 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe5d80af0c0 con 0x55ebbf7a3e40 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5fc766b80 10 client.27146 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.021+0000 7fe5fc766b80 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5fc766b80 20 client.27146 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf9a6800 0x55ebbf9a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5fc766b80 4 client.27146 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebbf84b2e0 con 0x55ebbf9a6800 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf9a6800 0x55ebbf9a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5c6ffd700 10 client.27146 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.022+0000 7fe5e6ffd700 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf9a6800 0x55ebbf9a8c20 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14e6524fe6e8278 server_cookie=93f71e6639ffa073 in_seq=0 out_seq=0 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5c6ffd700 1 -- 172.21.15.5:0/1368916874 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ebbf84b2e0 con 0x55ebbf9a6800 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5c6ffd700 10 client.27146 handle_command_reply: tid=0 2024-05-14T15:34:03.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 client.27146 shutdown 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 msgr2=0x55ebbf931710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf931710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8000c00 tx=0x7fe5d806f4e0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 shutdown_connections 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ebbf9a6800 0x55ebbf9a8c20 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebbf926470 0x55ebbf9a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbf7a3e40 0x55ebbf931710 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 --2- 172.21.15.5:0/1368916874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ebbf921c60 0x55ebbf931c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5c6ffd700 0 client.27146 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 >> 172.21.15.5:0/1368916874 conn(0x55ebbf7eb600 msgr2=0x55ebbf92c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 shutdown_connections 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 1 -- 172.21.15.5:0/1368916874 wait complete. 2024-05-14T15:34:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.023+0000 7fe5fc766b80 20 client.27146 trim_cache size 0 max 16384 2024-05-14T15:34:03.035 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:34:03.035 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:34:03.035 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:34:03.035 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:34:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5d00b29f0 msgr2=0x7fe5d00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5d00b29f0 0x7fe5d00b4f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8111040 tx=0x7fe5dc009260 comp rx=0 tx=0).stop 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 msgr2=0x7fe5e8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 0x7fe5e8103170 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800e9a0 tx=0x7fe5d800ecb0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 shutdown_connections 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe5d00b29f0 0x7fe5d00b4f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e810e8f0 0x7fe5e8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e8105480 0x7fe5e8103170 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 --2- 172.21.15.5:0/1857699211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e8105dd0 0x7fe5e81036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 >> 172.21.15.5:0/1857699211 conn(0x7fe5e80fef30 msgr2=0x7fe5e80ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 shutdown_connections 2024-05-14T15:34:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:03.024+0000 7fe5edd52700 1 -- 172.21.15.5:0/1857699211 wait complete. 2024-05-14T15:34:03.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 870002 9.95 MB/sec execute 324 sec latency 201.570 ms 2024-05-14T15:34:03.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:34:03.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:34:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:04.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:03 smithi005 ceph-mon[31559]: pgmap v287: 129 pgs: 129 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-05-14T15:34:04.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 871406 9.94 MB/sec execute 325 sec latency 24.367 ms 2024-05-14T15:34:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:03 smithi144 ceph-mon[35650]: pgmap v287: 129 pgs: 129 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-05-14T15:34:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:03 smithi193 ceph-mon[36001]: pgmap v287: 129 pgs: 129 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-05-14T15:34:05.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 873645 9.95 MB/sec execute 326 sec latency 19.946 ms 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 -- 172.21.15.5:0/313419788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f152810e930 msgr2=0x7f1528110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 --2- 172.21.15.5:0/313419788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f152810e930 0x7f1528110e10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f152000b0a0 tx=0x7f152000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 -- 172.21.15.5:0/313419788 shutdown_connections 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 --2- 172.21.15.5:0/313419788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f152810e930 0x7f1528110e10 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 --2- 172.21.15.5:0/313419788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1528105550 0x7f1528105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 --2- 172.21.15.5:0/313419788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105ea0 0x7f152810e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 -- 172.21.15.5:0/313419788 >> 172.21.15.5:0/313419788 conn(0x7f1528078e00 msgr2=0x7f1528079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 -- 172.21.15.5:0/313419788 shutdown_connections 2024-05-14T15:34:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.783+0000 7f152d87a700 1 -- 172.21.15.5:0/313419788 wait complete. 2024-05-14T15:34:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.784+0000 7f152d87a700 1 Processor -- start 2024-05-14T15:34:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.784+0000 7f152d87a700 1 -- start start 2024-05-14T15:34:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f152d87a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 0x7f1528103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f152d87a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1528105ea0 0x7f1528103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f152d87a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152810e930 0x7f1528103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f152d87a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1528114480 con 0x7f1528105ea0 2024-05-14T15:34:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f152d87a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1528114300 con 0x7f1528105550 2024-05-14T15:34:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f152d87a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1528114600 con 0x7f152810e930 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f15267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1528105ea0 0x7f1528103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 0x7f1528103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f15277fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152810e930 0x7f1528103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 0x7f1528103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50192/0 (socket says 172.21.15.5:50192) 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 -- 172.21.15.5:0/2969601302 learned_addr learned my addr 172.21.15.5:0/2969601302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f15267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1528105ea0 0x7f1528103780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36850/0 (socket says 172.21.15.5:36850) 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 -- 172.21.15.5:0/2969601302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152810e930 msgr2=0x7f1528103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152810e930 0x7f1528103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 -- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1528105ea0 msgr2=0x7f1528103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1528105ea0 0x7f1528103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f1526ffd700 1 -- 172.21.15.5:0/2969601302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281a9130 con 0x7f1528105550 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.785+0000 7f15277fe700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152810e930 0x7f1528103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.786+0000 7f1526ffd700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 0x7f1528103240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f151800d950 tx=0x7f151800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.786+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518016070 con 0x7f1528105550 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.786+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281a9420 con 0x7f1528105550 2024-05-14T15:34:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.786+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15281a99b0 con 0x7f1528105550 2024-05-14T15:34:05.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.786+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151800fd40 con 0x7f1528105550 2024-05-14T15:34:05.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.786+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518020060 con 0x7f1528105550 2024-05-14T15:34:05.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.787+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15280512c0 con 0x7f1528105550 2024-05-14T15:34:05.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.789+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1518019070 con 0x7f1528105550 2024-05-14T15:34:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.790+0000 7f152c878700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15100b29f0 0x7f15100b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.790+0000 7f15267fc700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15100b29f0 0x7f15100b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.790+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9518+0+0 (secure 0 0 0) 0x7f15180e2a30 con 0x7f1528105550 2024-05-14T15:34:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.790+0000 7f15267fc700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15100b29f0 0x7f15100b4f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15281107d0 tx=0x7f151c009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:05.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:05.794+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15180a9fd0 con 0x7f1528105550 2024-05-14T15:34:06.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:05 smithi005 ceph-mon[31559]: pgmap v288: 129 pgs: 129 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-05-14T15:34:06.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 874890 9.94 MB/sec execute 327 sec latency 44.504 ms 2024-05-14T15:34:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.161+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f152802e5f0 con 0x7f1528105550 2024-05-14T15:34:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.165+0000 7f152c878700 1 -- 172.21.15.5:0/2969601302 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7f15180a98d0 con 0x7f1528105550 2024-05-14T15:34:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:34:06.167 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.168+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15100b29f0 msgr2=0x7f15100b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.168+0000 7f152d87a700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15100b29f0 0x7f15100b4f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15281107d0 tx=0x7f151c009260 comp rx=0 tx=0).stop 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 msgr2=0x7f1528103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 0x7f1528103240 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f151800d950 tx=0x7f151800dc60 comp rx=0 tx=0).stop 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 shutdown_connections 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15100b29f0 0x7f15100b4f00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152810e930 0x7f1528103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1528105550 0x7f1528103240 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 --2- 172.21.15.5:0/2969601302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1528105ea0 0x7f1528103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.169+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 >> 172.21.15.5:0/2969601302 conn(0x7f1528078e00 msgr2=0x7f1528100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:06.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.170+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 shutdown_connections 2024-05-14T15:34:06.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:06.170+0000 7f152d87a700 1 -- 172.21.15.5:0/2969601302 wait complete. 2024-05-14T15:34:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:34:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:05 smithi144 ceph-mon[35650]: pgmap v288: 129 pgs: 129 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-05-14T15:34:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:05 smithi193 ceph-mon[36001]: pgmap v288: 129 pgs: 129 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-05-14T15:34:07.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:34:07.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 875210 9.91 MB/sec execute 328 sec latency 128.765 ms 2024-05-14T15:34:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:06 smithi144 ceph-mon[35650]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T15:34:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:06 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2969601302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:06 smithi193 ceph-mon[36001]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T15:34:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:06 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2969601302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:07.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:06 smithi005 ceph-mon[31559]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T15:34:07.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:06 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2969601302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:08.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 875980 9.89 MB/sec execute 329 sec latency 68.714 ms 2024-05-14T15:34:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:07 smithi144 ceph-mon[35650]: pgmap v290: 129 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 100 active+clean; 79 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-05-14T15:34:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:07 smithi193 ceph-mon[36001]: pgmap v290: 129 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 100 active+clean; 79 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-05-14T15:34:08.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:07 smithi005 ceph-mon[31559]: pgmap v290: 129 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 100 active+clean; 79 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-05-14T15:34:09.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 876911 9.89 MB/sec execute 330 sec latency 43.062 ms 2024-05-14T15:34:09.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.726+0000 7f78c8757700 1 -- 172.21.15.5:0/2325665948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103250 msgr2=0x7f78c0103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.726+0000 7f78c8757700 1 --2- 172.21.15.5:0/2325665948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103250 0x7f78c0103660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f78b800b0a0 tx=0x7f78b800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.729+0000 7f78c8757700 1 -- 172.21.15.5:0/2325665948 shutdown_connections 2024-05-14T15:34:09.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.729+0000 7f78c8757700 1 --2- 172.21.15.5:0/2325665948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c01086a0 0x7f78c0108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.729+0000 7f78c8757700 1 --2- 172.21.15.5:0/2325665948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103250 0x7f78c0103660 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.729+0000 7f78c8757700 1 --2- 172.21.15.5:0/2325665948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103ba0 0x7f78c01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.729+0000 7f78c8757700 1 -- 172.21.15.5:0/2325665948 >> 172.21.15.5:0/2325665948 conn(0x7f78c00fedc0 msgr2=0x7f78c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:09.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.733+0000 7f78c8757700 1 -- 172.21.15.5:0/2325665948 shutdown_connections 2024-05-14T15:34:09.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.733+0000 7f78c8757700 1 -- 172.21.15.5:0/2325665948 wait complete. 2024-05-14T15:34:09.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.734+0000 7f78c8757700 1 Processor -- start 2024-05-14T15:34:09.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.734+0000 7f78c8757700 1 -- start start 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c8757700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 0x7f78c0117710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c8757700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103ba0 0x7f78c0110840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c8757700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c01086a0 0x7f78c0110d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c64f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 0x7f78c0117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c64f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 0x7f78c0117710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36870/0 (socket says 172.21.15.5:36870) 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c64f3700 1 -- 172.21.15.5:0/1384909733 learned_addr learned my addr 172.21.15.5:0/1384909733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78c011aac0 con 0x7f78c0103250 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78c011a940 con 0x7f78c0103ba0 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.735+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78c011ac40 con 0x7f78c01086a0 2024-05-14T15:34:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.736+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c01086a0 0x7f78c0110d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.738+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103ba0 0x7f78c0110840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.739+0000 7f78c5cf2700 1 -- 172.21.15.5:0/1384909733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c01086a0 msgr2=0x7f78c0110d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.739+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c01086a0 0x7f78c0110d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.739+0000 7f78c5cf2700 1 -- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 msgr2=0x7f78c0117710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.739+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 0x7f78c0117710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.739+0000 7f78c5cf2700 1 -- 172.21.15.5:0/1384909733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c0111540 con 0x7f78c0103ba0 2024-05-14T15:34:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.740+0000 7f78c64f3700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 0x7f78c0117710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:34:09.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.741+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103ba0 0x7f78c0110840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d750 tx=0x7f78bc00db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:09.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.741+0000 7f78b77fe700 1 -- 172.21.15.5:0/1384909733 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc016070 con 0x7f78c0103ba0 2024-05-14T15:34:09.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.741+0000 7f78b77fe700 1 -- 172.21.15.5:0/1384909733 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc004830 con 0x7f78c0103ba0 2024-05-14T15:34:09.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.741+0000 7f78b77fe700 1 -- 172.21.15.5:0/1384909733 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc016070 con 0x7f78c0103ba0 2024-05-14T15:34:09.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.741+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c0115240 con 0x7f78c0103ba0 2024-05-14T15:34:09.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.742+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c0115730 con 0x7f78c0103ba0 2024-05-14T15:34:09.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.754+0000 7f78b77fe700 1 -- 172.21.15.5:0/1384909733 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f78bc004cc0 con 0x7f78c0103ba0 2024-05-14T15:34:09.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.755+0000 7f78b77fe700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78ac0b2e40 0x7f78ac0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.755+0000 7f78b77fe700 1 -- 172.21.15.5:0/1384909733 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9650+0+0 (secure 0 0 0) 0x7f78bc004020 con 0x7f78c0103ba0 2024-05-14T15:34:09.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.756+0000 7f78c64f3700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78ac0b2e40 0x7f78ac0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.756+0000 7f78d716bb80 1 Processor -- start 2024-05-14T15:34:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 -- start start 2024-05-14T15:34:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c39c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c397220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562b9c2b8940 con 0x562b9c218e40 2024-05-14T15:34:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b9c21dda0 con 0x562b9c39cd80 2024-05-14T15:34:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.758+0000 7f78d716bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b9c2b4900 con 0x562b9c396e10 2024-05-14T15:34:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.759+0000 7f78c6cf4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c397220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.759+0000 7f78c6cf4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c397220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53394/0 (socket says 172.21.15.5:53394) 2024-05-14T15:34:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.759+0000 7f78c6cf4700 1 -- 172.21.15.5:0/2693939470 learned_addr learned my addr 172.21.15.5:0/2693939470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.761+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.762+0000 7f78c64f3700 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c39c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.763+0000 7f78c6cf4700 1 -- 172.21.15.5:0/2693939470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 msgr2=0x562b9c3968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.763+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.763+0000 7f78c6cf4700 1 -- 172.21.15.5:0/2693939470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 msgr2=0x562b9c39c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.763+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c39c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.763+0000 7f78c6cf4700 1 -- 172.21.15.5:0/2693939470 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9c2058d0 con 0x562b9c396e10 2024-05-14T15:34:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.764+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c397220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f78b000cb50 tx=0x7f78b000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add7b68622c4143e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.765+0000 7f78c64f3700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78ac0b2e40 0x7f78ac0b5350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009e60 tx=0x7f78b8009190 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.765+0000 7f78b4ff9700 1 -- 172.21.15.5:0/2693939470 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b000ed10 con 0x562b9c396e10 2024-05-14T15:34:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.765+0000 7f78b4ff9700 1 -- 172.21.15.5:0/2693939470 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b00044d0 con 0x562b9c396e10 2024-05-14T15:34:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.765+0000 7f78b4ff9700 1 -- 172.21.15.5:0/2693939470 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0010690 con 0x562b9c396e10 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.768+0000 7f78d716bb80 1 -- 172.21.15.5:0/2693939470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 msgr2=0x562b9c397220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.768+0000 7f78d716bb80 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c397220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f78b000cb50 tx=0x7f78b000ce60 comp rx=0 tx=0).stop 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.768+0000 7f78d716bb80 1 -- 172.21.15.5:0/2693939470 shutdown_connections 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.768+0000 7f78d716bb80 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c397220 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.768+0000 7f78d716bb80 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c39c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.768+0000 7f78d716bb80 1 --2- 172.21.15.5:0/2693939470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.769+0000 7f78d716bb80 1 -- 172.21.15.5:0/2693939470 >> 172.21.15.5:0/2693939470 conn(0x562b9c260630 msgr2=0x562b9c391a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.771+0000 7f78d716bb80 1 -- 172.21.15.5:0/2693939470 shutdown_connections 2024-05-14T15:34:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.771+0000 7f78d716bb80 1 -- 172.21.15.5:0/2693939470 wait complete. 2024-05-14T15:34:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.772+0000 7f78d716bb80 1 -- start start 2024-05-14T15:34:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.772+0000 7f78d716bb80 1 Processor -- start 2024-05-14T15:34:09.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.773+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.773+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.774+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.774+0000 7f78d716bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562b9c3a3820 con 0x562b9c396e10 2024-05-14T15:34:09.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.774+0000 7f78d716bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b9c271ed0 con 0x562b9c218e40 2024-05-14T15:34:09.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.774+0000 7f78d716bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b9c3a39a0 con 0x562b9c39cd80 2024-05-14T15:34:09.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.774+0000 7f78d716bb80 10 client.0 _ll_register_callbacks cb 0x562b9c250950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:09.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.777+0000 7f78c64f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.777+0000 7f78c5cf2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.777+0000 7f78c64f3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50230/0 (socket says 172.21.15.5:50230) 2024-05-14T15:34:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.777+0000 7f78c64f3700 1 -- 172.21.15.5:0/1705738692 learned_addr learned my addr 172.21.15.5:0/1705738692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c64f3700 1 -- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 msgr2=0x562b9c3a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c64f3700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c64f3700 1 -- 172.21.15.5:0/1705738692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 msgr2=0x562b9c3a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c64f3700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c64f3700 1 -- 172.21.15.5:0/1705738692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9c26f760 con 0x562b9c218e40 2024-05-14T15:34:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78c64f3700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3a04e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78b8007b70 tx=0x7f78b800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc80bef0df277c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:09.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.778+0000 7f78a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.779+0000 7f78a9ffb700 1 -- 172.21.15.5:0/1705738692 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8024070 con 0x562b9c218e40 2024-05-14T15:34:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.779+0000 7f78a9ffb700 1 -- 172.21.15.5:0/1705738692 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9650+0+0 (secure 0 0 0) 0x7f78b801eaf0 con 0x562b9c218e40 2024-05-14T15:34:09.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.779+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b00044d0 con 0x562b9c218e40 2024-05-14T15:34:09.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.780+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:09.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.780+0000 7f78a9ffb700 1 -- 172.21.15.5:0/1705738692 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f78b801ecf0 con 0x562b9c218e40 2024-05-14T15:34:09.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.781+0000 7f78d716bb80 10 client.27164 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:09.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.781+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b9c3ae4b0 con 0x562b9c218e40 2024-05-14T15:34:09.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78a9ffb700 1 -- 172.21.15.5:0/1705738692 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b00044d0 con 0x562b9c218e40 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78a9ffb700 1 -- 172.21.15.5:0/1705738692 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f78b8027070 con 0x562b9c218e40 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78d716bb80 10 client.27164 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78d716bb80 10 client.27164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78d716bb80 20 client.27164 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78d716bb80 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c3b0a40 0x562b9c3b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78d716bb80 4 client.27164 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:09.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.783+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b9c2c6320 con 0x562b9c3b0a40 2024-05-14T15:34:09.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.785+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c3b0a40 0x562b9c3b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.787+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c3b0a40 0x562b9c3b2e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f5a1a7c8b9bbc2 server_cookie=b3c1dd8faa6d3f8e in_seq=0 out_seq=0 2024-05-14T15:34:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.787+0000 7f78a9ffb700 10 client.27164 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.789+0000 7f78a9ffb700 1 -- 172.21.15.5:0/1705738692 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b9c2c6320 con 0x562b9c3b0a40 2024-05-14T15:34:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.789+0000 7f78a9ffb700 10 client.27164 handle_command_reply: tid=0 2024-05-14T15:34:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.789+0000 7f78d716bb80 1 client.27164 shutdown 2024-05-14T15:34:09.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.792+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 msgr2=0x562b9c3a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.792+0000 7f78d716bb80 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3a04e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78b8007b70 tx=0x7f78b800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 shutdown_connections 2024-05-14T15:34:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c3b0a40 0x562b9c3b2e60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3a04e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 --2- 172.21.15.5:0/1705738692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 >> 172.21.15.5:0/1705738692 conn(0x562b9c260630 msgr2=0x562b9c3958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78a9ffb700 0 client.27164 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.793+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 shutdown_connections 2024-05-14T15:34:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.794+0000 7f78d716bb80 1 -- 172.21.15.5:0/1705738692 wait complete. 2024-05-14T15:34:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.794+0000 7f78d716bb80 20 client.27164 trim_cache size 0 max 16384 2024-05-14T15:34:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.826+0000 7f78d716bb80 1 Processor -- start 2024-05-14T15:34:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.827+0000 7f78d716bb80 1 -- start start 2024-05-14T15:34:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.827+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.827+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78d716bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01d3030 con 0x562b9c396e10 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78d716bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01ae8a0 con 0x562b9c218e40 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78d716bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01ac710 con 0x562b9c39cd80 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78c5cf2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78c5cf2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36908/0 (socket says 172.21.15.5:36908) 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.828+0000 7f78c5cf2700 1 -- 172.21.15.5:0/892413921 learned_addr learned my addr 172.21.15.5:0/892413921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c64f3700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c5cf2700 1 -- 172.21.15.5:0/892413921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 msgr2=0x562b9c3b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c5cf2700 1 -- 172.21.15.5:0/892413921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 msgr2=0x562b9c3968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.830+0000 7f78c5cf2700 1 -- 172.21.15.5:0/892413921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9c3a02b0 con 0x562b9c396e10 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.831+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.831+0000 7f78c64f3700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.831+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3ac900 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0a5f60 tx=0x7f78bc0ae590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c25d6c5400f4ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.831+0000 7f78abfff700 1 -- 172.21.15.5:0/892413921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0a58b0 con 0x562b9c396e10 2024-05-14T15:34:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.831+0000 7f78abfff700 1 -- 172.21.15.5:0/892413921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc005730 con 0x562b9c396e10 2024-05-14T15:34:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.831+0000 7f78abfff700 1 -- 172.21.15.5:0/892413921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0a5a50 con 0x562b9c396e10 2024-05-14T15:34:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.833+0000 7f78d716bb80 1 -- 172.21.15.5:0/892413921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 msgr2=0x562b9c3ac900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.833+0000 7f78d716bb80 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3ac900 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0a5f60 tx=0x7f78bc0ae590 comp rx=0 tx=0).stop 2024-05-14T15:34:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.834+0000 7f78d716bb80 1 -- 172.21.15.5:0/892413921 shutdown_connections 2024-05-14T15:34:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.834+0000 7f78d716bb80 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c39cd80 0x562b9c3b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.834+0000 7f78d716bb80 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c218e40 0x562b9c3968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.834+0000 7f78d716bb80 1 --2- 172.21.15.5:0/892413921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c396e10 0x562b9c3ac900 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.834+0000 7f78d716bb80 1 -- 172.21.15.5:0/892413921 >> 172.21.15.5:0/892413921 conn(0x562b9c260630 msgr2=0x562b9c39edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.835+0000 7f78d716bb80 1 -- 172.21.15.5:0/892413921 shutdown_connections 2024-05-14T15:34:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.835+0000 7f78d716bb80 1 -- 172.21.15.5:0/892413921 wait complete. 2024-05-14T15:34:09.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.835+0000 7f78d716bb80 1 -- start start 2024-05-14T15:34:09.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.835+0000 7f78d716bb80 1 Processor -- start 2024-05-14T15:34:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c3b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c3a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01acd60 con 0x562b9c218e40 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01ae1d0 con 0x562b9c39cd80 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01ad430 con 0x562b9c396e10 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78d716bb80 10 client.0 _ll_register_callbacks cb 0x562b9c250950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78c5cf2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c3a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78c5cf2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c3a8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53418/0 (socket says 172.21.15.5:53418) 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78c5cf2700 1 -- 172.21.15.5:0/3811581613 learned_addr learned my addr 172.21.15.5:0/3811581613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.836+0000 7f78c6cf4700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c64f3700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c3b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c5cf2700 1 -- 172.21.15.5:0/3811581613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 msgr2=0x562b9c3a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c5cf2700 1 -- 172.21.15.5:0/3811581613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 msgr2=0x562b9c3b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c3b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c5cf2700 1 -- 172.21.15.5:0/3811581613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9c3b7610 con 0x562b9c396e10 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78c5cf2700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c3a8c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0a6130 tx=0x7f78bc00b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b221dd73d21f2371 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.837+0000 7f78a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.838+0000 7f78a9ffb700 1 -- 172.21.15.5:0/3811581613 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc088060 con 0x562b9c396e10 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.838+0000 7f78a9ffb700 1 -- 172.21.15.5:0/3811581613 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9650+0+0 (secure 0 0 0) 0x7f78bc0df7b0 con 0x562b9c396e10 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.838+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc005730 con 0x562b9c396e10 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.838+0000 7f78a9ffb700 1 -- 172.21.15.5:0/3811581613 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f78bc016050 con 0x562b9c396e10 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.838+0000 7f78d716bb80 10 client.27136 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.838+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b9c41a350 con 0x562b9c396e10 2024-05-14T15:34:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.839+0000 7f78a9ffb700 1 -- 172.21.15.5:0/3811581613 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78bc0df9b0 con 0x562b9c396e10 2024-05-14T15:34:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.839+0000 7f78a9ffb700 1 -- 172.21.15.5:0/3811581613 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f78bc08b080 con 0x562b9c396e10 2024-05-14T15:34:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.839+0000 7f78d716bb80 10 client.27136 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.839+0000 7f78d716bb80 10 client.27136 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.839+0000 7f78d716bb80 20 client.27136 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.840+0000 7f78d716bb80 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c41a9a0 0x562b9c41cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.840+0000 7f78d716bb80 4 client.27136 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.840+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f78bc005730 con 0x562b9c41a9a0 2024-05-14T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.841+0000 7f78c64f3700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c41a9a0 0x562b9c41cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.841+0000 7f78c64f3700 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c41a9a0 0x562b9c41cdc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd790f64b7e1f1bd server_cookie=dc77c41883faab37 in_seq=0 out_seq=0 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.842+0000 7f78a9ffb700 10 client.27136 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.842+0000 7f78a9ffb700 1 -- 172.21.15.5:0/3811581613 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f78bc005730 con 0x562b9c41a9a0 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.842+0000 7f78a9ffb700 10 client.27136 handle_command_reply: tid=0 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.842+0000 7f78d716bb80 1 client.27136 shutdown 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.842+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 msgr2=0x562b9c3a8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.842+0000 7f78d716bb80 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c3a8c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0a6130 tx=0x7f78bc00b2e0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 shutdown_connections 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78d716bb80 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x562b9c41a9a0 0x562b9c41cdc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78d716bb80 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b9c39cd80 0x562b9c3a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78d716bb80 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9c218e40 0x562b9c3b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78d716bb80 1 --2- 172.21.15.5:0/3811581613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b9c396e10 0x562b9c3a8c80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 >> 172.21.15.5:0/3811581613 conn(0x562b9c260630 msgr2=0x562b9c39be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.845+0000 7f78a9ffb700 0 client.27136 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.846+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 shutdown_connections 2024-05-14T15:34:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.846+0000 7f78d716bb80 1 -- 172.21.15.5:0/3811581613 wait complete. 2024-05-14T15:34:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.846+0000 7f78d716bb80 20 client.27136 trim_cache size 0 max 16384 2024-05-14T15:34:09.848 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.849+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78ac0b2e40 msgr2=0x7f78ac0b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.849+0000 7f78c8757700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78ac0b2e40 0x7f78ac0b5350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009e60 tx=0x7f78b8009190 comp rx=0 tx=0).stop 2024-05-14T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.849+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103ba0 msgr2=0x7f78c0110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.849+0000 7f78c8757700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103ba0 0x7f78c0110840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d750 tx=0x7f78bc00db10 comp rx=0 tx=0).stop 2024-05-14T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.850+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 shutdown_connections 2024-05-14T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.850+0000 7f78c8757700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78ac0b2e40 0x7f78ac0b5350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.850+0000 7f78c8757700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78c01086a0 0x7f78c0110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.850+0000 7f78c8757700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c0103250 0x7f78c0117710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.850+0000 7f78c8757700 1 --2- 172.21.15.5:0/1384909733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78c0103ba0 0x7f78c0110840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.850+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 >> 172.21.15.5:0/1384909733 conn(0x7f78c00fedc0 msgr2=0x7f78c0106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.851+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 shutdown_connections 2024-05-14T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:09.851+0000 7f78c8757700 1 -- 172.21.15.5:0/1384909733 wait complete. 2024-05-14T15:34:10.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:09 smithi005 ceph-mon[31559]: pgmap v291: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 80 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-05-14T15:34:10.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 877330 9.86 MB/sec execute 331 sec latency 35.399 ms 2024-05-14T15:34:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:09 smithi144 ceph-mon[35650]: pgmap v291: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 80 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-05-14T15:34:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:09 smithi193 ceph-mon[36001]: pgmap v291: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 80 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-05-14T15:34:11.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:34:11.151 INFO:tasks.workunit.client.0.smithi005.stdout: 1 877986 9.84 MB/sec execute 332 sec latency 31.540 ms 2024-05-14T15:34:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:12.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 879262 9.82 MB/sec execute 333 sec latency 56.586 ms 2024-05-14T15:34:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:11 smithi144 ceph-mon[35650]: pgmap v292: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 80 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-05-14T15:34:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:11 smithi193 ceph-mon[36001]: pgmap v292: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 80 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-05-14T15:34:12.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:11 smithi005 ceph-mon[31559]: pgmap v292: 129 pgs: 19 active+clean+snaptrim, 53 active+clean+snaptrim_wait, 57 active+clean; 80 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-05-14T15:34:13.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 880841 9.83 MB/sec execute 334 sec latency 25.539 ms 2024-05-14T15:34:14.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 882459 9.82 MB/sec execute 335 sec latency 23.866 ms 2024-05-14T15:34:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:13 smithi144 ceph-mon[35650]: pgmap v293: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 75 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-05-14T15:34:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:13 smithi144 ceph-mon[35650]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T15:34:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:13 smithi193 ceph-mon[36001]: pgmap v293: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 75 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-05-14T15:34:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:13 smithi193 ceph-mon[36001]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T15:34:14.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:13 smithi005 ceph-mon[31559]: pgmap v293: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 75 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-05-14T15:34:14.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:13 smithi005 ceph-mon[31559]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T15:34:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1606017911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 msgr2=0x7f44b8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1606017911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 0x7f44b8108200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f44a80099c0 tx=0x7f44a8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1606017911 shutdown_connections 2024-05-14T15:34:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1606017911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b8108740 0x7f44b81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1606017911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44b81034c0 0x7f44b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1606017911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 0x7f44b8108200 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1606017911 >> 172.21.15.5:0/1606017911 conn(0x7f44b80fedc0 msgr2=0x7f44b81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:14.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1606017911 shutdown_connections 2024-05-14T15:34:14.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.360+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1606017911 wait complete. 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.361+0000 7f44bf8e1700 1 Processor -- start 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.361+0000 7f44bf8e1700 1 -- start start 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bf8e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 0x7f44b819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bf8e1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 0x7f44b819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bf8e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44b8108740 0x7f44b819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bf8e1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81133a0 con 0x7f44b8107df0 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bf8e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8113220 con 0x7f44b81034c0 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bf8e1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8113520 con 0x7f44b8108740 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 0x7f44b819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 0x7f44b819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50276/0 (socket says 172.21.15.5:50276) 2024-05-14T15:34:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 -- 172.21.15.5:0/1199406699 learned_addr learned my addr 172.21.15.5:0/1199406699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bce7c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 0x7f44b819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bde7e700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44b8108740 0x7f44b819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 -- 172.21.15.5:0/1199406699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44b8108740 msgr2=0x7f44b819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44b8108740 0x7f44b819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 -- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 msgr2=0x7f44b819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 0x7f44b819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.362+0000 7f44bd67d700 1 -- 172.21.15.5:0/1199406699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b8199a90 con 0x7f44b81034c0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.363+0000 7f44bd67d700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 0x7f44b819bca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f44b400d950 tx=0x7f44b400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.363+0000 7f44ae7fc700 1 -- 172.21.15.5:0/1199406699 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4016070 con 0x7f44b81034c0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.363+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b8199d80 con 0x7f44b81034c0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.363+0000 7f44ae7fc700 1 -- 172.21.15.5:0/1199406699 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b400fd40 con 0x7f44b81034c0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.363+0000 7f44ae7fc700 1 -- 172.21.15.5:0/1199406699 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b400e3b0 con 0x7f44b81034c0 2024-05-14T15:34:14.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.363+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b819a2e0 con 0x7f44b81034c0 2024-05-14T15:34:14.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.366+0000 7f44ce2f5b80 1 Processor -- start 2024-05-14T15:34:14.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.366+0000 7f44ae7fc700 1 -- 172.21.15.5:0/1199406699 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f44b4019070 con 0x7f44b81034c0 2024-05-14T15:34:14.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 -- start start 2024-05-14T15:34:14.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd02b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd02bc60 0x55fcbd02ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd030880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fcbcf4d860 con 0x55fcbd02bc60 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fcbceec7c0 con 0x55fcbd030470 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcbcf49820 con 0x55fcbceade40 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd02b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bd67d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd02bc60 0x55fcbd02ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd02b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:53432/0 (socket says 172.21.15.5:53432) 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 -- 172.21.15.5:0/841233831 learned_addr learned my addr 172.21.15.5:0/841233831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bce7c700 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd030880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 -- 172.21.15.5:0/841233831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 msgr2=0x55fcbd030880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd030880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 -- 172.21.15.5:0/841233831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd02bc60 msgr2=0x55fcbd02ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd02bc60 0x55fcbd02ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.367+0000 7f44bde7e700 1 -- 172.21.15.5:0/841233831 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcbce9a8d0 con 0x55fcbceade40 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44ae7fc700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44a40b29f0 0x7f44a40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44bde7e700 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd02b720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f44b00115c0 tx=0x7f44b0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b04780f58baba979 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44bd67d700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44a40b29f0 0x7f44a40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44a37fe700 1 -- 172.21.15.5:0/841233831 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0019070 con 0x55fcbceade40 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44a37fe700 1 -- 172.21.15.5:0/841233831 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0004020 con 0x55fcbceade40 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44a37fe700 1 -- 172.21.15.5:0/841233831 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0007760 con 0x55fcbceade40 2024-05-14T15:34:14.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.368+0000 7f44ae7fc700 1 -- 172.21.15.5:0/1199406699 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7f44b40e24e0 con 0x7f44b81034c0 2024-05-14T15:34:14.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/841233831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 msgr2=0x55fcbd02b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd02b720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f44b00115c0 tx=0x7f44b0009920 comp rx=0 tx=0).stop 2024-05-14T15:34:14.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44bd67d700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44a40b29f0 0x7f44a40b4f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44b40df8c0 tx=0x7f44b4091040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/841233831 shutdown_connections 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd030880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd02b720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/841233831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd02bc60 0x55fcbd02ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/841233831 >> 172.21.15.5:0/841233831 conn(0x55fcbcef5600 msgr2=0x55fcbd0264b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/841233831 shutdown_connections 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.369+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/841233831 wait complete. 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 -- start start 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 Processor -- start 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd0350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 0x55fcbd0355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd035b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd038420 con 0x55fcbd030470 2024-05-14T15:34:14.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fcbceb3c30 con 0x55fcbd02bc60 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcbd0385a0 con 0x55fcbceade40 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44ce2f5b80 10 client.0 _ll_register_callbacks cb 0x55fcbcee5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd035b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44bce7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 0x55fcbd0355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44bce7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 0x55fcbd0355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50290/0 (socket says 172.21.15.5:50290) 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44bce7c700 1 -- 172.21.15.5:0/3071234438 learned_addr learned my addr 172.21.15.5:0/3071234438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.370+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd035b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36948/0 (socket says 172.21.15.5:36948) 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd0350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bce7c700 1 -- 172.21.15.5:0/3071234438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 msgr2=0x55fcbd0350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bce7c700 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd0350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bce7c700 1 -- 172.21.15.5:0/3071234438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 msgr2=0x55fcbd035b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bce7c700 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd035b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bce7c700 1 -- 172.21.15.5:0/3071234438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcbcee9690 con 0x55fcbd02bc60 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44bce7c700 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 0x55fcbd0355e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44a80099c0 tx=0x7f44a8005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669473f99e43e416 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:14.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44a0ff9700 1 -- 172.21.15.5:0/3071234438 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003bd0 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b0004020 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.371+0000 7f44a0ff9700 1 -- 172.21.15.5:0/3071234438 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7f44a8003d70 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44a0ff9700 1 -- 172.21.15.5:0/3071234438 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f44a8032050 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 10 client.27182 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fcbd042f90 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44a0ff9700 1 -- 172.21.15.5:0/3071234438 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0004020 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44a0ff9700 1 -- 172.21.15.5:0/3071234438 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f44a8024470 con 0x55fcbd02bc60 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 10 client.27182 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 10 client.27182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 20 client.27182 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd045570 0x55fcbd047990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 4 client.27182 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcbcf5b240 con 0x55fcbd045570 2024-05-14T15:34:14.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.372+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd045570 0x55fcbd047990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44a0ff9700 10 client.27182 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd045570 0x55fcbd047990 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b99435eefc73afa2 server_cookie=79fde3cdaef808ed in_seq=0 out_seq=0 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44a0ff9700 1 -- 172.21.15.5:0/3071234438 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fcbcf5b240 con 0x55fcbd045570 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44a0ff9700 10 client.27182 handle_command_reply: tid=0 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44ce2f5b80 1 client.27182 shutdown 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 msgr2=0x55fcbd0355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.373+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 0x55fcbd0355e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44a80099c0 tx=0x7f44a8005790 comp rx=0 tx=0).stop 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 shutdown_connections 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd045570 0x55fcbd047990 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd035b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbceade40 0x55fcbd0350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3071234438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd02bc60 0x55fcbd0355e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44a0ff9700 0 client.27182 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 >> 172.21.15.5:0/3071234438 conn(0x55fcbcef5600 msgr2=0x55fcbd02a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 shutdown_connections 2024-05-14T15:34:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3071234438 wait complete. 2024-05-14T15:34:14.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.374+0000 7f44ce2f5b80 20 client.27182 trim_cache size 0 max 16384 2024-05-14T15:34:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.389+0000 7f44ce2f5b80 1 Processor -- start 2024-05-14T15:34:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.389+0000 7f44ce2f5b80 1 -- start start 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 0x55fcbd02fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd035960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd0414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b5230 con 0x55fcbceade40 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b2ac0 con 0x55fcbd030470 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b0910 con 0x55fcbd02bc60 2024-05-14T15:34:14.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 0x55fcbd02fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 0x55fcbd02fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36956/0 (socket says 172.21.15.5:36956) 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 -- 172.21.15.5:0/3757695360 learned_addr learned my addr 172.21.15.5:0/3757695360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd0414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bce7c700 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd035960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 -- 172.21.15.5:0/3757695360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 msgr2=0x55fcbd035960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd035960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 -- 172.21.15.5:0/3757695360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 msgr2=0x55fcbd0414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd0414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.390+0000 7f44bde7e700 1 -- 172.21.15.5:0/3757695360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcbd027dc0 con 0x55fcbceade40 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.391+0000 7f44bde7e700 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 0x55fcbd02fdc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f44b0000f60 tx=0x7f44b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d15cfa2118f928e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.391+0000 7f44a2ffd700 1 -- 172.21.15.5:0/3757695360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0019040 con 0x55fcbceade40 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.391+0000 7f44a2ffd700 1 -- 172.21.15.5:0/3757695360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0003a00 con 0x55fcbceade40 2024-05-14T15:34:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.391+0000 7f44a2ffd700 1 -- 172.21.15.5:0/3757695360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b002f6c0 con 0x55fcbceade40 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3757695360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 msgr2=0x55fcbd02fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 0x55fcbd02fdc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f44b0000f60 tx=0x7f44b000b040 comp rx=0 tx=0).stop 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3757695360 shutdown_connections 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbd030470 0x55fcbd0414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbceade40 0x55fcbd02fdc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/3757695360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd035960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3757695360 >> 172.21.15.5:0/3757695360 conn(0x55fcbcef5600 msgr2=0x55fcbd02f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3757695360 shutdown_connections 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/3757695360 wait complete. 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 -- start start 2024-05-14T15:34:14.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.392+0000 7f44ce2f5b80 1 Processor -- start 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbceade40 0x55fcbd03d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd03dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd03aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b0f80 con 0x55fcbd030470 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b23f0 con 0x55fcbceade40 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b1650 con 0x55fcbd02bc60 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44ce2f5b80 10 client.0 _ll_register_callbacks cb 0x55fcbcee5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbceade40 0x55fcbd03d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd03aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44bde7e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd03aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36968/0 (socket says 172.21.15.5:36968) 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44bde7e700 1 -- 172.21.15.5:0/2488372624 learned_addr learned my addr 172.21.15.5:0/2488372624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:14.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.393+0000 7f44bce7c700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd03dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bde7e700 1 -- 172.21.15.5:0/2488372624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 msgr2=0x55fcbd03dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bde7e700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd03dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bde7e700 1 -- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbceade40 msgr2=0x55fcbd03d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bde7e700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbceade40 0x55fcbd03d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bde7e700 1 -- 172.21.15.5:0/2488372624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcbd03b690 con 0x55fcbd030470 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bce7c700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd03dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44bde7e700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd03aef0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x55fcbd04db20 tx=0x7f44b0004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f092bdda219537eb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44a0ff9700 1 -- 172.21.15.5:0/2488372624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0019070 con 0x55fcbd030470 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44a0ff9700 1 -- 172.21.15.5:0/2488372624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7f44b0003c90 con 0x55fcbd030470 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b0003a00 con 0x55fcbd030470 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44a0ff9700 1 -- 172.21.15.5:0/2488372624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f44b0003e90 con 0x55fcbd030470 2024-05-14T15:34:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44ce2f5b80 10 client.17568 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.394+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fcbd0af800 con 0x55fcbd030470 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44a0ff9700 1 -- 172.21.15.5:0/2488372624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0003a00 con 0x55fcbd030470 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44a0ff9700 1 -- 172.21.15.5:0/2488372624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f44b00076c0 con 0x55fcbd030470 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44ce2f5b80 10 client.17568 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44ce2f5b80 10 client.17568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44ce2f5b80 20 client.17568 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd0afd90 0x55fcbd0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44ce2f5b80 4 client.17568 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcbcf552e0 con 0x55fcbd0afd90 2024-05-14T15:34:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.395+0000 7f44bce7c700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd0afd90 0x55fcbd0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44a0ff9700 10 client.17568 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44bce7c700 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd0afd90 0x55fcbd0b21b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fff9b66c468c1991 server_cookie=a37f2a4c5569d2dd in_seq=0 out_seq=0 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44a0ff9700 1 -- 172.21.15.5:0/2488372624 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcbcf552e0 con 0x55fcbd0afd90 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44a0ff9700 10 client.17568 handle_command_reply: tid=0 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 client.17568 shutdown 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 msgr2=0x55fcbd03aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd03aef0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x55fcbd04db20 tx=0x7f44b0004780 comp rx=0 tx=0).stop 2024-05-14T15:34:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 shutdown_connections 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x55fcbd0afd90 0x55fcbd0b21b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44a0ff9700 0 client.17568 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fcbd030470 0x55fcbd03aef0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcbceade40 0x55fcbd03d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 --2- 172.21.15.5:0/2488372624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcbd02bc60 0x55fcbd03dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.396+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 >> 172.21.15.5:0/2488372624 conn(0x55fcbcef5600 msgr2=0x55fcbd025be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.397+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 shutdown_connections 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.397+0000 7f44ce2f5b80 1 -- 172.21.15.5:0/2488372624 wait complete. 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.397+0000 7f44ce2f5b80 20 client.17568 trim_cache size 0 max 16384 2024-05-14T15:34:14.403 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.397+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44a40b29f0 msgr2=0x7f44a40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.397+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44a40b29f0 0x7f44a40b4f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44b40df8c0 tx=0x7f44b4091040 comp rx=0 tx=0).stop 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 msgr2=0x7f44b819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 0x7f44b819bca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f44b400d950 tx=0x7f44b400dc60 comp rx=0 tx=0).stop 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 shutdown_connections 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44a40b29f0 0x7f44a40b4f00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44b8108740 0x7f44b819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44b81034c0 0x7f44b819bca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 --2- 172.21.15.5:0/1199406699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44b8107df0 0x7f44b819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 >> 172.21.15.5:0/1199406699 conn(0x7f44b80fedc0 msgr2=0x7f44b81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:14.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 shutdown_connections 2024-05-14T15:34:14.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:14.398+0000 7f44bf8e1700 1 -- 172.21.15.5:0/1199406699 wait complete. 2024-05-14T15:34:15.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 884684 9.83 MB/sec execute 336 sec latency 12.685 ms 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 -- 172.21.15.5:0/666146667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 msgr2=0x7feb48102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 --2- 172.21.15.5:0/666146667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb48102750 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feb38009900 tx=0x7feb3802a410 comp rx=0 tx=0).stop 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 -- 172.21.15.5:0/666146667 shutdown_connections 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 --2- 172.21.15.5:0/666146667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48103300 0x7feb4810f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 --2- 172.21.15.5:0/666146667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48109050 0x7feb48102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 --2- 172.21.15.5:0/666146667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb48102750 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 -- 172.21.15.5:0/666146667 >> 172.21.15.5:0/666146667 conn(0x7feb48078280 msgr2=0x7feb480786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 -- 172.21.15.5:0/666146667 shutdown_connections 2024-05-14T15:34:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.725+0000 7feb4d89f640 1 -- 172.21.15.5:0/666146667 wait complete. 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.726+0000 7feb4d89f640 1 Processor -- start 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.726+0000 7feb4d89f640 1 -- start start 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.726+0000 7feb4d89f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48103300 0x7feb481a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.726+0000 7feb4d89f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb481a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.726+0000 7feb4d89f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48109050 0x7feb4819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.726+0000 7feb4d89f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb48113ec0 con 0x7feb48108bb0 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb481a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48103300 0x7feb481a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb477fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48109050 0x7feb4819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48103300 0x7feb481a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:32964/0 (socket says 172.21.15.5:32964) 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb46ffd640 1 -- 172.21.15.5:0/316183920 learned_addr learned my addr 172.21.15.5:0/316183920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb48113d40 con 0x7feb48103300 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb48114040 con 0x7feb48109050 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 -- 172.21.15.5:0/316183920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48109050 msgr2=0x7feb4819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48109050 0x7feb4819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 -- 172.21.15.5:0/316183920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48103300 msgr2=0x7feb481a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48103300 0x7feb481a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 -- 172.21.15.5:0/316183920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4819ae10 con 0x7feb48108bb0 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb477fe640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48109050 0x7feb4819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb3ffff640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb481a0aa0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7feb4806a440 tx=0x7feb3000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4c89d640 1 -- 172.21.15.5:0/316183920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3000cc40 con 0x7feb48108bb0 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4c89d640 1 -- 172.21.15.5:0/316183920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb3000cde0 con 0x7feb48108bb0 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4819b0a0 con 0x7feb48108bb0 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4c89d640 1 -- 172.21.15.5:0/316183920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30010430 con 0x7feb48108bb0 2024-05-14T15:34:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.727+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb481a7300 con 0x7feb48108bb0 2024-05-14T15:34:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.730+0000 7feb5482c740 1 Processor -- start 2024-05-14T15:34:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.730+0000 7feb5482c740 1 -- start start 2024-05-14T15:34:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.730+0000 7feb4c89d640 1 -- 172.21.15.5:0/316183920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7feb30003ed0 con 0x7feb48108bb0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.730+0000 7feb5482c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf2ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.730+0000 7feb5482c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 0x55fd6cf2f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb5482c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf2f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb5482c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd6ce945d0 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb5482c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fd6ce4d890 con 0x55fd6cb8e1b0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb5482c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd6ce90590 con 0x55fd6cb652f0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf2f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 0x55fd6cf2f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf2ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 0x55fd6cf2f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54754/0 (socket says 172.21.15.5:54754) 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 -- 172.21.15.5:0/3974367139 learned_addr learned my addr 172.21.15.5:0/3974367139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 -- 172.21.15.5:0/3974367139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 msgr2=0x55fd6cf2ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf2ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 -- 172.21.15.5:0/3974367139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb6d4d0 msgr2=0x55fd6cf2f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf2f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 -- 172.21.15.5:0/3974367139 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd6cba6210 con 0x55fd6cb652f0 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb4c89d640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feb340b29d0 0x7feb340b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb46ffd640 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 0x55fd6cf2f120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb38006010 tx=0x7feb38031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e879d75c0c701531 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb477fe640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feb340b29d0 0x7feb340b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.731+0000 7feb4c89d640 1 -- 172.21.15.5:0/316183920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7feb300e2d40 con 0x7feb48108bb0 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.732+0000 7feb3dffb640 1 -- 172.21.15.5:0/3974367139 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38040070 con 0x55fd6cb652f0 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.732+0000 7feb3dffb640 1 -- 172.21.15.5:0/3974367139 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb38004030 con 0x55fd6cb652f0 2024-05-14T15:34:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.732+0000 7feb3dffb640 1 -- 172.21.15.5:0/3974367139 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38035040 con 0x55fd6cb652f0 2024-05-14T15:34:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.732+0000 7feb5482c740 1 -- 172.21.15.5:0/3974367139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 msgr2=0x55fd6cf2f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.732+0000 7feb5482c740 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 0x55fd6cf2f120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb38006010 tx=0x7feb38031b10 comp rx=0 tx=0).stop 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.732+0000 7feb477fe640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feb340b29d0 0x7feb340b4ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55fd6cf2bd00 tx=0x7feb40009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 -- 172.21.15.5:0/3974367139 shutdown_connections 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb652f0 0x55fd6cf2f120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf2f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 --2- 172.21.15.5:0/3974367139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf2ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 -- 172.21.15.5:0/3974367139 >> 172.21.15.5:0/3974367139 conn(0x55fd6cb82300 msgr2=0x55fd6cf2b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 -- 172.21.15.5:0/3974367139 shutdown_connections 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 -- 172.21.15.5:0/3974367139 wait complete. 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 -- start start 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.733+0000 7feb5482c740 1 Processor -- start 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf45160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd6cf3bcb0 con 0x55fd6cb652f0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fd6ce51370 con 0x55fd6cb8e1b0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd6cf3be30 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb5482c740 10 client.0 _ll_register_callbacks cb 0x55fd6cba5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53740/0 (socket says 172.21.15.5:53740) 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb46ffd640 1 -- 172.21.15.5:0/902819150 learned_addr learned my addr 172.21.15.5:0/902819150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb477fe640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf45160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 -- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 msgr2=0x55fd6cf45160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf45160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 -- 172.21.15.5:0/902819150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 msgr2=0x55fd6cf391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 -- 172.21.15.5:0/902819150 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6cc72620 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb46ffd640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.734+0000 7feb3ffff640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf396f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb300e0070 tx=0x7feb300a5b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23667d202da832dc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb26ffd640 1 -- 172.21.15.5:0/902819150 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300a9550 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb26ffd640 1 -- 172.21.15.5:0/902819150 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7feb300a9e60 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb38004030 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb26ffd640 1 -- 172.21.15.5:0/902819150 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb300a9370 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 10 client.27166 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd6cf48040 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb26ffd640 1 -- 172.21.15.5:0/902819150 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb38004030 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb26ffd640 1 -- 172.21.15.5:0/902819150 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7feb300b25d0 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 10 client.27166 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 20 client.27166 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb4cd80 0x55fd6cf4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 4 client.27166 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.735+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd6cea21e0 con 0x55fd6cb4cd80 2024-05-14T15:34:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.736+0000 7feb46ffd640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb4cd80 0x55fd6cf4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.736+0000 7feb46ffd640 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb4cd80 0x55fd6cf4b530 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7f791323ceb971 server_cookie=e27ba9bfbf9fca4e in_seq=0 out_seq=0 2024-05-14T15:34:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.736+0000 7feb26ffd640 10 client.27166 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb26ffd640 1 -- 172.21.15.5:0/902819150 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fd6cea21e0 con 0x55fd6cb4cd80 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb26ffd640 10 client.27166 handle_command_reply: tid=0 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 client.27166 shutdown 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 msgr2=0x55fd6cf396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf396f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feb300e0070 tx=0x7feb300a5b80 comp rx=0 tx=0).stop 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 shutdown_connections 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb4cd80 0x55fd6cf4b530 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf396f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb26ffd640 0 client.27166 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf45160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 --2- 172.21.15.5:0/902819150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 >> 172.21.15.5:0/902819150 conn(0x55fd6cb82300 msgr2=0x55fd6cf2ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 shutdown_connections 2024-05-14T15:34:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 1 -- 172.21.15.5:0/902819150 wait complete. 2024-05-14T15:34:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.738+0000 7feb5482c740 20 client.27166 trim_cache size 0 max 16384 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 Processor -- start 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 -- start start 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf37f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf50d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb481b3de0 con 0x55fd6cb652f0 2024-05-14T15:34:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb481b16b0 con 0x55fd6cb8e1b0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb5482c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb481af500 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf33840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53744/0 (socket says 172.21.15.5:53744) 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 -- 172.21.15.5:0/303056169 learned_addr learned my addr 172.21.15.5:0/303056169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb477fe640 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf50d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb3ffff640 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf37f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 -- 172.21.15.5:0/303056169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 msgr2=0x55fd6cf37f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf37f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 -- 172.21.15.5:0/303056169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 msgr2=0x55fd6cf50d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf50d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 -- 172.21.15.5:0/303056169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd6cf3b840 con 0x55fd6cb652f0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb477fe640 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.754+0000 7feb46ffd640 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf33840 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feb380095d0 tx=0x7feb380051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc8acc9e070c66d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.755+0000 7feb3d7fa640 1 -- 172.21.15.5:0/303056169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38054070 con 0x55fd6cb652f0 2024-05-14T15:34:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.755+0000 7feb3d7fa640 1 -- 172.21.15.5:0/303056169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb38005640 con 0x55fd6cb652f0 2024-05-14T15:34:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.755+0000 7feb3d7fa640 1 -- 172.21.15.5:0/303056169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38040030 con 0x55fd6cb652f0 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.755+0000 7feb5482c740 1 -- 172.21.15.5:0/303056169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 msgr2=0x55fd6cf33840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.755+0000 7feb5482c740 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf33840 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7feb380095d0 tx=0x7feb380051d0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- 172.21.15.5:0/303056169 shutdown_connections 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cf37f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf50d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 --2- 172.21.15.5:0/303056169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cf33840 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- 172.21.15.5:0/303056169 >> 172.21.15.5:0/303056169 conn(0x55fd6cb82300 msgr2=0x55fd6cf37870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- 172.21.15.5:0/303056169 shutdown_connections 2024-05-14T15:34:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- 172.21.15.5:0/303056169 wait complete. 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- start start 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 Processor -- start 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cfabad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cfac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb481afb70 con 0x55fd6cb652f0 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb481b0fe0 con 0x55fd6cb8e1b0 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.756+0000 7feb5482c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb481b0240 con 0x55fd6cb6d4d0 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb5482c740 10 client.0 _ll_register_callbacks cb 0x7feb300e6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cfabad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cfabad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53758/0 (socket says 172.21.15.5:53758) 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 -- 172.21.15.5:0/3049404332 learned_addr learned my addr 172.21.15.5:0/3049404332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb477fe640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cfac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 -- 172.21.15.5:0/3049404332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 msgr2=0x55fd6cfac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cfac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 -- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 msgr2=0x55fd6cf40bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf40bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 -- 172.21.15.5:0/3049404332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6cf41180 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb46ffd640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cfabad0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7feb38052040 tx=0x7feb38031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c929686458daff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb26ffd640 1 -- 172.21.15.5:0/3049404332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38054070 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb26ffd640 1 -- 172.21.15.5:0/3049404332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7feb3804f540 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb38005640 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.757+0000 7feb26ffd640 1 -- 172.21.15.5:0/3049404332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb38063050 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 10 client.17586 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd6cfb3250 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb26ffd640 1 -- 172.21.15.5:0/3049404332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb38005640 con 0x55fd6cb652f0 2024-05-14T15:34:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb26ffd640 1 -- 172.21.15.5:0/3049404332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7feb38004d20 con 0x55fd6cb652f0 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 10 client.17586 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 20 client.17586 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb221c0 0x55fd6cfb5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 4 client.17586 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd6ce9c050 con 0x55fd6cb221c0 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.758+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb221c0 0x55fd6cfb5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.759+0000 7feb26ffd640 10 client.17586 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:15.759+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb221c0 0x55fd6cfb5530 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8f5f4a73377bb1 server_cookie=77d801f12d70e77c in_seq=0 out_seq=0 2024-05-14T15:34:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb26ffd640 1 -- 172.21.15.5:0/3049404332 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd6ce9c050 con 0x55fd6cb221c0 2024-05-14T15:34:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb26ffd640 10 client.17586 handle_command_reply: tid=0 2024-05-14T15:34:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb5482c740 1 client.17586 shutdown 2024-05-14T15:34:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 msgr2=0x55fd6cfabad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb5482c740 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cfabad0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7feb38052040 tx=0x7feb38031c30 comp rx=0 tx=0).stop 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 shutdown_connections 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb5482c740 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fd6cb221c0 0x55fd6cfb5530 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb5482c740 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd6cb6d4d0 0x55fd6cfac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.053+0000 7feb26ffd640 0 client.17586 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.054+0000 7feb5482c740 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd6cb8e1b0 0x55fd6cf40bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.054+0000 7feb5482c740 1 --2- 172.21.15.5:0/3049404332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd6cb652f0 0x55fd6cfabad0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.054+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 >> 172.21.15.5:0/3049404332 conn(0x55fd6cb82300 msgr2=0x55fd6cf3a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.054+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 shutdown_connections 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.054+0000 7feb5482c740 1 -- 172.21.15.5:0/3049404332 wait complete. 2024-05-14T15:34:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.054+0000 7feb5482c740 20 client.17586 trim_cache size 0 max 16384 2024-05-14T15:34:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:34:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:34:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:34:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feb340b29d0 msgr2=0x7feb340b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feb340b29d0 0x7feb340b4ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55fd6cf2bd00 tx=0x7feb40009500 comp rx=0 tx=0).stop 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 msgr2=0x7feb481a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb481a0aa0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7feb4806a440 tx=0x7feb3000ee50 comp rx=0 tx=0).stop 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 shutdown_connections 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7feb340b29d0 0x7feb340b4ee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb48109050 0x7feb4819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb48108bb0 0x7feb481a0aa0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 --2- 172.21.15.5:0/316183920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb48103300 0x7feb481a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 >> 172.21.15.5:0/316183920 conn(0x7feb48078280 msgr2=0x7feb480ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 shutdown_connections 2024-05-14T15:34:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:16.055+0000 7feb4d89f640 1 -- 172.21.15.5:0/316183920 wait complete. 2024-05-14T15:34:16.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 885796 9.82 MB/sec execute 337 sec latency 89.726 ms 2024-05-14T15:34:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:16 smithi144 ceph-mon[35650]: pgmap v295: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 75 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-05-14T15:34:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:16 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:34:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:16 smithi193 ceph-mon[36001]: pgmap v295: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 75 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-05-14T15:34:16.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:16 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:34:16.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:16 smithi005 ceph-mon[31559]: pgmap v295: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 75 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 334 op/s 2024-05-14T15:34:16.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:16 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:34:17.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 887730 9.82 MB/sec execute 338 sec latency 136.572 ms 2024-05-14T15:34:17.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:34:17.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:34:17.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:34:17.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:34:17.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:34:17.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:34:17.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:34:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:34:17.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:34:17.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:34:17.731 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:18.128 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:18 smithi005 ceph-mon[31559]: pgmap v296: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 74 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-05-14T15:34:18.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 887998 9.80 MB/sec execute 339 sec latency 412.261 ms 2024-05-14T15:34:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:18 smithi144 ceph-mon[35650]: pgmap v296: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 74 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-05-14T15:34:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:18 smithi193 ceph-mon[36001]: pgmap v296: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 74 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-05-14T15:34:19.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 889125 9.79 MB/sec execute 340 sec latency 42.682 ms 2024-05-14T15:34:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.575+0000 7fa956a78700 1 -- 172.21.15.5:0/3348339330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9501029c0 msgr2=0x7fa950102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.575+0000 7fa956a78700 1 --2- 172.21.15.5:0/3348339330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9501029c0 0x7fa950102dd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00aaa0 tx=0x7fa93c00adb0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 -- 172.21.15.5:0/3348339330 shutdown_connections 2024-05-14T15:34:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 --2- 172.21.15.5:0/3348339330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa950103310 0x7fa95010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 --2- 172.21.15.5:0/3348339330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 0x7fa950108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 --2- 172.21.15.5:0/3348339330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9501029c0 0x7fa950102dd0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 -- 172.21.15.5:0/3348339330 >> 172.21.15.5:0/3348339330 conn(0x7fa9500fe520 msgr2=0x7fa950100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 -- 172.21.15.5:0/3348339330 shutdown_connections 2024-05-14T15:34:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.576+0000 7fa956a78700 1 -- 172.21.15.5:0/3348339330 wait complete. 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 Processor -- start 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 -- start start 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9501029c0 0x7fa95019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 0x7fa95019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 0x7fa950198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501122e0 con 0x7fa950103310 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 0x7fa95019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa950112160 con 0x7fa9501089f0 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa956a78700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa950112460 con 0x7fa9501029c0 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 0x7fa95019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53778/0 (socket says 172.21.15.5:53778) 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa94ffff700 1 -- 172.21.15.5:0/4142220771 learned_addr learned my addr 172.21.15.5:0/4142220771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa955015700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 0x7fa950198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.577+0000 7fa954814700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9501029c0 0x7fa95019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa955015700 1 -- 172.21.15.5:0/4142220771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9501029c0 msgr2=0x7fa95019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa955015700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9501029c0 0x7fa95019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa955015700 1 -- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 msgr2=0x7fa95019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa955015700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 0x7fa95019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa955015700 1 -- 172.21.15.5:0/4142220771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa950199170 con 0x7fa9501089f0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa94ffff700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 0x7fa95019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa955015700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 0x7fa950198ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa94800efc0 tx=0x7fa94800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa94dffb700 1 -- 172.21.15.5:0/4142220771 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480092e0 con 0x7fa9501089f0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa94dffb700 1 -- 172.21.15.5:0/4142220771 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa948004bb0 con 0x7fa9501089f0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa94dffb700 1 -- 172.21.15.5:0/4142220771 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948004020 con 0x7fa9501089f0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa950199400 con 0x7fa9501089f0 2024-05-14T15:34:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.578+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9501998f0 con 0x7fa9501089f0 2024-05-14T15:34:19.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.581+0000 7fa94dffb700 1 -- 172.21.15.5:0/4142220771 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa94801b030 con 0x7fa9501089f0 2024-05-14T15:34:19.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.582+0000 7fa94dffb700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9400b2a70 0x7fa9400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.582+0000 7fa954814700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9400b2a70 0x7fa9400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.582+0000 7fa94dffb700 1 -- 172.21.15.5:0/4142220771 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7fa9480e24f0 con 0x7fa9501089f0 2024-05-14T15:34:19.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.582+0000 7fa954814700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9400b2a70 0x7fa9400b4f80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa950199c00 tx=0x7fa944009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:19.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 Processor -- start 2024-05-14T15:34:19.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 -- start start 2024-05-14T15:34:19.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951320930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951325140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951325a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559951240860 con 0x559951320e70 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599511df7c0 con 0x559951325680 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa96548cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55995123c820 con 0x5599511a0e40 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951320930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951325140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951320930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54820/0 (socket says 172.21.15.5:54820) 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 -- 172.21.15.5:0/3431943065 learned_addr learned my addr 172.21.15.5:0/3431943065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa955015700 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951325a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 -- 172.21.15.5:0/3431943065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 msgr2=0x559951325a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951325a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 -- 172.21.15.5:0/3431943065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 msgr2=0x559951325140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951325140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.583+0000 7fa954814700 1 -- 172.21.15.5:0/3431943065 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995118d8d0 con 0x5599511a0e40 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.584+0000 7fa954814700 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951320930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa944011d30 tx=0x7fa944014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b695efd501141b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:19.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.584+0000 7fa93affd700 1 -- 172.21.15.5:0/3431943065 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944011d60 con 0x5599511a0e40 2024-05-14T15:34:19.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.584+0000 7fa93affd700 1 -- 172.21.15.5:0/3431943065 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa944004030 con 0x5599511a0e40 2024-05-14T15:34:19.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.584+0000 7fa93affd700 1 -- 172.21.15.5:0/3431943065 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944003c70 con 0x5599511a0e40 2024-05-14T15:34:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 -- 172.21.15.5:0/3431943065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 msgr2=0x559951320930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951320930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa944011d30 tx=0x7fa944014000 comp rx=0 tx=0).stop 2024-05-14T15:34:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 -- 172.21.15.5:0/3431943065 shutdown_connections 2024-05-14T15:34:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951325a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951320930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 --2- 172.21.15.5:0/3431943065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951325140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 -- 172.21.15.5:0/3431943065 >> 172.21.15.5:0/3431943065 conn(0x5599511e8600 msgr2=0x5599513194b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 -- 172.21.15.5:0/3431943065 shutdown_connections 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 -- 172.21.15.5:0/3431943065 wait complete. 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 -- start start 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.585+0000 7fa96548cb80 1 Processor -- start 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 0x55995132a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x55995132a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559951325680 0x55995132ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55995132d630 con 0x5599511a0e40 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599511a6c30 con 0x559951320e70 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55995132d7b0 con 0x559951325680 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa96548cb80 10 client.0 _ll_register_callbacks cb 0x5599511d8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 0x55995132a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 0x55995132a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53788/0 (socket says 172.21.15.5:53788) 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 -- 172.21.15.5:0/4223095408 learned_addr learned my addr 172.21.15.5:0/4223095408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa94ffff700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x55995132a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa955015700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559951325680 0x55995132ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 -- 172.21.15.5:0/4223095408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559951325680 msgr2=0x55995132ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559951325680 0x55995132ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 -- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 msgr2=0x55995132a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x55995132a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa954814700 1 -- 172.21.15.5:0/4223095408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599511dc690 con 0x5599511a0e40 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.586+0000 7fa94ffff700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x55995132a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa954814700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 0x55995132a310 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa944009aa0 tx=0x7fa9440054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5672bf18b4f66780 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa92ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa92ffff700 1 -- 172.21.15.5:0/4223095408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944003870 con 0x5599511a0e40 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa92ffff700 1 -- 172.21.15.5:0/4223095408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7fa944003a10 con 0x5599511a0e40 2024-05-14T15:34:19.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa944004030 con 0x5599511a0e40 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa92ffff700 1 -- 172.21.15.5:0/4223095408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa944003c10 con 0x5599511a0e40 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa96548cb80 10 client.17598 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599513381b0 con 0x5599511a0e40 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa92ffff700 1 -- 172.21.15.5:0/4223095408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa944004030 con 0x5599511a0e40 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.587+0000 7fa92ffff700 1 -- 172.21.15.5:0/4223095408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa94402e070 con 0x5599511a0e40 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa96548cb80 10 client.17598 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa96548cb80 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa96548cb80 20 client.17598 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa96548cb80 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55995133a680 0x55995133caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa96548cb80 4 client.17598 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55995124e240 con 0x55995133a680 2024-05-14T15:34:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa94ffff700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55995133a680 0x55995133caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa92ffff700 10 client.17598 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.588+0000 7fa94ffff700 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55995133a680 0x55995133caa0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fac4f74d47838f6 server_cookie=f4a3b76bfe33164b in_seq=0 out_seq=0 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa92ffff700 1 -- 172.21.15.5:0/4223095408 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55995124e240 con 0x55995133a680 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa92ffff700 10 client.17598 handle_command_reply: tid=0 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 client.17598 shutdown 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 msgr2=0x55995132a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 0x55995132a310 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa944009aa0 tx=0x7fa9440054a0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 shutdown_connections 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55995133a680 0x55995133caa0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559951325680 0x55995132ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599511a0e40 0x55995132a310 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 --2- 172.21.15.5:0/4223095408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x55995132a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 >> 172.21.15.5:0/4223095408 conn(0x5599511e8600 msgr2=0x55995131f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa92ffff700 0 client.17598 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 shutdown_connections 2024-05-14T15:34:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.589+0000 7fa96548cb80 1 -- 172.21.15.5:0/4223095408 wait complete. 2024-05-14T15:34:19.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.590+0000 7fa96548cb80 20 client.17598 trim_cache size 0 max 16384 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.605+0000 7fa96548cb80 1 Processor -- start 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.605+0000 7fa96548cb80 1 -- start start 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x55995132abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951326aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951329330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa96548cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501b0f60 con 0x559951320e70 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa96548cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501ae7f0 con 0x559951325680 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa96548cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501ac640 con 0x5599511a0e40 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951326aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951326aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53802/0 (socket says 172.21.15.5:53802) 2024-05-14T15:34:19.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x55995132abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa94ffff700 1 -- 172.21.15.5:0/810380282 learned_addr learned my addr 172.21.15.5:0/810380282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa955015700 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951329330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 -- 172.21.15.5:0/810380282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 msgr2=0x559951329330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951329330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 -- 172.21.15.5:0/810380282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 msgr2=0x559951326aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951326aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 -- 172.21.15.5:0/810380282 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559951324310 con 0x5599511a0e40 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa94ffff700 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951326aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.606+0000 7fa954814700 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x55995132abd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa944016040 tx=0x7fa944011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f20605690c9d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.607+0000 7fa93a7fc700 1 -- 172.21.15.5:0/810380282 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440037c0 con 0x5599511a0e40 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.607+0000 7fa93a7fc700 1 -- 172.21.15.5:0/810380282 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa944003960 con 0x5599511a0e40 2024-05-14T15:34:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.607+0000 7fa93a7fc700 1 -- 172.21.15.5:0/810380282 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440306a0 con 0x5599511a0e40 2024-05-14T15:34:19.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 -- 172.21.15.5:0/810380282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 msgr2=0x55995132abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x55995132abd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa944016040 tx=0x7fa944011a50 comp rx=0 tx=0).stop 2024-05-14T15:34:19.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 -- 172.21.15.5:0/810380282 shutdown_connections 2024-05-14T15:34:19.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951325680 0x559951329330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x55995132abd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 --2- 172.21.15.5:0/810380282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951320e70 0x559951326aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 -- 172.21.15.5:0/810380282 >> 172.21.15.5:0/810380282 conn(0x5599511e8600 msgr2=0x559951328c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 -- 172.21.15.5:0/810380282 shutdown_connections 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 -- 172.21.15.5:0/810380282 wait complete. 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 -- start start 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.608+0000 7fa96548cb80 1 Processor -- start 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951332a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x559951332f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 0x559951341030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501accb0 con 0x559951325680 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501ae120 con 0x559951320e70 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501ad380 con 0x5599511a0e40 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa954814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951332a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x559951332f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa955015700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 0x559951341030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa96548cb80 10 client.0 _ll_register_callbacks cb 0x5599511d8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa954814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951332a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54850/0 (socket says 172.21.15.5:54850) 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa954814700 1 -- 172.21.15.5:0/1223853609 learned_addr learned my addr 172.21.15.5:0/1223853609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa955015700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 0x559951341030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53806/0 (socket says 172.21.15.5:53806) 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 -- 172.21.15.5:0/1223853609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 msgr2=0x559951332a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951332a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 -- 172.21.15.5:0/1223853609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 msgr2=0x559951341030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 0x559951341030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 -- 172.21.15.5:0/1223853609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559951341760 con 0x559951320e70 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa954814700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951332a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa955015700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 0x559951341030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa94ffff700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x559951332f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x559951208980 tx=0x7fa93c00c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb3b7f899179077d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.609+0000 7fa92ffff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa92ffff700 1 -- 172.21.15.5:0/1223853609 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c01b040 con 0x559951320e70 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa944003960 con 0x559951320e70 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa92ffff700 1 -- 172.21.15.5:0/1223853609 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9574+0+0 (secure 0 0 0) 0x7fa93c003a10 con 0x559951320e70 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa92ffff700 1 -- 172.21.15.5:0/1223853609 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa93c005070 con 0x559951320e70 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 10 client.27224 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559951342a40 con 0x559951320e70 2024-05-14T15:34:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa92ffff700 1 -- 172.21.15.5:0/1223853609 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa944003960 con 0x559951320e70 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa92ffff700 1 -- 172.21.15.5:0/1223853609 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa93c02ace0 con 0x559951320e70 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 10 client.27224 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 20 client.27224 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5599513a4100 0x5599513a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 4 client.27224 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.610+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599512482e0 con 0x5599513a4100 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa954814700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5599513a4100 0x5599513a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa954814700 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5599513a4100 0x5599513a6520 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccdeda63422f2c40 server_cookie=c854c8f8cb5c9f4f in_seq=0 out_seq=0 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa92ffff700 10 client.27224 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa92ffff700 1 -- 172.21.15.5:0/1223853609 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599512482e0 con 0x5599513a4100 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa92ffff700 10 client.27224 handle_command_reply: tid=0 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 client.27224 shutdown 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 msgr2=0x559951332f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x559951332f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x559951208980 tx=0x7fa93c00c650 comp rx=0 tx=0).stop 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 shutdown_connections 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5599513a4100 0x5599513a6520 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559951325680 0x559951341030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599511a0e40 0x559951332a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 --2- 172.21.15.5:0/1223853609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559951320e70 0x559951332f80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 >> 172.21.15.5:0/1223853609 conn(0x5599511e8600 msgr2=0x559951318be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.611+0000 7fa92ffff700 0 client.27224 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 shutdown_connections 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa96548cb80 1 -- 172.21.15.5:0/1223853609 wait complete. 2024-05-14T15:34:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa96548cb80 20 client.27224 trim_cache size 0 max 16384 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9400b2a70 msgr2=0x7fa9400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9400b2a70 0x7fa9400b4f80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa950199c00 tx=0x7fa944009260 comp rx=0 tx=0).stop 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 msgr2=0x7fa950198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 0x7fa950198ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa94800efc0 tx=0x7fa94800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 shutdown_connections 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9400b2a70 0x7fa9400b4f80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9501089f0 0x7fa950198ac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9501029c0 0x7fa95019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 --2- 172.21.15.5:0/4142220771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950103310 0x7fa95019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.612+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 >> 172.21.15.5:0/4142220771 conn(0x7fa9500fe520 msgr2=0x7fa950100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.613+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 shutdown_connections 2024-05-14T15:34:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:19.613+0000 7fa956a78700 1 -- 172.21.15.5:0/4142220771 wait complete. 2024-05-14T15:34:20.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 891118 9.79 MB/sec execute 341 sec latency 53.601 ms 2024-05-14T15:34:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:20 smithi144 ceph-mon[35650]: pgmap v297: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-05-14T15:34:20.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:20 smithi193 ceph-mon[36001]: pgmap v297: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-05-14T15:34:20.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:20 smithi005 ceph-mon[31559]: pgmap v297: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-05-14T15:34:21.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 892178 9.78 MB/sec execute 342 sec latency 21.342 ms 2024-05-14T15:34:21.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:21 smithi005 ceph-mon[31559]: pgmap v298: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-05-14T15:34:21.496 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:34:21.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:34:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:21 smithi144 ceph-mon[35650]: pgmap v298: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-05-14T15:34:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:21 smithi193 ceph-mon[36001]: pgmap v298: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 266 op/s 2024-05-14T15:34:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:22.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 893469 9.76 MB/sec execute 343 sec latency 20.936 ms 2024-05-14T15:34:23.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 895095 9.77 MB/sec execute 344 sec latency 12.310 ms 2024-05-14T15:34:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 -- 172.21.15.5:0/3941107355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 msgr2=0x7f782411d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 --2- 172.21.15.5:0/3941107355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f782411d3f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f78140098f0 tx=0x7f7814009c00 comp rx=0 tx=0).stop 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 -- 172.21.15.5:0/3941107355 shutdown_connections 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 --2- 172.21.15.5:0/3941107355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782411d930 0x7f782411fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 --2- 172.21.15.5:0/3941107355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782410a930 0x7f782410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 --2- 172.21.15.5:0/3941107355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f782411d3f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.641+0000 7f782c02d700 1 -- 172.21.15.5:0/3941107355 >> 172.21.15.5:0/3941107355 conn(0x7f782406f4d0 msgr2=0x7f782406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.642+0000 7f782c02d700 1 -- 172.21.15.5:0/3941107355 shutdown_connections 2024-05-14T15:34:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.642+0000 7f782c02d700 1 -- 172.21.15.5:0/3941107355 wait complete. 2024-05-14T15:34:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 Processor -- start 2024-05-14T15:34:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 -- start start 2024-05-14T15:34:23.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782410a930 0x7f78241a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f78241aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782411d930 0x7f78241ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78241232f0 con 0x7f782410b280 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7824123170 con 0x7f782410a930 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.643+0000 7f782c02d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7824123470 con 0x7f782411d930 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.644+0000 7f782a5ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782411d930 0x7f78241ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.644+0000 7f782a5ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782411d930 0x7f78241ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54866/0 (socket says 172.21.15.5:54866) 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.644+0000 7f782a5ca700 1 -- 172.21.15.5:0/3629902450 learned_addr learned my addr 172.21.15.5:0/3629902450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.644+0000 7f7829dc9700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782410a930 0x7f78241a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.644+0000 7f78295c8700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f78241aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f78295c8700 1 -- 172.21.15.5:0/3629902450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782411d930 msgr2=0x7f78241ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f78295c8700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782411d930 0x7f78241ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f78295c8700 1 -- 172.21.15.5:0/3629902450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782410a930 msgr2=0x7f78241a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f78295c8700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782410a930 0x7f78241a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f78295c8700 1 -- 172.21.15.5:0/3629902450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78241aec50 con 0x7f782410b280 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f7829dc9700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782410a930 0x7f78241a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.645+0000 7f78295c8700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f78241aa320 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7814009810 tx=0x7f781400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:23.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.646+0000 7f781affd700 1 -- 172.21.15.5:0/3629902450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781400bc40 con 0x7f782410b280 2024-05-14T15:34:23.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.646+0000 7f781affd700 1 -- 172.21.15.5:0/3629902450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781400bde0 con 0x7f782410b280 2024-05-14T15:34:23.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.646+0000 7f781affd700 1 -- 172.21.15.5:0/3629902450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814003e60 con 0x7f782410b280 2024-05-14T15:34:23.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.646+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241aeee0 con 0x7f782410b280 2024-05-14T15:34:23.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.646+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241af370 con 0x7f782410b280 2024-05-14T15:34:23.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.651+0000 7f781affd700 1 -- 172.21.15.5:0/3629902450 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7814005110 con 0x7f782410b280 2024-05-14T15:34:23.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.652+0000 7f783aa41b80 1 Processor -- start 2024-05-14T15:34:23.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.652+0000 7f783aa41b80 1 -- start start 2024-05-14T15:34:23.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.652+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce36e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce372fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce373900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f783aa41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0ce2908e0 con 0x55a0ce36ece0 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f783aa41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0ce22f840 con 0x55a0ce3734f0 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f783aa41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0ce28c8a0 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f782a5ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce36e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f782a5ca700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce36e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54874/0 (socket says 172.21.15.5:54874) 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f782a5ca700 1 -- 172.21.15.5:0/2897783021 learned_addr learned my addr 172.21.15.5:0/2897783021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.653+0000 7f781affd700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78100b2af0 0x7f78100b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f781affd700 1 -- 172.21.15.5:0/3629902450 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f78140e37b0 con 0x7f782410b280 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f782a5ca700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78100b2af0 0x7f78100b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f782a5ca700 1 -- 172.21.15.5:0/2897783021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 msgr2=0x55a0ce373900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f782a5ca700 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce373900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f782a5ca700 1 -- 172.21.15.5:0/2897783021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 msgr2=0x55a0ce372fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f782a5ca700 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce372fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.654+0000 7f782a5ca700 1 -- 172.21.15.5:0/2897783021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ce1dd950 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.655+0000 7f782a5ca700 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce36e7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f781c0060b0 tx=0x7f781c007dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07da603b0497feb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.655+0000 7f782a5ca700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78100b2af0 0x7f78100b5000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f781c01b000 tx=0x7f781c01d000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.655+0000 7f780ffff700 1 -- 172.21.15.5:0/2897783021 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c01e040 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.655+0000 7f780ffff700 1 -- 172.21.15.5:0/2897783021 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c017e50 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.656+0000 7f783aa41b80 1 -- 172.21.15.5:0/2897783021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 msgr2=0x55a0ce36e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.656+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce36e7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f781c0060b0 tx=0x7f781c007dc0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.657+0000 7f783aa41b80 1 -- 172.21.15.5:0/2897783021 shutdown_connections 2024-05-14T15:34:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.657+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce373900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.657+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce36e7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.657+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2897783021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce372fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.657+0000 7f783aa41b80 1 -- 172.21.15.5:0/2897783021 >> 172.21.15.5:0/2897783021 conn(0x55a0ce238680 msgr2=0x55a0ce369530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.659+0000 7f783aa41b80 1 -- 172.21.15.5:0/2897783021 shutdown_connections 2024-05-14T15:34:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.659+0000 7f783aa41b80 1 -- 172.21.15.5:0/2897783021 wait complete. 2024-05-14T15:34:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.660+0000 7f783aa41b80 1 -- start start 2024-05-14T15:34:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.660+0000 7f783aa41b80 1 Processor -- start 2024-05-14T15:34:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce3782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce3787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce3734f0 0x55a0ce378d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f783aa41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0ce37b440 con 0x55a0ce3734f0 2024-05-14T15:34:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f783aa41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0ce1f6cb0 con 0x55a0ce36ece0 2024-05-14T15:34:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f783aa41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0ce37b5c0 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce3782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce3782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54876/0 (socket says 172.21.15.5:54876) 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 -- 172.21.15.5:0/2077015638 learned_addr learned my addr 172.21.15.5:0/2077015638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f782a5ca700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce3734f0 0x55a0ce378d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f78295c8700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce3787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 -- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 msgr2=0x55a0ce3787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce3787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 -- 172.21.15.5:0/2077015638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce3734f0 msgr2=0x55a0ce378d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce3734f0 0x55a0ce378d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.661+0000 7f7829dc9700 1 -- 172.21.15.5:0/2077015638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ce22c710 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f783aa41b80 10 client.0 _ll_register_callbacks cb 0x55a0ce2289a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f7829dc9700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce3782a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f78200106c0 tx=0x7f78200109d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62291c316950e618 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f780d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781c017e50 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f780d7fa700 1 -- 172.21.15.5:0/2077015638 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820011910 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f780d7fa700 1 -- 172.21.15.5:0/2077015638 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f7820011ab0 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f780d7fa700 1 -- 172.21.15.5:0/2077015638 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7820025050 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f783aa41b80 10 client.27202 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55a0ce385e00 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.662+0000 7f780d7fa700 1 -- 172.21.15.5:0/2077015638 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7820011cb0 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f783aa41b80 10 client.27202 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f783aa41b80 10 client.27202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f783aa41b80 20 client.27202 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce388330 0x55a0ce38a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f783aa41b80 4 client.27202 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0ce29e2c0 con 0x55a0ce388330 2024-05-14T15:34:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f78295c8700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce388330 0x55a0ce38a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f78295c8700 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce388330 0x55a0ce38a750 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208b9f8b5b00e3dc server_cookie=1b7e3f5d0ceb45bc in_seq=0 out_seq=0 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.663+0000 7f780d7fa700 10 client.27202 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.664+0000 7f780d7fa700 1 -- 172.21.15.5:0/2077015638 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0ce29e2c0 con 0x55a0ce388330 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.664+0000 7f780d7fa700 10 client.27202 handle_command_reply: tid=0 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.664+0000 7f783aa41b80 1 client.27202 shutdown 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 msgr2=0x55a0ce3782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce3782a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f78200106c0 tx=0x7f78200109d0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 shutdown_connections 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce388330 0x55a0ce38a750 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce3734f0 0x55a0ce378d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce3782a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 --2- 172.21.15.5:0/2077015638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce3787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 >> 172.21.15.5:0/2077015638 conn(0x55a0ce238680 msgr2=0x55a0ce36d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f780d7fa700 0 client.27202 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 shutdown_connections 2024-05-14T15:34:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 1 -- 172.21.15.5:0/2077015638 wait complete. 2024-05-14T15:34:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.665+0000 7f783aa41b80 20 client.27202 trim_cache size 0 max 16384 2024-05-14T15:34:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 Processor -- start 2024-05-14T15:34:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 -- start start 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce369690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce372ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce3734f0 0x55a0ce378b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78241ca610 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78241c7ea0 con 0x55a0ce36ece0 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f783aa41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7824119880 con 0x55a0ce3734f0 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce369690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce369690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53842/0 (socket says 172.21.15.5:53842) 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 -- 172.21.15.5:0/1667817389 learned_addr learned my addr 172.21.15.5:0/1667817389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f782a5ca700 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce3734f0 0x55a0ce378b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f78295c8700 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce372ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 -- 172.21.15.5:0/1667817389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce3734f0 msgr2=0x55a0ce378b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce3734f0 0x55a0ce378b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 -- 172.21.15.5:0/1667817389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 msgr2=0x55a0ce372ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce372ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.688+0000 7f7829dc9700 1 -- 172.21.15.5:0/1667817389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ce376830 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.689+0000 7f782a5ca700 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce3734f0 0x55a0ce378b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.689+0000 7f7829dc9700 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce369690 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7820023040 tx=0x7f7820013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d5231b82938d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.689+0000 7f780f7fe700 1 -- 172.21.15.5:0/1667817389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782002e070 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.689+0000 7f780f7fe700 1 -- 172.21.15.5:0/1667817389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c017e50 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.689+0000 7f780f7fe700 1 -- 172.21.15.5:0/1667817389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820011970 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 -- 172.21.15.5:0/1667817389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 msgr2=0x55a0ce369690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce369690 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7820023040 tx=0x7f7820013000 comp rx=0 tx=0).stop 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 -- 172.21.15.5:0/1667817389 shutdown_connections 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce3734f0 0x55a0ce378b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce369690 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 --2- 172.21.15.5:0/1667817389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce36ece0 0x55a0ce372ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 -- 172.21.15.5:0/1667817389 >> 172.21.15.5:0/1667817389 conn(0x55a0ce238680 msgr2=0x55a0ce36d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 -- 172.21.15.5:0/1667817389 shutdown_connections 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 -- 172.21.15.5:0/1667817389 wait complete. 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 -- start start 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.690+0000 7f783aa41b80 1 Processor -- start 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce380ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce37d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce37ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7824119ed0 con 0x55a0ce36ece0 2024-05-14T15:34:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78241c7870 con 0x55a0ce3734f0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f782411a5a0 con 0x55a0ce1f0ec0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f78295c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce37d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f78295c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce37d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53858/0 (socket says 172.21.15.5:53858) 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f78295c8700 1 -- 172.21.15.5:0/46752537 learned_addr learned my addr 172.21.15.5:0/46752537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f782a5ca700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce37ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f7829dc9700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce380ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.691+0000 7f783aa41b80 10 client.0 _ll_register_callbacks cb 0x55a0ce2289a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f782a5ca700 1 -- 172.21.15.5:0/46752537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 msgr2=0x55a0ce380ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f782a5ca700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce380ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f782a5ca700 1 -- 172.21.15.5:0/46752537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 msgr2=0x55a0ce37d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f782a5ca700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce37d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f782a5ca700 1 -- 172.21.15.5:0/46752537 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ce37e320 con 0x55a0ce3734f0 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f78295c8700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce37d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f782a5ca700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce37ddb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f781c009d60 tx=0x7f781c01af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3851fc4e3bf7138 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f780d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f780d7fa700 1 -- 172.21.15.5:0/46752537 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c01e040 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781c017e50 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f780d7fa700 1 -- 172.21.15.5:0/46752537 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f781c02d630 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.692+0000 7f780d7fa700 1 -- 172.21.15.5:0/46752537 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f781c03e050 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 10 client.27239 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a0ce3f3780 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f780d7fa700 1 -- 172.21.15.5:0/46752537 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c017e50 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f780d7fa700 1 -- 172.21.15.5:0/46752537 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f781c018040 con 0x55a0ce3734f0 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 20 client.27239 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce3f3b00 0x55a0ce3f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 4 client.27239 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:34:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.693+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0ce298360 con 0x55a0ce3f3b00 2024-05-14T15:34:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.694+0000 7f7829dc9700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce3f3b00 0x55a0ce3f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.694+0000 7f7829dc9700 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce3f3b00 0x55a0ce3f5f20 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d30e7b776b84e53 server_cookie=51332b64280010cb in_seq=0 out_seq=0 2024-05-14T15:34:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.694+0000 7f780d7fa700 10 client.27239 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f780d7fa700 1 -- 172.21.15.5:0/46752537 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0ce298360 con 0x55a0ce3f3b00 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f780d7fa700 10 client.27239 handle_command_reply: tid=0 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 client.27239 shutdown 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 msgr2=0x55a0ce37ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce37ddb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f781c009d60 tx=0x7f781c01af00 comp rx=0 tx=0).stop 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 shutdown_connections 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55a0ce3f3b00 0x55a0ce3f5f20 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ce3734f0 0x55a0ce37ddb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0ce1f0ec0 0x55a0ce380ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f780d7fa700 0 client.27239 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 --2- 172.21.15.5:0/46752537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0ce36ece0 0x55a0ce37d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.697+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 >> 172.21.15.5:0/46752537 conn(0x55a0ce238680 msgr2=0x55a0ce3693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 shutdown_connections 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f783aa41b80 1 -- 172.21.15.5:0/46752537 wait complete. 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f783aa41b80 20 client.27239 trim_cache size 0 max 16384 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "36920e7f-90b8-46c4-a799-c84adb4dfffc", 2024-05-14T15:34:23.702 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:34:23.703 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:34:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78100b2af0 msgr2=0x7f78100b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78100b2af0 0x7f78100b5000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f781c01b000 tx=0x7f781c01d000 comp rx=0 tx=0).stop 2024-05-14T15:34:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 msgr2=0x7f78241aa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f78241aa320 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7814009810 tx=0x7f781400b840 comp rx=0 tx=0).stop 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 shutdown_connections 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78100b2af0 0x7f78100b5000 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782411d930 0x7f78241ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f782410a930 0x7f78241a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 --2- 172.21.15.5:0/3629902450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f782410b280 0x7f78241aa320 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.698+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 >> 172.21.15.5:0/3629902450 conn(0x7f782406f4d0 msgr2=0x7f782411b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.699+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 shutdown_connections 2024-05-14T15:34:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:23.699+0000 7f782c02d700 1 -- 172.21.15.5:0/3629902450 wait complete. 2024-05-14T15:34:24.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 896216 9.76 MB/sec execute 345 sec latency 27.776 ms 2024-05-14T15:34:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:23 smithi144 ceph-mon[35650]: pgmap v299: 129 pgs: 129 active+clean; 67 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 474 op/s 2024-05-14T15:34:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:23 smithi144 ceph-mon[35650]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T15:34:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:23 smithi193 ceph-mon[36001]: pgmap v299: 129 pgs: 129 active+clean; 67 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 474 op/s 2024-05-14T15:34:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:23 smithi193 ceph-mon[36001]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T15:34:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:23 smithi005 ceph-mon[31559]: pgmap v299: 129 pgs: 129 active+clean; 67 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 474 op/s 2024-05-14T15:34:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:23 smithi005 ceph-mon[31559]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T15:34:24.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:34:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:25.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:24 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:34:25.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:24 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:25.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:24 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:34:25.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:24 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:25.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 898296 9.76 MB/sec execute 346 sec latency 21.660 ms 2024-05-14T15:34:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:24 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:34:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:24 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:24 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:34:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:24 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:24 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:34:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:24 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:24 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:34:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:24 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:26.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 899652 9.75 MB/sec execute 347 sec latency 15.072 ms 2024-05-14T15:34:26.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:25 smithi005 ceph-mon[31559]: pgmap v301: 129 pgs: 129 active+clean; 67 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 474 op/s 2024-05-14T15:34:26.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:25 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:34:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:25 smithi144 ceph-mon[35650]: pgmap v301: 129 pgs: 129 active+clean; 67 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 474 op/s 2024-05-14T15:34:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:25 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:34:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:25 smithi193 ceph-mon[36001]: pgmap v301: 129 pgs: 129 active+clean; 67 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 474 op/s 2024-05-14T15:34:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:25 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:34:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 -- 172.21.15.5:0/1343352004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1034e0 msgr2=0x7fa94c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 --2- 172.21.15.5:0/1343352004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1034e0 0x7fa94c1038f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0099c0 tx=0x7fa93c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 -- 172.21.15.5:0/1343352004 shutdown_connections 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 --2- 172.21.15.5:0/1343352004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c108760 0x7fa94c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 --2- 172.21.15.5:0/1343352004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c1034e0 0x7fa94c1038f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 --2- 172.21.15.5:0/1343352004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94c107e10 0x7fa94c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 -- 172.21.15.5:0/1343352004 >> 172.21.15.5:0/1343352004 conn(0x7fa94c0fef30 msgr2=0x7fa94c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 -- 172.21.15.5:0/1343352004 shutdown_connections 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.829+0000 7fa9518a7700 1 -- 172.21.15.5:0/1343352004 wait complete. 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 Processor -- start 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 -- start start 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 0x7fa94c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94c107e10 0x7fa94c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c108760 0x7fa94c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c113390 con 0x7fa94c108760 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c113210 con 0x7fa94c1034e0 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.830+0000 7fa9518a7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c113510 con 0x7fa94c107e10 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c108760 0x7fa94c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 0x7fa94c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 0x7fa94c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:46970/0 (socket says 172.21.15.5:46970) 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94affd700 1 -- 172.21.15.5:0/580103266 learned_addr learned my addr 172.21.15.5:0/580103266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94c107e10 0x7fa94c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 -- 172.21.15.5:0/580103266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94c107e10 msgr2=0x7fa94c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94c107e10 0x7fa94c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 -- 172.21.15.5:0/580103266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 msgr2=0x7fa94c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 0x7fa94c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 -- 172.21.15.5:0/580103266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c1a7000 con 0x7fa94c108760 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94affd700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 0x7fa94c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c108760 0x7fa94c19c750 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa94400efc0 tx=0x7fa94400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa9508a5700 1 -- 172.21.15.5:0/580103266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440092e0 con 0x7fa94c108760 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c1a7290 con 0x7fa94c108760 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa9508a5700 1 -- 172.21.15.5:0/580103266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa944004bb0 con 0x7fa94c108760 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa9508a5700 1 -- 172.21.15.5:0/580103266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944004020 con 0x7fa94c108760 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.831+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c1a77d0 con 0x7fa94c108760 2024-05-14T15:34:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.835+0000 7fa9508a5700 1 -- 172.21.15.5:0/580103266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa94401b030 con 0x7fa94c108760 2024-05-14T15:34:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.836+0000 7fa9508a5700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9340b29f0 0x7fa9340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.836+0000 7fa9602bbb80 1 Processor -- start 2024-05-14T15:34:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.836+0000 7fa9602bbb80 1 -- start start 2024-05-14T15:34:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.836+0000 7fa94affd700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9340b29f0 0x7fa9340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.836+0000 7fa9508a5700 1 -- 172.21.15.5:0/580103266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa9440e2470 con 0x7fa94c108760 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07541b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc07546350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 0x55dc07546ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc0745f860 con 0x55dc07542070 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc073fe7c0 con 0x55dc073bfe40 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dc0745b820 con 0x55dc07546890 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc07546350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 0x55dc07546ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc07546350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47928/0 (socket says 172.21.15.5:47928) 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 0x55dc07546ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51106/0 (socket says 172.21.15.5:51106) 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94a7fc700 1 -- 172.21.15.5:0/2037884585 learned_addr learned my addr 172.21.15.5:0/2037884585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94affd700 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07541b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94affd700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9340b29f0 0x7fa9340b4f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009990 tx=0x7fa93c01a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 -- 172.21.15.5:0/2037884585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 msgr2=0x55dc07541b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07541b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 -- 172.21.15.5:0/2037884585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 msgr2=0x55dc07546350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc07546350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 -- 172.21.15.5:0/2037884585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc073ac8d0 con 0x55dc07546890 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.837+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 0x55dc07546ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9440df7c0 tx=0x7fa9440a8b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393e36bde1ffcaf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.838+0000 7fa931ffb700 1 -- 172.21.15.5:0/2037884585 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440b1b10 con 0x55dc07546890 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.838+0000 7fa931ffb700 1 -- 172.21.15.5:0/2037884585 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9440a8ef0 con 0x55dc07546890 2024-05-14T15:34:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.838+0000 7fa931ffb700 1 -- 172.21.15.5:0/2037884585 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440a9640 con 0x55dc07546890 2024-05-14T15:34:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2037884585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 msgr2=0x55dc07546ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 0x55dc07546ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9440df7c0 tx=0x7fa9440a8b50 comp rx=0 tx=0).stop 2024-05-14T15:34:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2037884585 shutdown_connections 2024-05-14T15:34:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc07546890 0x55dc07546ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07541b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2037884585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc07546350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2037884585 >> 172.21.15.5:0/2037884585 conn(0x55dc07407600 msgr2=0x55dc075384b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2037884585 shutdown_connections 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2037884585 wait complete. 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 -- start start 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.839+0000 7fa9602bbb80 1 Processor -- start 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 0x55dc0754b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0754ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07546890 0x55dc0754bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc0754e840 con 0x55dc07546890 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc073c5c30 con 0x55dc07542070 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dc0754e9c0 con 0x55dc073bfe40 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa9602bbb80 10 client.0 _ll_register_callbacks cb 0x55dc073f7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 0x55dc0754b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 0x55dc0754b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51108/0 (socket says 172.21.15.5:51108) 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 -- 172.21.15.5:0/2434297342 learned_addr learned my addr 172.21.15.5:0/2434297342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07546890 0x55dc0754bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0754ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 -- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 msgr2=0x55dc0754ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0754ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 -- 172.21.15.5:0/2434297342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07546890 msgr2=0x55dc0754bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07546890 0x55dc0754bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.840+0000 7fa94affd700 1 -- 172.21.15.5:0/2434297342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc073fb690 con 0x55dc073bfe40 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa94affd700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 0x55dc0754b550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000c00 tx=0x7fa93c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11d44412c66c73f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa92affd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa92affd700 1 -- 172.21.15.5:0/2434297342 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c034060 con 0x55dc073bfe40 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa92affd700 1 -- 172.21.15.5:0/2434297342 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa93c026480 con 0x55dc073bfe40 2024-05-14T15:34:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9440a8ef0 con 0x55dc073bfe40 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa92affd700 1 -- 172.21.15.5:0/2434297342 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa93c03b050 con 0x55dc073bfe40 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa9602bbb80 10 client.27220 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc0753d460 con 0x55dc073bfe40 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa92affd700 1 -- 172.21.15.5:0/2434297342 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9440a8ef0 con 0x55dc073bfe40 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.841+0000 7fa92affd700 1 -- 172.21.15.5:0/2434297342 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa93c00f740 con 0x55dc073bfe40 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa9602bbb80 10 client.27220 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa9602bbb80 10 client.27220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa9602bbb80 20 client.27220 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc0753faa0 0x55dc0753ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa9602bbb80 4 client.27220 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc0746d240 con 0x55dc0753faa0 2024-05-14T15:34:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc0753faa0 0x55dc0753ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa92affd700 10 client.27220 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.842+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc0753faa0 0x55dc0753ff30 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391ce4094b0b8521 server_cookie=e1caacd37b09e961 in_seq=0 out_seq=0 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa92affd700 1 -- 172.21.15.5:0/2434297342 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc0746d240 con 0x55dc0753faa0 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa92affd700 10 client.27220 handle_command_reply: tid=0 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 client.27220 shutdown 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 msgr2=0x55dc0754b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 0x55dc0754b550 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa93c000c00 tx=0x7fa93c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 shutdown_connections 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc0753faa0 0x55dc0753ff30 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07546890 0x55dc0754bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc073bfe40 0x55dc0754b550 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa92affd700 0 client.27220 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2434297342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0754ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 >> 172.21.15.5:0/2434297342 conn(0x55dc07407600 msgr2=0x55dc0753c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 shutdown_connections 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2434297342 wait complete. 2024-05-14T15:34:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.844+0000 7fa9602bbb80 20 client.27220 trim_cache size 0 max 16384 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 Processor -- start 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 -- start start 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07538610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0754be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc0755e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1b0db0 con 0x55dc07542070 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1ae640 con 0x55dc073bfe40 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.865+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1ac4b0 con 0x55dc0753faa0 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc0755e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc0755e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47944/0 (socket says 172.21.15.5:47944) 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0754be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94b7fe700 1 -- 172.21.15.5:0/329868577 learned_addr learned my addr 172.21.15.5:0/329868577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94affd700 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07538610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 -- 172.21.15.5:0/329868577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 msgr2=0x55dc07538610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07538610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 -- 172.21.15.5:0/329868577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 msgr2=0x55dc0755e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc0755e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 -- 172.21.15.5:0/329868577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc075454a0 con 0x55dc0753faa0 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc0755e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0754be10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55dc07540630 tx=0x7fa940009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d8bc40e0ff1dbc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa9317fa700 1 -- 172.21.15.5:0/329868577 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94002b070 con 0x55dc0753faa0 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa9317fa700 1 -- 172.21.15.5:0/329868577 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9400268b0 con 0x55dc0753faa0 2024-05-14T15:34:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.866+0000 7fa9317fa700 1 -- 172.21.15.5:0/329868577 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940003d40 con 0x55dc0753faa0 2024-05-14T15:34:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 -- 172.21.15.5:0/329868577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 msgr2=0x55dc0754be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0754be10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55dc07540630 tx=0x7fa940009260 comp rx=0 tx=0).stop 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 -- 172.21.15.5:0/329868577 shutdown_connections 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc07542070 0x55dc0755e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc073bfe40 0x55dc07538610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/329868577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0754be10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 -- 172.21.15.5:0/329868577 >> 172.21.15.5:0/329868577 conn(0x55dc07407600 msgr2=0x55dc0754a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 -- 172.21.15.5:0/329868577 shutdown_connections 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.867+0000 7fa9602bbb80 1 -- 172.21.15.5:0/329868577 wait complete. 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 -- start start 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 Processor -- start 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 0x55dc0755d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0755d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0755dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1acb00 con 0x55dc073bfe40 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1adf70 con 0x55dc07542070 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.868+0000 7fa9602bbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1ad1d0 con 0x55dc0753faa0 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa9602bbb80 10 client.0 _ll_register_callbacks cb 0x55dc073f7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 0x55dc0755d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 0x55dc0755d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47946/0 (socket says 172.21.15.5:47946) 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 -- 172.21.15.5:0/2944115450 learned_addr learned my addr 172.21.15.5:0/2944115450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0755d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94b7fe700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0755dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 -- 172.21.15.5:0/2944115450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 msgr2=0x55dc0755d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0755d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 -- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 msgr2=0x55dc0755dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:26.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0755dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 -- 172.21.15.5:0/2944115450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc075c1160 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0755d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa94affd700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 0x55dc0755d130 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009750 tx=0x7fa93c025690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c65a45e38bb45f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa92affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa92affd700 1 -- 172.21.15.5:0/2944115450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c034040 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa92affd700 1 -- 172.21.15.5:0/2944115450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa93c00f670 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.869+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9400268b0 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa92affd700 1 -- 172.21.15.5:0/2944115450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa93c04d050 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 10 client.17637 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc075c23d0 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa92affd700 1 -- 172.21.15.5:0/2944115450 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9400268b0 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa92affd700 1 -- 172.21.15.5:0/2944115450 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa93c026480 con 0x55dc073bfe40 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 10 client.17637 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:26.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 20 client.17637 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc07546890 0x55dc075c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 4 client.17637 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc074672e0 con 0x55dc07546890 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.870+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc07546890 0x55dc075c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.871+0000 7fa92affd700 10 client.17637 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.871+0000 7fa94a7fc700 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc07546890 0x55dc075c4a20 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fdc65db9df816f server_cookie=aaa95239fa082f03 in_seq=0 out_seq=0 2024-05-14T15:34:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa92affd700 1 -- 172.21.15.5:0/2944115450 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc074672e0 con 0x55dc07546890 2024-05-14T15:34:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa92affd700 10 client.17637 handle_command_reply: tid=0 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 client.17637 shutdown 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 msgr2=0x55dc0755d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 0x55dc0755d130 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009750 tx=0x7fa93c025690 comp rx=0 tx=0).stop 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 shutdown_connections 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc07546890 0x55dc075c4a20 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc07542070 0x55dc0755dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc073bfe40 0x55dc0755d130 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 --2- 172.21.15.5:0/2944115450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc0753faa0 0x55dc0755d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 >> 172.21.15.5:0/2944115450 conn(0x55dc07407600 msgr2=0x55dc0753c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa92affd700 0 client.17637 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 shutdown_connections 2024-05-14T15:34:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 1 -- 172.21.15.5:0/2944115450 wait complete. 2024-05-14T15:34:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.872+0000 7fa9602bbb80 20 client.17637 trim_cache size 0 max 16384 2024-05-14T15:34:26.879 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:34:26.879 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:34:26.879 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:34:26.879 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9340b29f0 msgr2=0x7fa9340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9340b29f0 0x7fa9340b4f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009990 tx=0x7fa93c01a040 comp rx=0 tx=0).stop 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c108760 msgr2=0x7fa94c19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c108760 0x7fa94c19c750 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa94400efc0 tx=0x7fa94400c3a0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 shutdown_connections 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9340b29f0 0x7fa9340b4f00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa94c108760 0x7fa94c19c750 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa94c1034e0 0x7fa94c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 --2- 172.21.15.5:0/580103266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94c107e10 0x7fa94c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 >> 172.21.15.5:0/580103266 conn(0x7fa94c0fef30 msgr2=0x7fa94c1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 shutdown_connections 2024-05-14T15:34:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:26.873+0000 7fa9518a7700 1 -- 172.21.15.5:0/580103266 wait complete. 2024-05-14T15:34:27.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 901807 9.76 MB/sec execute 348 sec latency 16.918 ms 2024-05-14T15:34:27.702 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:34:27.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:34:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:27 smithi005 ceph-mon[31559]: pgmap v302: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T15:34:28.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 902807 9.74 MB/sec execute 349 sec latency 19.458 ms 2024-05-14T15:34:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:27 smithi144 ceph-mon[35650]: pgmap v302: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T15:34:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:27 smithi193 ceph-mon[36001]: pgmap v302: 129 pgs: 129 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T15:34:29.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 905271 9.76 MB/sec execute 350 sec latency 28.205 ms 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 -- 172.21.15.5:0/1087199235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80108b90 msgr2=0x7f3a80102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 --2- 172.21.15.5:0/1087199235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80108b90 0x7f3a80102730 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009900 tx=0x7f3a7c02a410 comp rx=0 tx=0).stop 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 -- 172.21.15.5:0/1087199235 shutdown_connections 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 --2- 172.21.15.5:0/1087199235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 --2- 172.21.15.5:0/1087199235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80109030 0x7f3a80102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 --2- 172.21.15.5:0/1087199235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80108b90 0x7f3a80102730 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 -- 172.21.15.5:0/1087199235 >> 172.21.15.5:0/1087199235 conn(0x7f3a800fe390 msgr2=0x7f3a801007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 -- 172.21.15.5:0/1087199235 shutdown_connections 2024-05-14T15:34:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.169+0000 7f3a87726640 1 -- 172.21.15.5:0/1087199235 wait complete. 2024-05-14T15:34:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 Processor -- start 2024-05-14T15:34:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 -- start start 2024-05-14T15:34:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80108b90 0x7f3a8019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80109030 0x7f3a8019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80112d80 con 0x7f3a80109030 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80112c00 con 0x7f3a80108b90 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a87726640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80112f00 con 0x7f3a801032e0 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80108b90 0x7f3a8019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80109030 0x7f3a8019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51156/0 (socket says 172.21.15.5:51156) 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.170+0000 7f3a854c2640 1 -- 172.21.15.5:0/216108339 learned_addr learned my addr 172.21.15.5:0/216108339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a854c2640 1 -- 172.21.15.5:0/216108339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80108b90 msgr2=0x7f3a8019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a854c2640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80108b90 0x7f3a8019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a854c2640 1 -- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80109030 msgr2=0x7f3a8019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a854c2640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80109030 0x7f3a8019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a854c2640 1 -- 172.21.15.5:0/216108339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a801a6a50 con 0x7f3a801032e0 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80108b90 0x7f3a8019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a854c2640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8019b740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c032fd0 tx=0x7f3a7c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a767fc640 1 -- 172.21.15.5:0/216108339 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c03f030 con 0x7f3a801032e0 2024-05-14T15:34:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a767fc640 1 -- 172.21.15.5:0/216108339 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c004490 con 0x7f3a801032e0 2024-05-14T15:34:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a801a6ce0 con 0x7f3a801032e0 2024-05-14T15:34:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a767fc640 1 -- 172.21.15.5:0/216108339 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c035070 con 0x7f3a801032e0 2024-05-14T15:34:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.171+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a801a71d0 con 0x7f3a801032e0 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 Processor -- start 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 -- start start 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdbdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9f42f0 0x55785cdbe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 0x55785cdbe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55785cd235d0 con 0x55785ca1d1b0 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55785ccdc890 con 0x55785c9fc4d0 2024-05-14T15:34:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.174+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55785cd1f590 con 0x55785c9f42f0 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a767fc640 1 -- 172.21.15.5:0/216108339 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3a7c042030 con 0x7f3a801032e0 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdbdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 0x55785cdbe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdbdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47970/0 (socket says 172.21.15.5:47970) 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 0x55785cdbe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47040/0 (socket says 172.21.15.5:47040) 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a85cc3640 1 -- 172.21.15.5:0/2610664154 learned_addr learned my addr 172.21.15.5:0/2610664154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a854c2640 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9f42f0 0x55785cdbe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 -- 172.21.15.5:0/2610664154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9f42f0 msgr2=0x55785cdbe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9f42f0 0x55785cdbe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 -- 172.21.15.5:0/2610664154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 msgr2=0x55785cdbdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdbdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 -- 172.21.15.5:0/2610664154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785ca35210 con 0x55785c9fc4d0 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdbdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.175+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 0x55785cdbe660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a80109460 tx=0x7f3a70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326f497b86c6273b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.176+0000 7f3a6b7fe640 1 -- 172.21.15.5:0/2610664154 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70004310 con 0x55785c9fc4d0 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.176+0000 7f3a6b7fe640 1 -- 172.21.15.5:0/2610664154 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a700044b0 con 0x55785c9fc4d0 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.176+0000 7f3a6b7fe640 1 -- 172.21.15.5:0/2610664154 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700035f0 con 0x55785c9fc4d0 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.176+0000 7f3a767fc640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3a6c0b2950 0x7f3a6c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.176+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3a6c0b2950 0x7f3a6c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.176+0000 7f3a767fc640 1 -- 172.21.15.5:0/216108339 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f3a7c102d20 con 0x7f3a801032e0 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3a6c0b2950 0x7f3a6c0b4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55785cdbad00 tx=0x7f3a78009500 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2610664154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 msgr2=0x55785cdbe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 0x55785cdbe660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a80109460 tx=0x7f3a70011810 comp rx=0 tx=0).stop 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2610664154 shutdown_connections 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9f42f0 0x55785cdbe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9fc4d0 0x55785cdbe660 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2610664154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdbdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2610664154 >> 172.21.15.5:0/2610664154 conn(0x55785ca11300 msgr2=0x55785cdba340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2610664154 shutdown_connections 2024-05-14T15:34:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.177+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2610664154 wait complete. 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.178+0000 7f3a8e5d8740 1 -- start start 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.178+0000 7f3a8e5d8740 1 Processor -- start 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdc81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785c9fc4d0 0x55785cdc86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785ca1d1b0 0x55785cdd41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55785cdcad10 con 0x55785c9fc4d0 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55785cce0370 con 0x55785c9f42f0 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55785cdcae90 con 0x55785ca1d1b0 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a8e5d8740 10 client.0 _ll_register_callbacks cb 0x55785ca34a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdc81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdc81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47048/0 (socket says 172.21.15.5:47048) 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785c9fc4d0 0x55785cdc86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a854c2640 1 -- 172.21.15.5:0/4207731955 learned_addr learned my addr 172.21.15.5:0/4207731955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785ca1d1b0 0x55785cdd41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.179+0000 7f3a854c2640 1 -- 172.21.15.5:0/4207731955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785ca1d1b0 msgr2=0x55785cdd41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a854c2640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785ca1d1b0 0x55785cdd41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a854c2640 1 -- 172.21.15.5:0/4207731955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785c9fc4d0 msgr2=0x55785cdc86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a854c2640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785c9fc4d0 0x55785cdc86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a854c2640 1 -- 172.21.15.5:0/4207731955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785cb01620 con 0x55785c9f42f0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785ca1d1b0 0x55785cdd41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a854c2640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdc81b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c031440 tx=0x7f3a7c03b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b906796ccc04ab7d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a68ff9640 1 -- 172.21.15.5:0/4207731955 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c049070 con 0x55785c9f42f0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a68ff9640 1 -- 172.21.15.5:0/4207731955 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f3a7c0b1070 con 0x55785c9f42f0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a700044b0 con 0x55785c9f42f0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a68ff9640 1 -- 172.21.15.5:0/4207731955 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c0c9420 con 0x55785c9f42f0 2024-05-14T15:34:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a8e5d8740 10 client.27269 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.180+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55785cdd7040 con 0x55785c9f42f0 2024-05-14T15:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a68ff9640 1 -- 172.21.15.5:0/4207731955 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a700044b0 con 0x55785c9f42f0 2024-05-14T15:34:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a68ff9640 1 -- 172.21.15.5:0/4207731955 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3a7c0e40f0 con 0x55785c9f42f0 2024-05-14T15:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a8e5d8740 10 client.27269 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a8e5d8740 10 client.27269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a8e5d8740 20 client.27269 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9dbd80 0x55785cdda560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a8e5d8740 4 client.27269 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55785cd311e0 con 0x55785c9dbd80 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.181+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9dbd80 0x55785cdda560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.182+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9dbd80 0x55785cdda560 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7b79bc95000940 server_cookie=c49052a42c052985 in_seq=0 out_seq=0 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.182+0000 7f3a68ff9640 10 client.27269 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.184+0000 7f3a68ff9640 1 -- 172.21.15.5:0/4207731955 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55785cd311e0 con 0x55785c9dbd80 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.184+0000 7f3a68ff9640 10 client.27269 handle_command_reply: tid=0 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.184+0000 7f3a8e5d8740 1 client.27269 shutdown 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 msgr2=0x55785cdc81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdc81b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c031440 tx=0x7f3a7c03b560 comp rx=0 tx=0).stop 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 shutdown_connections 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9dbd80 0x55785cdda560 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785ca1d1b0 0x55785cdd41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785c9fc4d0 0x55785cdc86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/4207731955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdc81b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a68ff9640 0 client.27269 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 >> 172.21.15.5:0/4207731955 conn(0x55785ca11300 msgr2=0x55785cdbf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 shutdown_connections 2024-05-14T15:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/4207731955 wait complete. 2024-05-14T15:34:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.185+0000 7f3a8e5d8740 20 client.27269 trim_cache size 0 max 16384 2024-05-14T15:34:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.202+0000 7f3a8e5d8740 1 Processor -- start 2024-05-14T15:34:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.202+0000 7f3a8e5d8740 1 -- start start 2024-05-14T15:34:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.202+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.202+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cddfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.202+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cde02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47056/0 (socket says 172.21.15.5:47056) 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cddfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cddfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51186/0 (socket says 172.21.15.5:51186) 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cde02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cde02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47978/0 (socket says 172.21.15.5:47978) 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a854c2640 1 -- 172.21.15.5:0/2746362716 learned_addr learned my addr 172.21.15.5:0/2746362716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801b3ca0 con 0x55785ca1d1b0 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801b1570 con 0x55785c9f42f0 2024-05-14T15:34:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.203+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801af3e0 con 0x55785c9fc4d0 2024-05-14T15:34:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a84cc1640 1 -- 172.21.15.5:0/2746362716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 msgr2=0x55785cdca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a84cc1640 1 -- 172.21.15.5:0/2746362716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 msgr2=0x55785cde02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cde02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a84cc1640 1 -- 172.21.15.5:0/2746362716 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785cdcabf0 con 0x55785c9fc4d0 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cddfd60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55785cdb5c40 tx=0x7f3a70004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e60c03aea577b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a6affd640 1 -- 172.21.15.5:0/2746362716 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7001e040 con 0x55785c9fc4d0 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a6affd640 1 -- 172.21.15.5:0/2746362716 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7001f040 con 0x55785c9fc4d0 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.204+0000 7f3a6affd640 1 -- 172.21.15.5:0/2746362716 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70003c30 con 0x55785c9fc4d0 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 msgr2=0x55785cddfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cddfd60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55785cdb5c40 tx=0x7f3a70004ba0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 shutdown_connections 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cddfd60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cde02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/2746362716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 >> 172.21.15.5:0/2746362716 conn(0x55785ca11300 msgr2=0x55785cdc6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 shutdown_connections 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/2746362716 wait complete. 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 -- start start 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.205+0000 7f3a8e5d8740 1 Processor -- start 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdd0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cdcb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdcb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801afa30 con 0x55785ca1d1b0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801b0ea0 con 0x55785c9f42f0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801b0100 con 0x55785c9fc4d0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a8e5d8740 10 client.0 _ll_register_callbacks cb 0x7f3a7c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdcb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a854c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdd0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a84cc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cdcb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdcb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47990/0 (socket says 172.21.15.5:47990) 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 -- 172.21.15.5:0/1333137601 learned_addr learned my addr 172.21.15.5:0/1333137601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 -- 172.21.15.5:0/1333137601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 msgr2=0x55785cdcb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cdcb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 -- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 msgr2=0x55785cdd0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdd0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a85cc3640 1 -- 172.21.15.5:0/1333137601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785cdcbea0 con 0x55785ca1d1b0 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a84cc1640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cdcb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.206+0000 7f3a854c2640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdd0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a85cc3640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdcb730 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3a7801c610 tx=0x7f3a78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a06beb8f9b0515 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a68ff9640 1 -- 172.21.15.5:0/1333137601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78029410 con 0x55785ca1d1b0 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a68ff9640 1 -- 172.21.15.5:0/1333137601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f3a780295b0 con 0x55785ca1d1b0 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a7001f040 con 0x55785ca1d1b0 2024-05-14T15:34:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a68ff9640 1 -- 172.21.15.5:0/1333137601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a78038050 con 0x55785ca1d1b0 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a8e5d8740 10 client.17655 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55785ce41980 con 0x55785ca1d1b0 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a68ff9640 1 -- 172.21.15.5:0/1333137601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7001f040 con 0x55785ca1d1b0 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.207+0000 7f3a68ff9640 1 -- 172.21.15.5:0/1333137601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f3a78031960 con 0x55785ca1d1b0 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a8e5d8740 10 client.17655 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a8e5d8740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a8e5d8740 20 client.17655 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9b11c0 0x55785ce44190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a8e5d8740 4 client.17655 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55785cd2b050 con 0x55785c9b11c0 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a854c2640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9b11c0 0x55785ce44190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a68ff9640 10 client.17655 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.208+0000 7f3a854c2640 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9b11c0 0x55785ce44190 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e5745626e1829 server_cookie=2134eb052d9023ce in_seq=0 out_seq=0 2024-05-14T15:34:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a68ff9640 1 -- 172.21.15.5:0/1333137601 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55785cd2b050 con 0x55785c9b11c0 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a68ff9640 10 client.17655 handle_command_reply: tid=0 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 client.17655 shutdown 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 msgr2=0x55785cdcb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdcb730 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3a7801c610 tx=0x7f3a78004ab0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 shutdown_connections 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55785c9b11c0 0x55785ce44190 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55785c9fc4d0 0x55785cdcb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55785ca1d1b0 0x55785cdcb730 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a68ff9640 0 client.17655 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.498+0000 7f3a8e5d8740 1 --2- 172.21.15.5:0/1333137601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55785c9f42f0 0x55785cdd0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.499+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 >> 172.21.15.5:0/1333137601 conn(0x55785ca11300 msgr2=0x55785cdbd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.499+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 shutdown_connections 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.499+0000 7f3a8e5d8740 1 -- 172.21.15.5:0/1333137601 wait complete. 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.499+0000 7f3a8e5d8740 20 client.17655 trim_cache size 0 max 16384 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:34:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3a6c0b2950 msgr2=0x7f3a6c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3a6c0b2950 0x7f3a6c0b4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55785cdbad00 tx=0x7f3a78009500 comp rx=0 tx=0).stop 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 msgr2=0x7f3a8019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8019b740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c032fd0 tx=0x7f3a7c031a20 comp rx=0 tx=0).stop 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 shutdown_connections 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f3a6c0b2950 0x7f3a6c0b4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a80109030 0x7f3a8019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a80108b90 0x7f3a8019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 --2- 172.21.15.5:0/216108339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a801032e0 0x7f3a8019b740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 >> 172.21.15.5:0/216108339 conn(0x7f3a800fe390 msgr2=0x7f3a800fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 shutdown_connections 2024-05-14T15:34:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:29.500+0000 7f3a87726640 1 -- 172.21.15.5:0/216108339 wait complete. 2024-05-14T15:34:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 -- 172.21.15.5:0/634633418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c01034c0 msgr2=0x7fa6c01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 --2- 172.21.15.5:0/634633418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c01034c0 0x7fa6c01038d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0099c0 tx=0x7fa6bc009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 -- 172.21.15.5:0/634633418 shutdown_connections 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 --2- 172.21.15.5:0/634633418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 0x7fa6c01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 --2- 172.21.15.5:0/634633418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c01034c0 0x7fa6c01038d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 --2- 172.21.15.5:0/634633418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c0107df0 0x7fa6c0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 -- 172.21.15.5:0/634633418 >> 172.21.15.5:0/634633418 conn(0x7fa6c00fedc0 msgr2=0x7fa6c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 -- 172.21.15.5:0/634633418 shutdown_connections 2024-05-14T15:34:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.734+0000 7fa6c7080700 1 -- 172.21.15.5:0/634633418 wait complete. 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 Processor -- start 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 -- start start 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c01034c0 0x7fa6c0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 0x7fa6c019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 0x7fa6c019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0113370 con 0x7fa6c0107df0 2024-05-14T15:34:29.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01131f0 con 0x7fa6c0108740 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6c7080700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01134f0 con 0x7fa6c01034c0 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.735+0000 7fa6b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 0x7fa6c019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 0x7fa6c019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48020/0 (socket says 172.21.15.5:48020) 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6b7fff700 1 -- 172.21.15.5:0/1085764703 learned_addr learned my addr 172.21.15.5:0/1085764703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 0x7fa6c019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c4e1c700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c01034c0 0x7fa6c0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 -- 172.21.15.5:0/1085764703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c01034c0 msgr2=0x7fa6c0199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c01034c0 0x7fa6c0199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 -- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 msgr2=0x7fa6c019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 0x7fa6c019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 -- 172.21.15.5:0/1085764703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c019ec10 con 0x7fa6c0108740 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6b7fff700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 0x7fa6c019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c561d700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 0x7fa6c019e570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800efc0 tx=0x7fa6b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80092e0 con 0x7fa6c0108740 2024-05-14T15:34:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8004bb0 con 0x7fa6c0108740 2024-05-14T15:34:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c019eea0 con 0x7fa6c0108740 2024-05-14T15:34:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.736+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8004020 con 0x7fa6c0108740 2024-05-14T15:34:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.737+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c019f390 con 0x7fa6c0108740 2024-05-14T15:34:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.738+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c00512c0 con 0x7fa6c0108740 2024-05-14T15:34:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.739+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa6b801b030 con 0x7fa6c0108740 2024-05-14T15:34:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.740+0000 7fa6b5ffb700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa6a80b29f0 0x7fa6a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.740+0000 7fa6c4e1c700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa6a80b29f0 0x7fa6a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.740+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa6b80e24e0 con 0x7fa6c0108740 2024-05-14T15:34:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.740+0000 7fa6c4e1c700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa6a80b29f0 0x7fa6a80b4f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc003760 tx=0x7fa6bc01a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:29.745+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6b8022020 con 0x7fa6c0108740 2024-05-14T15:34:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.034+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6c010b170 con 0x7fa6c0108740 2024-05-14T15:34:30.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:29 smithi005 ceph-mon[31559]: pgmap v303: 129 pgs: 129 active+clean; 74 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-05-14T15:34:30.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.035+0000 7fa6b5ffb700 1 -- 172.21.15.5:0/1085764703 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7fa6b8010040 con 0x7fa6c0108740 2024-05-14T15:34:30.037 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:34:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:34:30.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa6a80b29f0 msgr2=0x7fa6a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa6a80b29f0 0x7fa6a80b4f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc003760 tx=0x7fa6bc01a040 comp rx=0 tx=0).stop 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 msgr2=0x7fa6c019e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 0x7fa6c019e570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800efc0 tx=0x7fa6b800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 shutdown_connections 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa6a80b29f0 0x7fa6a80b4f00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c0108740 0x7fa6c019e570 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c01034c0 0x7fa6c0199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 --2- 172.21.15.5:0/1085764703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6c0107df0 0x7fa6c019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 >> 172.21.15.5:0/1085764703 conn(0x7fa6c00fedc0 msgr2=0x7fa6c01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 shutdown_connections 2024-05-14T15:34:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:30.038+0000 7fa6c7080700 1 -- 172.21.15.5:0/1085764703 wait complete. 2024-05-14T15:34:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:34:30.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 905875 9.74 MB/sec execute 351 sec latency 139.521 ms 2024-05-14T15:34:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:29 smithi144 ceph-mon[35650]: pgmap v303: 129 pgs: 129 active+clean; 74 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-05-14T15:34:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:29 smithi193 ceph-mon[36001]: pgmap v303: 129 pgs: 129 active+clean; 74 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-05-14T15:34:30.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:34:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:30 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1085764703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:31.075 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:31.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 907668 9.73 MB/sec execute 352 sec latency 56.350 ms 2024-05-14T15:34:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:30 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1085764703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:31.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:30 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1085764703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:32.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 909246 9.74 MB/sec execute 353 sec latency 27.324 ms 2024-05-14T15:34:32.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:31 smithi144 ceph-mon[35650]: pgmap v304: 129 pgs: 129 active+clean; 74 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-05-14T15:34:32.204 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:31 smithi193 ceph-mon[36001]: pgmap v304: 129 pgs: 129 active+clean; 74 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-05-14T15:34:32.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:31 smithi005 ceph-mon[31559]: pgmap v304: 129 pgs: 129 active+clean; 74 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 352 op/s 2024-05-14T15:34:32.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.937+0000 7fb88581c700 1 -- 172.21.15.5:0/2719723724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb880107df0 msgr2=0x7fb880108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.937+0000 7fb88581c700 1 --2- 172.21.15.5:0/2719723724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb880107df0 0x7fb880108200 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb8740099c0 tx=0x7fb874009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 -- 172.21.15.5:0/2719723724 shutdown_connections 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 --2- 172.21.15.5:0/2719723724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb880108740 0x7fb8801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 --2- 172.21.15.5:0/2719723724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8801034c0 0x7fb8801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 --2- 172.21.15.5:0/2719723724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb880107df0 0x7fb880108200 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 -- 172.21.15.5:0/2719723724 >> 172.21.15.5:0/2719723724 conn(0x7fb8800fedc0 msgr2=0x7fb8801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 -- 172.21.15.5:0/2719723724 shutdown_connections 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 -- 172.21.15.5:0/2719723724 wait complete. 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 Processor -- start 2024-05-14T15:34:32.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.938+0000 7fb88581c700 1 -- start start 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb88581c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 0x7fb88019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb88581c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb880107df0 0x7fb88019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb88581c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb880108740 0x7fb88019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb88581c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb880113370 con 0x7fb8801034c0 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb88581c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801131f0 con 0x7fb880108740 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb88581c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801134f0 con 0x7fb880107df0 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 0x7fb88019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 0x7fb88019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48040/0 (socket says 172.21.15.5:48040) 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 -- 172.21.15.5:0/1427669622 learned_addr learned my addr 172.21.15.5:0/1427669622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87e7fc700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb880107df0 0x7fb88019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb880108740 0x7fb88019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 -- 172.21.15.5:0/1427669622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb880107df0 msgr2=0x7fb88019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb880107df0 0x7fb88019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 -- 172.21.15.5:0/1427669622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb880108740 msgr2=0x7fb88019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb880108740 0x7fb88019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 -- 172.21.15.5:0/1427669622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88019fd20 con 0x7fb8801034c0 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.939+0000 7fb87effd700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 0x7fb88019ae50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e9a0 tx=0x7fb87000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.940+0000 7fb88481a700 1 -- 172.21.15.5:0/1427669622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87000ca10 con 0x7fb8801034c0 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.940+0000 7fb88481a700 1 -- 172.21.15.5:0/1427669622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87000cbb0 con 0x7fb8801034c0 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.940+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8801a0010 con 0x7fb8801034c0 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.940+0000 7fb88481a700 1 -- 172.21.15.5:0/1427669622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870018680 con 0x7fb8801034c0 2024-05-14T15:34:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.940+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801a7930 con 0x7fb8801034c0 2024-05-14T15:34:32.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.943+0000 7fb88481a700 1 -- 172.21.15.5:0/1427669622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb870018820 con 0x7fb8801034c0 2024-05-14T15:34:32.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.943+0000 7fb88481a700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb8680b29f0 0x7fb8680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87e7fc700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb8680b29f0 0x7fb8680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb88481a700 1 -- 172.21.15.5:0/1427669622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fb8700e2c70 con 0x7fb8801034c0 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 Processor -- start 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 -- start start 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87e7fc700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb8680b29f0 0x7fb8680b4f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb87401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c42cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c42d190 0x55a16c426ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c427630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a16c344940 con 0x55a16c427220 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a16c2a9da0 con 0x55a16c42d190 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb894230b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a16c340900 con 0x55a16c2a4e40 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c427630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c42d190 0x55a16c426ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c427630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48044/0 (socket says 172.21.15.5:48044) 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87f7fe700 1 -- 172.21.15.5:0/2117911870 learned_addr learned my addr 172.21.15.5:0/2117911870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.944+0000 7fb87effd700 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c42cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87f7fe700 1 -- 172.21.15.5:0/2117911870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 msgr2=0x55a16c42cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c42cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87f7fe700 1 -- 172.21.15.5:0/2117911870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c42d190 msgr2=0x55a16c426ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c42d190 0x55a16c426ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87f7fe700 1 -- 172.21.15.5:0/2117911870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16c2918d0 con 0x55a16c427220 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87effd700 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c42cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87e7fc700 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c42d190 0x55a16c426ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c427630 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb878011680 tx=0x7fb878009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617a0fd8a3660a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb865ffb700 1 -- 172.21.15.5:0/2117911870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878019030 con 0x55a16c427220 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb865ffb700 1 -- 172.21.15.5:0/2117911870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8780092e0 con 0x55a16c427220 2024-05-14T15:34:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.945+0000 7fb865ffb700 1 -- 172.21.15.5:0/2117911870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878007500 con 0x55a16c427220 2024-05-14T15:34:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 -- 172.21.15.5:0/2117911870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 msgr2=0x55a16c427630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c427630 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb878011680 tx=0x7fb878009920 comp rx=0 tx=0).stop 2024-05-14T15:34:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 -- 172.21.15.5:0/2117911870 shutdown_connections 2024-05-14T15:34:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c427630 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c42cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 --2- 172.21.15.5:0/2117911870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c42d190 0x55a16c426ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 -- 172.21.15.5:0/2117911870 >> 172.21.15.5:0/2117911870 conn(0x55a16c2ec630 msgr2=0x55a16c41da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 -- 172.21.15.5:0/2117911870 shutdown_connections 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.946+0000 7fb894230b80 1 -- 172.21.15.5:0/2117911870 wait complete. 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 -- start start 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 Processor -- start 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c2a4e40 0x55a16c4308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c430df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a16c433c30 con 0x55a16c427220 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a16c2fded0 con 0x55a16c2a4e40 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a16c433db0 con 0x55a16c42d190 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.947+0000 7fb894230b80 10 client.0 _ll_register_callbacks cb 0x55a16c2dc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c2a4e40 0x55a16c4308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c2a4e40 0x55a16c4308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:47124/0 (socket says 172.21.15.5:47124) 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c430df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87effd700 1 -- 172.21.15.5:0/1289299338 learned_addr learned my addr 172.21.15.5:0/1289299338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 -- 172.21.15.5:0/1289299338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c2a4e40 msgr2=0x55a16c4308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c2a4e40 0x55a16c4308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 -- 172.21.15.5:0/1289299338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 msgr2=0x55a16c430df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c430df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 -- 172.21.15.5:0/1289299338 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16c2fb760 con 0x55a16c42d190 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb878011650 tx=0x7fb878003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c156ccdecdf06a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb85effd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb85effd700 1 -- 172.21.15.5:0/1289299338 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878004cd0 con 0x55a16c42d190 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb85effd700 1 -- 172.21.15.5:0/1289299338 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fb878007500 con 0x55a16c42d190 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.948+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8780092e0 con 0x55a16c42d190 2024-05-14T15:34:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb85effd700 1 -- 172.21.15.5:0/1289299338 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb878007700 con 0x55a16c42d190 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 10 client.27262 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a16c43e810 con 0x55a16c42d190 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb85effd700 1 -- 172.21.15.5:0/1289299338 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8780092e0 con 0x55a16c42d190 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb85effd700 1 -- 172.21.15.5:0/1289299338 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb8780051e0 con 0x55a16c42d190 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 10 client.27262 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 10 client.27262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 20 client.27262 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4410b0 0x55a16c443580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 4 client.27262 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.949+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16c352320 con 0x55a16c4410b0 2024-05-14T15:34:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.950+0000 7fb87effd700 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4410b0 0x55a16c443580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.950+0000 7fb85effd700 10 client.27262 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.950+0000 7fb87effd700 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4410b0 0x55a16c443580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c184da12ec18aa9 server_cookie=40f5bf0787cb57e6 in_seq=0 out_seq=0 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb85effd700 1 -- 172.21.15.5:0/1289299338 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a16c352320 con 0x55a16c4410b0 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb85effd700 10 client.27262 handle_command_reply: tid=0 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 client.27262 shutdown 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 msgr2=0x55a16c431330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb878011650 tx=0x7fb878003ce0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 shutdown_connections 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4410b0 0x55a16c443580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c2a4e40 0x55a16c4308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 --2- 172.21.15.5:0/1289299338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c427220 0x55a16c430df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb85effd700 0 client.27262 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 >> 172.21.15.5:0/1289299338 conn(0x55a16c2ec630 msgr2=0x55a16c425cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 shutdown_connections 2024-05-14T15:34:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 1 -- 172.21.15.5:0/1289299338 wait complete. 2024-05-14T15:34:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.951+0000 7fb894230b80 20 client.27262 trim_cache size 0 max 16384 2024-05-14T15:34:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.966+0000 7fb894230b80 1 Processor -- start 2024-05-14T15:34:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.966+0000 7fb894230b80 1 -- start start 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb894230b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 0x55a16c426ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb894230b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c4434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb894230b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb894230b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801b0d80 con 0x55a16c2a4e40 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb894230b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801ae610 con 0x55a16c427220 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb894230b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801ac480 con 0x55a16c42d190 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 0x55a16c426ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c4434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 0x55a16c426ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48064/0 (socket says 172.21.15.5:48064) 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 -- 172.21.15.5:0/775785375 learned_addr learned my addr 172.21.15.5:0/775785375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 -- 172.21.15.5:0/775785375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 msgr2=0x55a16c431170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 -- 172.21.15.5:0/775785375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 msgr2=0x55a16c4434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c4434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 -- 172.21.15.5:0/775785375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16c4316b0 con 0x55a16c2a4e40 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.967+0000 7fb87effd700 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 0x55a16c426ce0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb8700a9ae0 tx=0x7fb870009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3099fd9c594b19e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.968+0000 7fb8657fa700 1 -- 172.21.15.5:0/775785375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87008e070 con 0x55a16c2a4e40 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.968+0000 7fb8657fa700 1 -- 172.21.15.5:0/775785375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8700a9660 con 0x55a16c2a4e40 2024-05-14T15:34:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.968+0000 7fb8657fa700 1 -- 172.21.15.5:0/775785375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870018b10 con 0x55a16c2a4e40 2024-05-14T15:34:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 -- 172.21.15.5:0/775785375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 msgr2=0x55a16c426ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 0x55a16c426ce0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb8700a9ae0 tx=0x7fb870009790 comp rx=0 tx=0).stop 2024-05-14T15:34:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 -- 172.21.15.5:0/775785375 shutdown_connections 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c42d190 0x55a16c431170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c2a4e40 0x55a16c426ce0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 --2- 172.21.15.5:0/775785375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c4434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 -- 172.21.15.5:0/775785375 >> 172.21.15.5:0/775785375 conn(0x55a16c2ec630 msgr2=0x55a16c417580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 -- 172.21.15.5:0/775785375 shutdown_connections 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 -- 172.21.15.5:0/775785375 wait complete. 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 -- start start 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.969+0000 7fb894230b80 1 Processor -- start 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c4493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c439040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 0x55a16c439640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801acad0 con 0x55a16c42d190 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801adf40 con 0x55a16c427220 2024-05-14T15:34:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801ad1a0 con 0x55a16c2a4e40 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb894230b80 10 client.0 _ll_register_callbacks cb 0x55a16c2dc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 0x55a16c439640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c4493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c4493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51256/0 (socket says 172.21.15.5:51256) 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 0x55a16c439640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48078/0 (socket says 172.21.15.5:48078) 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87effd700 1 -- 172.21.15.5:0/3645219804 learned_addr learned my addr 172.21.15.5:0/3645219804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87e7fc700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c439040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 -- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 msgr2=0x55a16c4493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c4493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 -- 172.21.15.5:0/3645219804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 msgr2=0x55a16c439040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c439040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87f7fe700 1 -- 172.21.15.5:0/3645219804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16c447e60 con 0x55a16c42d190 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.970+0000 7fb87effd700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c4493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb87f7fe700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 0x55a16c439640 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb878002230 tx=0x7fb878007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759d87dd1c111dc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb85effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb85effd700 1 -- 172.21.15.5:0/3645219804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878019040 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb85effd700 1 -- 172.21.15.5:0/3645219804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fb878004190 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8700a9660 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb85effd700 1 -- 172.21.15.5:0/3645219804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb878004390 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 10 client.17679 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a16c4aac70 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb85effd700 1 -- 172.21.15.5:0/3645219804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8700a9660 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb85effd700 1 -- 172.21.15.5:0/3645219804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb878033030 con 0x55a16c42d190 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 10 client.17679 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 10 client.17679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 20 client.17679 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4ab2c0 0x55a16c4ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 4 client.17679 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.971+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a16c34c3c0 con 0x55a16c4ab2c0 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb87effd700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4ab2c0 0x55a16c4ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb85effd700 10 client.17679 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb87effd700 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4ab2c0 0x55a16c4ad6e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a9619735e6ac8fe server_cookie=fd89575c916bae16 in_seq=0 out_seq=0 2024-05-14T15:34:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb85effd700 1 -- 172.21.15.5:0/3645219804 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a16c34c3c0 con 0x55a16c4ab2c0 2024-05-14T15:34:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb85effd700 10 client.17679 handle_command_reply: tid=0 2024-05-14T15:34:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb894230b80 1 client.17679 shutdown 2024-05-14T15:34:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 msgr2=0x55a16c439640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.972+0000 7fb894230b80 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 0x55a16c439640 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb878002230 tx=0x7fb878007500 comp rx=0 tx=0).stop 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 shutdown_connections 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16c42d190 0x55a16c439640 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a16c2a4e40 0x55a16c4493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55a16c4ab2c0 0x55a16c4ad6e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 --2- 172.21.15.5:0/3645219804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16c427220 0x55a16c439040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb85effd700 0 client.17679 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 >> 172.21.15.5:0/3645219804 conn(0x55a16c2ec630 msgr2=0x55a16c42a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 shutdown_connections 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 1 -- 172.21.15.5:0/3645219804 wait complete. 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb894230b80 20 client.17679 trim_cache size 0 max 16384 2024-05-14T15:34:32.978 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb8680b29f0 msgr2=0x7fb8680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb88581c700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb8680b29f0 0x7fb8680b4f00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb87401a040 comp rx=0 tx=0).stop 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 msgr2=0x7fb88019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.973+0000 7fb88581c700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 0x7fb88019ae50 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e9a0 tx=0x7fb87000ecb0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 shutdown_connections 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb8680b29f0 0x7fb8680b4f00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb880108740 0x7fb88019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801034c0 0x7fb88019ae50 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 --2- 172.21.15.5:0/1427669622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb880107df0 0x7fb88019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 >> 172.21.15.5:0/1427669622 conn(0x7fb8800fedc0 msgr2=0x7fb88006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 shutdown_connections 2024-05-14T15:34:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:32.974+0000 7fb88581c700 1 -- 172.21.15.5:0/1427669622 wait complete. 2024-05-14T15:34:33.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 910618 9.72 MB/sec execute 354 sec latency 22.064 ms 2024-05-14T15:34:33.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:34:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:34.152 INFO:tasks.workunit.client.0.smithi005.stdout: 1 912609 9.73 MB/sec execute 355 sec latency 20.001 ms 2024-05-14T15:34:34.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:33 smithi005 ceph-mon[31559]: pgmap v305: 129 pgs: 129 active+clean; 64 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-05-14T15:34:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:33 smithi144 ceph-mon[35650]: pgmap v305: 129 pgs: 129 active+clean; 64 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-05-14T15:34:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:33 smithi193 ceph-mon[36001]: pgmap v305: 129 pgs: 129 active+clean; 64 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-05-14T15:34:35.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 913940 9.72 MB/sec execute 356 sec latency 38.613 ms 2024-05-14T15:34:36.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:35 smithi005 ceph-mon[31559]: pgmap v306: 129 pgs: 129 active+clean; 64 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-14T15:34:36.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 916035 9.73 MB/sec execute 357 sec latency 16.432 ms 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 -- 172.21.15.5:0/3991907842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34107df0 msgr2=0x7f8f34108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/3991907842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34107df0 0x7f8f34108200 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8f240099c0 tx=0x7f8f24009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 -- 172.21.15.5:0/3991907842 shutdown_connections 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/3991907842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f34108740 0x7f8f341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/3991907842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f341034c0 0x7f8f341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/3991907842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34107df0 0x7f8f34108200 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.263+0000 7f8f3be0e700 1 -- 172.21.15.5:0/3991907842 >> 172.21.15.5:0/3991907842 conn(0x7f8f340fedc0 msgr2=0x7f8f341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.264+0000 7f8f3be0e700 1 -- 172.21.15.5:0/3991907842 shutdown_connections 2024-05-14T15:34:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.264+0000 7f8f3be0e700 1 -- 172.21.15.5:0/3991907842 wait complete. 2024-05-14T15:34:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.264+0000 7f8f3be0e700 1 Processor -- start 2024-05-14T15:34:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.264+0000 7f8f3be0e700 1 -- start start 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3be0e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f341034c0 0x7f8f3419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3be0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 0x7f8f3419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3be0e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 0x7f8f3419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3be0e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34113350 con 0x7f8f34108740 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3be0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341131d0 con 0x7f8f34107df0 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3be0e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341134d0 con 0x7f8f341034c0 2024-05-14T15:34:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3a3ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 0x7f8f3419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 0x7f8f3419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3a3ab700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 0x7f8f3419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47334/0 (socket says 172.21.15.5:47334) 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 0x7f8f3419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49386/0 (socket says 172.21.15.5:49386) 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3a3ab700 1 -- 172.21.15.5:0/4283812281 learned_addr learned my addr 172.21.15.5:0/4283812281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f39baa700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f341034c0 0x7f8f3419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 -- 172.21.15.5:0/4283812281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f341034c0 msgr2=0x7f8f3419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f341034c0 0x7f8f3419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 -- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 msgr2=0x7f8f3419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 0x7f8f3419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f393a9700 1 -- 172.21.15.5:0/4283812281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f341a6f80 con 0x7f8f34107df0 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.265+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 0x7f8f3419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.266+0000 7f8f393a9700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 0x7f8f3419c170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098e0 tx=0x7f8f24004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.266+0000 7f8f2affd700 1 -- 172.21.15.5:0/4283812281 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24026070 con 0x7f8f34107df0 2024-05-14T15:34:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.266+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f341a7210 con 0x7f8f34107df0 2024-05-14T15:34:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.266+0000 7f8f2affd700 1 -- 172.21.15.5:0/4283812281 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f24003c70 con 0x7f8f34107df0 2024-05-14T15:34:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.266+0000 7f8f2affd700 1 -- 172.21.15.5:0/4283812281 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400f9c0 con 0x7f8f34107df0 2024-05-14T15:34:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.266+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341a76c0 con 0x7f8f34107df0 2024-05-14T15:34:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.270+0000 7f8f2affd700 1 -- 172.21.15.5:0/4283812281 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8f24029070 con 0x7f8f34107df0 2024-05-14T15:34:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.270+0000 7f8f4a822b80 1 Processor -- start 2024-05-14T15:34:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 -- start start 2024-05-14T15:34:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebc5ae40 0x556aebde2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebddcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebddd220 0x556aebddd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556aebcfa940 con 0x556aebddd220 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556aebc5fda0 con 0x556aebc5ae40 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556aebcf6900 con 0x556aebde30e0 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f2affd700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f200b29f0 0x7f8f200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebddcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f393a9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebddd220 0x556aebddd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebddcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59792/0 (socket says 172.21.15.5:59792) 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 -- 172.21.15.5:0/1852984619 learned_addr learned my addr 172.21.15.5:0/1852984619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebc5ae40 0x556aebde2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f200b29f0 0x7f8f200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f2affd700 1 -- 172.21.15.5:0/4283812281 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f8f240e33e0 con 0x7f8f34107df0 2024-05-14T15:34:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 -- 172.21.15.5:0/1852984619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebc5ae40 msgr2=0x556aebde2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebc5ae40 0x556aebde2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 -- 172.21.15.5:0/1852984619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebddd220 msgr2=0x556aebddd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebddd220 0x556aebddd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.271+0000 7f8f39baa700 1 -- 172.21.15.5:0/1852984619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aebc478d0 con 0x556aebde30e0 2024-05-14T15:34:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.272+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f200b29f0 0x7f8f200b4f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009fd0 tx=0x7f8f2c00b040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.272+0000 7f8f39baa700 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebddcce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009d80 tx=0x7f8f30011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41f5df5ee2613712 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.272+0000 7f8f1ffff700 1 -- 172.21.15.5:0/1852984619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30004230 con 0x556aebde30e0 2024-05-14T15:34:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.272+0000 7f8f1ffff700 1 -- 172.21.15.5:0/1852984619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f300043d0 con 0x556aebde30e0 2024-05-14T15:34:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.272+0000 7f8f1ffff700 1 -- 172.21.15.5:0/1852984619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30019920 con 0x556aebde30e0 2024-05-14T15:34:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 -- 172.21.15.5:0/1852984619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 msgr2=0x556aebddcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebddcce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009d80 tx=0x7f8f30011720 comp rx=0 tx=0).stop 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 -- 172.21.15.5:0/1852984619 shutdown_connections 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebddd220 0x556aebddd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebc5ae40 0x556aebde2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/1852984619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebddcce0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 -- 172.21.15.5:0/1852984619 >> 172.21.15.5:0/1852984619 conn(0x556aebca2630 msgr2=0x556aebdd3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 -- 172.21.15.5:0/1852984619 shutdown_connections 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 -- 172.21.15.5:0/1852984619 wait complete. 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.273+0000 7f8f4a822b80 1 -- start start 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 Processor -- start 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebde6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebde7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556aebde9be0 con 0x556aebc5ae40 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556aebcb3ed0 con 0x556aebddd220 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556aebde9d60 con 0x556aebde30e0 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f4a822b80 10 client.0 _ll_register_callbacks cb 0x556aebc92950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47350/0 (socket says 172.21.15.5:47350) 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f39baa700 1 -- 172.21.15.5:0/505233133 learned_addr learned my addr 172.21.15.5:0/505233133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f393a9700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebde6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.274+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebde7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f393a9700 1 -- 172.21.15.5:0/505233133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 msgr2=0x556aebde7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f393a9700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebde7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f393a9700 1 -- 172.21.15.5:0/505233133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 msgr2=0x556aebde6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f393a9700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f393a9700 1 -- 172.21.15.5:0/505233133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aebcb1760 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f39baa700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f393a9700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebde6ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f240e0730 tx=0x7f8f24003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efb28f21a8f6609 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/505233133 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240a6210 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/505233133 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f8f240e3120 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f300043d0 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/505233133 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f24009e60 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f4a822b80 10 client.27311 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.275+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556aebdf4820 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/505233133 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f300043d0 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/505233133 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8f240a5cc0 con 0x556aebddd220 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f4a822b80 10 client.27311 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f4a822b80 10 client.27311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:34:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f4a822b80 20 client.27311 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebdf6de0 0x556aebdf92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f4a822b80 4 client.27311 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aebd08320 con 0x556aebdf6de0 2024-05-14T15:34:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f39baa700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebdf6de0 0x556aebdf92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f39baa700 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebdf6de0 0x556aebdf92b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=103018554a5c222 server_cookie=c1f29a1d3500bad0 in_seq=0 out_seq=0 2024-05-14T15:34:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.276+0000 7f8f1d7fa700 10 client.27311 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.277+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/505233133 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556aebd08320 con 0x556aebdf6de0 2024-05-14T15:34:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.277+0000 7f8f1d7fa700 10 client.27311 handle_command_reply: tid=0 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.277+0000 7f8f4a822b80 1 client.27311 shutdown 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.277+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 msgr2=0x556aebde6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.277+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebde6ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f240e0730 tx=0x7f8f24003680 comp rx=0 tx=0).stop 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 shutdown_connections 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebdf6de0 0x556aebdf92b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebde7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f1d7fa700 0 client.27311 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/505233133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebde6ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 >> 172.21.15.5:0/505233133 conn(0x556aebca2630 msgr2=0x556aebdd78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 shutdown_connections 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 1 -- 172.21.15.5:0/505233133 wait complete. 2024-05-14T15:34:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.278+0000 7f8f4a822b80 20 client.27311 trim_cache size 0 max 16384 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 Processor -- start 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 -- start start 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebddd220 0x556aebde7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 0x556aebdcdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341b0d60 con 0x556aebc5ae40 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341ae5f0 con 0x556aebde30e0 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.294+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341ac480 con 0x556aebddd220 2024-05-14T15:34:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 0x556aebdcdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47366/0 (socket says 172.21.15.5:47366) 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f39baa700 1 -- 172.21.15.5:0/3576841398 learned_addr learned my addr 172.21.15.5:0/3576841398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 0x556aebdcdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49416/0 (socket says 172.21.15.5:49416) 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f393a9700 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebddd220 0x556aebde7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 -- 172.21.15.5:0/3576841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebddd220 msgr2=0x556aebde7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebddd220 0x556aebde7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 -- 172.21.15.5:0/3576841398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 msgr2=0x556aebde2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 -- 172.21.15.5:0/3576841398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aebde4f00 con 0x556aebde30e0 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f393a9700 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebddd220 0x556aebde7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f39baa700 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 0x556aebdcdc90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00f010 tx=0x7f8f2c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720f96bef2a70d74 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.295+0000 7f8f1f7fe700 1 -- 172.21.15.5:0/3576841398 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c01ae20 con 0x556aebde30e0 2024-05-14T15:34:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.296+0000 7f8f1f7fe700 1 -- 172.21.15.5:0/3576841398 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c004bb0 con 0x556aebde30e0 2024-05-14T15:34:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.296+0000 7f8f1f7fe700 1 -- 172.21.15.5:0/3576841398 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c004020 con 0x556aebde30e0 2024-05-14T15:34:36.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:35 smithi144 ceph-mon[35650]: pgmap v306: 129 pgs: 129 active+clean; 64 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-14T15:34:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.296+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3576841398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 msgr2=0x556aebdcdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.296+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 0x556aebdcdc90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00f010 tx=0x7f8f2c009920 comp rx=0 tx=0).stop 2024-05-14T15:34:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.296+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3576841398 shutdown_connections 2024-05-14T15:34:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.296+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebde30e0 0x556aebdcdc90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebde2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3576841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebddd220 0x556aebde7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3576841398 >> 172.21.15.5:0/3576841398 conn(0x556aebca2630 msgr2=0x556aebdcd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3576841398 shutdown_connections 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3576841398 wait complete. 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 -- start start 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.297+0000 7f8f4a822b80 1 Processor -- start 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebdef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebdfdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebdfe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341acad0 con 0x556aebc5ae40 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341adf20 con 0x556aebddd220 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341ad180 con 0x556aebde30e0 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebdef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f4a822b80 10 client.0 _ll_register_callbacks cb 0x556aebc92950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebdef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47376/0 (socket says 172.21.15.5:47376) 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 -- 172.21.15.5:0/3526690518 learned_addr learned my addr 172.21.15.5:0/3526690518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f393a9700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebdfdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f3a3ab700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebdfe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 -- 172.21.15.5:0/3526690518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 msgr2=0x556aebdfe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebdfe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 -- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 msgr2=0x556aebdfdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebdfdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f39baa700 1 -- 172.21.15.5:0/3526690518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aebdef9e0 con 0x556aebc5ae40 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.298+0000 7f8f393a9700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebdfdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f39baa700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebdef1e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009230 tx=0x7f8f300042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920c7f8d9ce4a8c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/3526690518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f300043a0 con 0x556aebc5ae40 2024-05-14T15:34:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/3526690518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f8f30005250 con 0x556aebc5ae40 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f2c004bb0 con 0x556aebc5ae40 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/3526690518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f3003b050 con 0x556aebc5ae40 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f4a822b80 10 client.17697 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.299+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556aebe610a0 con 0x556aebc5ae40 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/3526690518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c004bb0 con 0x556aebc5ae40 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/3526690518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8f3002bce0 con 0x556aebc5ae40 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f4a822b80 10 client.17697 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f4a822b80 10 client.17697 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f4a822b80 20 client.17697 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebe61610 0x556aebe63a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f4a822b80 4 client.17697 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556aebd023c0 con 0x556aebe61610 2024-05-14T15:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.300+0000 7f8f393a9700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebe61610 0x556aebe63a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.301+0000 7f8f1d7fa700 10 client.17697 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.301+0000 7f8f393a9700 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebe61610 0x556aebe63a30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81843a4e9a115696 server_cookie=a70dc126d3d3412c in_seq=0 out_seq=0 2024-05-14T15:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.301+0000 7f8f1d7fa700 1 -- 172.21.15.5:0/3526690518 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556aebd023c0 con 0x556aebe61610 2024-05-14T15:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.301+0000 7f8f1d7fa700 10 client.17697 handle_command_reply: tid=0 2024-05-14T15:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.301+0000 7f8f4a822b80 1 client.17697 shutdown 2024-05-14T15:34:36.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:35 smithi193 ceph-mon[36001]: pgmap v306: 129 pgs: 129 active+clean; 64 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-14T15:34:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 msgr2=0x556aebdef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebdef1e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009230 tx=0x7f8f300042b0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 shutdown_connections 2024-05-14T15:34:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556aebe61610 0x556aebe63a30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556aebde30e0 0x556aebdfe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556aebc5ae40 0x556aebdef1e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 --2- 172.21.15.5:0/3526690518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556aebddd220 0x556aebdfdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f1d7fa700 0 client.17697 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 >> 172.21.15.5:0/3526690518 conn(0x556aebca2630 msgr2=0x556aebdd3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 shutdown_connections 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 1 -- 172.21.15.5:0/3526690518 wait complete. 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.302+0000 7f8f4a822b80 20 client.17697 trim_cache size 0 max 16384 2024-05-14T15:34:36.308 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f200b29f0 msgr2=0x7f8f200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f200b29f0 0x7f8f200b4f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009fd0 tx=0x7f8f2c00b040 comp rx=0 tx=0).stop 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 msgr2=0x7f8f3419c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 0x7f8f3419c170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8f240098e0 tx=0x7f8f24004270 comp rx=0 tx=0).stop 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 shutdown_connections 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f200b29f0 0x7f8f200b4f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f34108740 0x7f8f3419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f341034c0 0x7f8f3419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 --2- 172.21.15.5:0/4283812281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f34107df0 0x7f8f3419c170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 >> 172.21.15.5:0/4283812281 conn(0x7f8f340fedc0 msgr2=0x7f8f341096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 shutdown_connections 2024-05-14T15:34:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:36.303+0000 7f8f3be0e700 1 -- 172.21.15.5:0/4283812281 wait complete. 2024-05-14T15:34:37.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 916903 9.73 MB/sec execute 358 sec latency 66.906 ms 2024-05-14T15:34:37.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:34:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:38.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:37 smithi005 ceph-mon[31559]: pgmap v307: 129 pgs: 129 active+clean; 65 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-14T15:34:38.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 918322 9.75 MB/sec execute 359 sec latency 17.877 ms 2024-05-14T15:34:38.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:37 smithi144 ceph-mon[35650]: pgmap v307: 129 pgs: 129 active+clean; 65 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-14T15:34:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:37 smithi193 ceph-mon[36001]: pgmap v307: 129 pgs: 129 active+clean; 65 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-05-14T15:34:39.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 920523 9.76 MB/sec execute 360 sec latency 16.666 ms 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/299056664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 msgr2=0x7fbbac108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/299056664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 0x7fbbac108220 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbb940099c0 tx=0x7fbb94009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/299056664 shutdown_connections 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/299056664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbac108760 0x7fbbac1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/299056664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/299056664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 0x7fbbac108220 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/299056664 >> 172.21.15.5:0/299056664 conn(0x7fbbac0fef50 msgr2=0x7fbbac101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/299056664 shutdown_connections 2024-05-14T15:34:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.449+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/299056664 wait complete. 2024-05-14T15:34:39.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:39 smithi005 ceph-mon[31559]: pgmap v308: 129 pgs: 129 active+clean; 70 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 Processor -- start 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 -- start start 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 0x7fbbac19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbac108760 0x7fbbac19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1144a0 con 0x7fbbac107e10 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac114320 con 0x7fbbac1034e0 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb2e9c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac114620 con 0x7fbbac108760 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 0x7fbbac19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb0c38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb0c38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49442/0 (socket says 172.21.15.5:49442) 2024-05-14T15:34:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb0c38700 1 -- 172.21.15.5:0/2966334080 learned_addr learned my addr 172.21.15.5:0/2966334080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.450+0000 7fbbb1439700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbac108760 0x7fbbac19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb0c38700 1 -- 172.21.15.5:0/2966334080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbac108760 msgr2=0x7fbbac19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbac108760 0x7fbbac19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb0c38700 1 -- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 msgr2=0x7fbbac19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 0x7fbbac19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb0c38700 1 -- 172.21.15.5:0/2966334080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19fd40 con 0x7fbbac1034e0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac19ae70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00e9a0 tx=0x7fbb9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbba9ffb700 1 -- 172.21.15.5:0/2966334080 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00ca10 con 0x7fbbac1034e0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac1a0030 con 0x7fbbac1034e0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbba9ffb700 1 -- 172.21.15.5:0/2966334080 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c00cbb0 con 0x7fbbac1034e0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbba9ffb700 1 -- 172.21.15.5:0/2966334080 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c018680 con 0x7fbbac1034e0 2024-05-14T15:34:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.451+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a7950 con 0x7fbbac1034e0 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.454+0000 7fbba9ffb700 1 -- 172.21.15.5:0/2966334080 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbb9c010430 con 0x7fbbac1034e0 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbba9ffb700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb980b29f0 0x7fbb980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbabfff700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb980b29f0 0x7fbb980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbba9ffb700 1 -- 172.21.15.5:0/2966334080 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fbb9c0e2bd0 con 0x7fbbac1034e0 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbc18b0b80 1 Processor -- start 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbc18b0b80 1 -- start start 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbabfff700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb980b29f0 0x7fbb980b4f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb94000c00 tx=0x7fbb9401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b99608720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b9960cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.455+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9960d470 0x555b9960d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b9952a860 con 0x555b9960d470 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555b994c97c0 con 0x555b9948ae40 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b99526820 con 0x555b99608c60 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b99608720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b99608720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49456/0 (socket says 172.21.15.5:49456) 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb1439700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9960d470 0x555b9960d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 -- 172.21.15.5:0/749473960 learned_addr learned my addr 172.21.15.5:0/749473960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbabfff700 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b9960cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 -- 172.21.15.5:0/749473960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 msgr2=0x555b9960cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b9960cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 -- 172.21.15.5:0/749473960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9960d470 msgr2=0x555b9960d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9960d470 0x555b9960d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 -- 172.21.15.5:0/749473960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b994778d0 con 0x555b9948ae40 2024-05-14T15:34:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b99608720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0a5ad0 tx=0x7fbb9c0b22e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65da699a3557d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbba2ffd700 1 -- 172.21.15.5:0/749473960 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0a5050 con 0x555b9948ae40 2024-05-14T15:34:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbba2ffd700 1 -- 172.21.15.5:0/749473960 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c0b2040 con 0x555b9948ae40 2024-05-14T15:34:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.456+0000 7fbba2ffd700 1 -- 172.21.15.5:0/749473960 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0a9f10 con 0x555b9948ae40 2024-05-14T15:34:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/749473960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 msgr2=0x555b99608720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b99608720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0a5ad0 tx=0x7fbb9c0b22e0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/749473960 shutdown_connections 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9960d470 0x555b9960d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b99608720 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/749473960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b9960cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.457+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/749473960 >> 172.21.15.5:0/749473960 conn(0x555b994d2600 msgr2=0x555b996034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.458+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/749473960 shutdown_connections 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.458+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/749473960 wait complete. 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.458+0000 7fbbc18b0b80 1 -- start start 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.458+0000 7fbbc18b0b80 1 Processor -- start 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9948ae40 0x555b99612190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b996126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99612c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b99615420 con 0x555b9948ae40 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555b99490c30 con 0x555b9960d470 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b996155a0 con 0x555b99608c60 2024-05-14T15:34:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9948ae40 0x555b99612190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbc18b0b80 10 client.0 _ll_register_callbacks cb 0x555b994c2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b996126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb1439700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99612c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b996126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59872/0 (socket says 172.21.15.5:59872) 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbabfff700 1 -- 172.21.15.5:0/1623006894 learned_addr learned my addr 172.21.15.5:0/1623006894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 -- 172.21.15.5:0/1623006894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 msgr2=0x555b996126d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b996126d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 -- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 msgr2=0x555b99612c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99612c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 -- 172.21.15.5:0/1623006894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b994c6690 con 0x555b9948ae40 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9948ae40 0x555b99612190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c019000 tx=0x7fbb9c0aa230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ab8a908d815771 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.459+0000 7fbb8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbb8bfff700 1 -- 172.21.15.5:0/1623006894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0ade00 con 0x555b9948ae40 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbb8bfff700 1 -- 172.21.15.5:0/1623006894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fbb9c0a6040 con 0x555b9948ae40 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9c0b2040 con 0x555b9948ae40 2024-05-14T15:34:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbb8bfff700 1 -- 172.21.15.5:0/1623006894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c0b28b0 con 0x555b9948ae40 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbbc18b0b80 10 client.17709 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b99620040 con 0x555b9948ae40 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbb8bfff700 1 -- 172.21.15.5:0/1623006894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c0b2040 con 0x555b9948ae40 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbb8bfff700 1 -- 172.21.15.5:0/1623006894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbb9c010720 con 0x555b9948ae40 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbbc18b0b80 10 client.17709 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbbc18b0b80 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.460+0000 7fbbc18b0b80 20 client.17709 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.461+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b99622510 0x555b99624930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.461+0000 7fbbc18b0b80 4 client.17709 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.461+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b99538240 con 0x555b99622510 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.461+0000 7fbbabfff700 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b99622510 0x555b99624930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.461+0000 7fbb8bfff700 10 client.17709 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.461+0000 7fbbabfff700 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b99622510 0x555b99624930 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce75e0837ff994c7 server_cookie=8e2496d54fc53790 in_seq=0 out_seq=0 2024-05-14T15:34:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.462+0000 7fbb8bfff700 1 -- 172.21.15.5:0/1623006894 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b99538240 con 0x555b99622510 2024-05-14T15:34:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.462+0000 7fbb8bfff700 10 client.17709 handle_command_reply: tid=0 2024-05-14T15:34:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 client.17709 shutdown 2024-05-14T15:34:39.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9948ae40 msgr2=0x555b99612190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9948ae40 0x555b99612190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c019000 tx=0x7fbb9c0aa230 comp rx=0 tx=0).stop 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 shutdown_connections 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b99622510 0x555b99624930 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99612c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b9948ae40 0x555b99612190 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1623006894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b99608c60 0x555b996126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 >> 172.21.15.5:0/1623006894 conn(0x555b994d2600 msgr2=0x555b99607450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbb8bfff700 0 client.17709 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 shutdown_connections 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1623006894 wait complete. 2024-05-14T15:34:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.463+0000 7fbbc18b0b80 20 client.17709 trim_cache size 0 max 16384 2024-05-14T15:34:39.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 Processor -- start 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 -- start start 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b995fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b9960bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9960d470 0x555b9961e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1b3050 con 0x555b99608c60 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1b08e0 con 0x555b9948ae40 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.478+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1ae750 con 0x555b9960d470 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b9960bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b9960bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47438/0 (socket says 172.21.15.5:47438) 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 -- 172.21.15.5:0/1715440385 learned_addr learned my addr 172.21.15.5:0/1715440385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b995fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbb1439700 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9960d470 0x555b9961e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 -- 172.21.15.5:0/1715440385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9960d470 msgr2=0x555b9961e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9960d470 0x555b9961e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 -- 172.21.15.5:0/1715440385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 msgr2=0x555b995fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b995fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 -- 172.21.15.5:0/1715440385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b99610080 con 0x555b99608c60 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b995fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbbabfff700 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b9960bd60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009990 tx=0x7fbb9401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb37f2bbbf652ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbba27fc700 1 -- 172.21.15.5:0/1715440385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94038070 con 0x555b99608c60 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbba27fc700 1 -- 172.21.15.5:0/1715440385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb94033e50 con 0x555b99608c60 2024-05-14T15:34:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.479+0000 7fbba27fc700 1 -- 172.21.15.5:0/1715440385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400f460 con 0x555b99608c60 2024-05-14T15:34:39.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1715440385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 msgr2=0x555b9960bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b9960bd60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009990 tx=0x7fbb9401f000 comp rx=0 tx=0).stop 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1715440385 shutdown_connections 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9960d470 0x555b9961e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9948ae40 0x555b995fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/1715440385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b9960bd60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1715440385 >> 172.21.15.5:0/1715440385 conn(0x555b994d2600 msgr2=0x555b99602be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1715440385 shutdown_connections 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.480+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/1715440385 wait complete. 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 -- start start 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 Processor -- start 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9948ae40 0x555b99627b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b99624520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99624ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1aeda0 con 0x555b99608c60 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b99624520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1b0210 con 0x555b9960d470 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbc18b0b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac1af470 con 0x555b9948ae40 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b99624520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47452/0 (socket says 172.21.15.5:47452) 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbabfff700 1 -- 172.21.15.5:0/554573959 learned_addr learned my addr 172.21.15.5:0/554573959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9948ae40 0x555b99627b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.481+0000 7fbbb1439700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99624ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbc18b0b80 10 client.0 _ll_register_callbacks cb 0x555b99612080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbabfff700 1 -- 172.21.15.5:0/554573959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9948ae40 msgr2=0x555b99627b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbabfff700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9948ae40 0x555b99627b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbabfff700 1 -- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 msgr2=0x555b99624ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbabfff700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99624ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbabfff700 1 -- 172.21.15.5:0/554573959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b99625290 con 0x555b99608c60 2024-05-14T15:34:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbb1439700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99624ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9948ae40 0x555b99627b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbabfff700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b99624520 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009290 tx=0x7fbb940092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19ced6204d97df server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbb8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbb8bfff700 1 -- 172.21.15.5:0/554573959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94042020 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbb8bfff700 1 -- 172.21.15.5:0/554573959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fbb940339d0 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb94033e50 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbb8bfff700 1 -- 172.21.15.5:0/554573959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb94038030 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbc18b0b80 10 client.17715 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.482+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b9968b9f0 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbb8bfff700 1 -- 172.21.15.5:0/554573959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb94033e50 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbb8bfff700 1 -- 172.21.15.5:0/554573959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fbb94021070 con 0x555b99608c60 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbc18b0b80 10 client.17715 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbc18b0b80 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbc18b0b80 20 client.17715 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b9968c010 0x555b9968e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbc18b0b80 4 client.17715 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b9961f2a0 con 0x555b9968c010 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b9968c010 0x555b9968e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.483+0000 7fbb8bfff700 10 client.17715 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.484+0000 7fbbb0c38700 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b9968c010 0x555b9968e430 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2886de94bd97a6 server_cookie=72e41352ebd2cd16 in_seq=0 out_seq=0 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.484+0000 7fbb8bfff700 1 -- 172.21.15.5:0/554573959 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b9961f2a0 con 0x555b9968c010 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.484+0000 7fbb8bfff700 10 client.17715 handle_command_reply: tid=0 2024-05-14T15:34:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.484+0000 7fbbc18b0b80 1 client.17715 shutdown 2024-05-14T15:34:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 msgr2=0x555b99624520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b99624520 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009290 tx=0x7fbb940092c0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 shutdown_connections 2024-05-14T15:34:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x555b9968c010 0x555b9968e430 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555b9960d470 0x555b99624ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b9948ae40 0x555b99627b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 --2- 172.21.15.5:0/554573959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b99608c60 0x555b99624520 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbb8bfff700 0 client.17715 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 >> 172.21.15.5:0/554573959 conn(0x555b994d2600 msgr2=0x555b99610970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 shutdown_connections 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 1 -- 172.21.15.5:0/554573959 wait complete. 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbc18b0b80 20 client.17715 trim_cache size 0 max 16384 2024-05-14T15:34:39.491 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb980b29f0 msgr2=0x7fbb980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb980b29f0 0x7fbb980b4f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb94000c00 tx=0x7fbb9401a040 comp rx=0 tx=0).stop 2024-05-14T15:34:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 msgr2=0x7fbbac19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.485+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac19ae70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00e9a0 tx=0x7fbb9c00ecb0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 shutdown_connections 2024-05-14T15:34:39.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb980b29f0 0x7fbb980b4f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbbac108760 0x7fbbac19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac1034e0 0x7fbbac19ae70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 --2- 172.21.15.5:0/2966334080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbac107e10 0x7fbbac19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 >> 172.21.15.5:0/2966334080 conn(0x7fbbac0fef50 msgr2=0x7fbbac107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 shutdown_connections 2024-05-14T15:34:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:39.486+0000 7fbbb2e9c700 1 -- 172.21.15.5:0/2966334080 wait complete. 2024-05-14T15:34:39.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:39 smithi144 ceph-mon[35650]: pgmap v308: 129 pgs: 129 active+clean; 70 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-05-14T15:34:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:39 smithi193 ceph-mon[36001]: pgmap v308: 129 pgs: 129 active+clean; 70 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-05-14T15:34:40.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 922023 9.75 MB/sec execute 361 sec latency 20.617 ms 2024-05-14T15:34:41.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 924238 9.76 MB/sec execute 362 sec latency 17.090 ms 2024-05-14T15:34:41.343 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:34:41.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:34:41.639 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:42.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:41 smithi005 ceph-mon[31559]: pgmap v309: 129 pgs: 129 active+clean; 70 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-05-14T15:34:42.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 925770 9.75 MB/sec execute 363 sec latency 22.747 ms 2024-05-14T15:34:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:41 smithi144 ceph-mon[35650]: pgmap v309: 129 pgs: 129 active+clean; 70 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-05-14T15:34:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:41 smithi193 ceph-mon[36001]: pgmap v309: 129 pgs: 129 active+clean; 70 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2024-05-14T15:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.622+0000 7fec46fee640 1 -- 172.21.15.5:0/3645727461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40109060 msgr2=0x7fec40102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.622+0000 7fec46fee640 1 --2- 172.21.15.5:0/3645727461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40109060 0x7fec40102ca0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009900 tx=0x7fec2c02a440 comp rx=0 tx=0).stop 2024-05-14T15:34:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.623+0000 7fec46fee640 1 -- 172.21.15.5:0/3645727461 shutdown_connections 2024-05-14T15:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.623+0000 7fec46fee640 1 --2- 172.21.15.5:0/3645727461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec40103310 0x7fec4010f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.623+0000 7fec46fee640 1 --2- 172.21.15.5:0/3645727461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40109060 0x7fec40102ca0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.623+0000 7fec46fee640 1 --2- 172.21.15.5:0/3645727461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40108bc0 0x7fec40102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.623+0000 7fec46fee640 1 -- 172.21.15.5:0/3645727461 >> 172.21.15.5:0/3645727461 conn(0x7fec400781b0 msgr2=0x7fec40078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.624+0000 7fec46fee640 1 -- 172.21.15.5:0/3645727461 shutdown_connections 2024-05-14T15:34:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.624+0000 7fec46fee640 1 -- 172.21.15.5:0/3645727461 wait complete. 2024-05-14T15:34:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.625+0000 7fec46fee640 1 Processor -- start 2024-05-14T15:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.628+0000 7fec46fee640 1 -- start start 2024-05-14T15:34:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.628+0000 7fec46fee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40103310 0x7fec401b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.628+0000 7fec46fee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40108bc0 0x7fec4010c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.629+0000 7fec46fee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec40109060 0x7fec4010cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.629+0000 7fec46fee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec40113f20 con 0x7fec40108bc0 2024-05-14T15:34:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.629+0000 7fec46fee640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec40113da0 con 0x7fec40103310 2024-05-14T15:34:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.629+0000 7fec46fee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec401140a0 con 0x7fec40109060 2024-05-14T15:34:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.633+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40108bc0 0x7fec4010c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.633+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40108bc0 0x7fec4010c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47470/0 (socket says 172.21.15.5:47470) 2024-05-14T15:34:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.633+0000 7fec3ffff640 1 -- 172.21.15.5:0/3826309379 learned_addr learned my addr 172.21.15.5:0/3826309379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.634+0000 7fec44d8a640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40103310 0x7fec401b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.634+0000 7fec4558b640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec40109060 0x7fec4010cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.635+0000 7fec44d8a640 1 -- 172.21.15.5:0/3826309379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec40109060 msgr2=0x7fec4010cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.635+0000 7fec44d8a640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec40109060 0x7fec4010cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.635+0000 7fec44d8a640 1 -- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40108bc0 msgr2=0x7fec4010c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.635+0000 7fec44d8a640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40108bc0 0x7fec4010c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.635+0000 7fec44d8a640 1 -- 172.21.15.5:0/3826309379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4010d110 con 0x7fec40103310 2024-05-14T15:34:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.636+0000 7fec44d8a640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40103310 0x7fec401b22e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fec3400d6a0 tx=0x7fec3400db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.638+0000 7fec3dffb640 1 -- 172.21.15.5:0/3826309379 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34017070 con 0x7fec40103310 2024-05-14T15:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.638+0000 7fec3dffb640 1 -- 172.21.15.5:0/3826309379 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec34009820 con 0x7fec40103310 2024-05-14T15:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.638+0000 7fec3dffb640 1 -- 172.21.15.5:0/3826309379 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34004c40 con 0x7fec40103310 2024-05-14T15:34:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.640+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4010d400 con 0x7fec40103310 2024-05-14T15:34:42.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.641+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401ae050 con 0x7fec40103310 2024-05-14T15:34:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 Processor -- start 2024-05-14T15:34:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 -- start start 2024-05-14T15:34:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 0x556cffba4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffba5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7e34d0 0x556cffba5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cffb0a5d0 con 0x556cff7db2f0 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556cffac3890 con 0x556cff7e34d0 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.645+0000 7fec4df7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556cffb06590 con 0x556cff8041b0 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 0x556cffba4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 0x556cffba4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59916/0 (socket says 172.21.15.5:59916) 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 -- 172.21.15.5:0/3718754390 learned_addr learned my addr 172.21.15.5:0/3718754390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 -- 172.21.15.5:0/3718754390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7e34d0 msgr2=0x556cffba5660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:34:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7e34d0 0x556cffba5660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 -- 172.21.15.5:0/3718754390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 msgr2=0x556cffba5120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:34:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.646+0000 7fec4558b640 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffba5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.647+0000 7fec4558b640 1 -- 172.21.15.5:0/3718754390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cff81c210 con 0x556cff8041b0 2024-05-14T15:34:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.650+0000 7fec4558b640 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 0x556cffba4be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec38009df0 tx=0x7fec3800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49f0312748967cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.651+0000 7fec3dffb640 1 -- 172.21.15.5:0/3826309379 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fec340099d0 con 0x7fec40103310 2024-05-14T15:34:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.652+0000 7fec3dffb640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fec300b2b60 0x7fec300b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.652+0000 7fec3dffb640 1 -- 172.21.15.5:0/3826309379 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fec340e1e40 con 0x7fec40103310 2024-05-14T15:34:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.656+0000 7fec2affd640 1 -- 172.21.15.5:0/3718754390 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38003970 con 0x556cff8041b0 2024-05-14T15:34:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.656+0000 7fec2affd640 1 -- 172.21.15.5:0/3718754390 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38004030 con 0x556cff8041b0 2024-05-14T15:34:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.657+0000 7fec4df7a740 1 -- 172.21.15.5:0/3718754390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 msgr2=0x556cffba4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.657+0000 7fec4df7a740 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 0x556cffba4be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec38009df0 tx=0x7fec3800f730 comp rx=0 tx=0).stop 2024-05-14T15:34:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.660+0000 7fec44d8a640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fec300b2b60 0x7fec300b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.662+0000 7fec4df7a740 1 -- 172.21.15.5:0/3718754390 shutdown_connections 2024-05-14T15:34:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.662+0000 7fec4df7a740 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7e34d0 0x556cffba5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.662+0000 7fec4df7a740 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffba5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.662+0000 7fec4df7a740 1 --2- 172.21.15.5:0/3718754390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff8041b0 0x556cffba4be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.662+0000 7fec4df7a740 1 -- 172.21.15.5:0/3718754390 >> 172.21.15.5:0/3718754390 conn(0x556cff7f8300 msgr2=0x556cffba1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.664+0000 7fec44d8a640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fec300b2b60 0x7fec300b5070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec340a6030 tx=0x7fec340057f0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.667+0000 7fec4df7a740 1 -- 172.21.15.5:0/3718754390 shutdown_connections 2024-05-14T15:34:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.667+0000 7fec4df7a740 1 -- 172.21.15.5:0/3718754390 wait complete. 2024-05-14T15:34:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.668+0000 7fec4df7a740 1 -- start start 2024-05-14T15:34:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.668+0000 7fec4df7a740 1 Processor -- start 2024-05-14T15:34:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 0x556cffbaf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7e34d0 0x556cffbaf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbbb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cffbb1db0 con 0x556cff7e34d0 2024-05-14T15:34:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556cffac7370 con 0x556cff8041b0 2024-05-14T15:34:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556cffbb1f30 con 0x556cff7db2f0 2024-05-14T15:34:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.669+0000 7fec4df7a740 10 client.0 _ll_register_callbacks cb 0x556cff81ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 0x556cffbaf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 0x556cffbaf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59930/0 (socket says 172.21.15.5:59930) 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 -- 172.21.15.5:0/1598099674 learned_addr learned my addr 172.21.15.5:0/1598099674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbbb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 -- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 msgr2=0x556cffbbb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbbb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 -- 172.21.15.5:0/1598099674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7e34d0 msgr2=0x556cffbaf620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7e34d0 0x556cffbaf620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.670+0000 7fec4558b640 1 -- 172.21.15.5:0/1598099674 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cff8e8620 con 0x556cff7db2f0 2024-05-14T15:34:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.671+0000 7fec4558b640 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 0x556cffbaf0e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec380114a0 tx=0x7fec380096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ae8893f658f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.671+0000 7fec1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.671+0000 7fec1ffff640 1 -- 172.21.15.5:0/1598099674 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38023040 con 0x556cff7db2f0 2024-05-14T15:34:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.671+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec38004030 con 0x556cff7db2f0 2024-05-14T15:34:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.671+0000 7fec1ffff640 1 -- 172.21.15.5:0/1598099674 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fec38003da0 con 0x556cff7db2f0 2024-05-14T15:34:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.672+0000 7fec1ffff640 1 -- 172.21.15.5:0/1598099674 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fec380041e0 con 0x556cff7db2f0 2024-05-14T15:34:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.673+0000 7fec4df7a740 10 client.27313 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.673+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cffbbe060 con 0x556cff7db2f0 2024-05-14T15:34:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.673+0000 7fec1ffff640 1 -- 172.21.15.5:0/1598099674 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38004030 con 0x556cff7db2f0 2024-05-14T15:34:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec1ffff640 1 -- 172.21.15.5:0/1598099674 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fec3802a900 con 0x556cff7db2f0 2024-05-14T15:34:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec4df7a740 10 client.27313 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec4df7a740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec4df7a740 20 client.27313 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7c2d80 0x556cffbc1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec4df7a740 4 client.27313 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.674+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cffb181e0 con 0x556cff7c2d80 2024-05-14T15:34:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.679+0000 7fec3ffff640 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7c2d80 0x556cffbc1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.687+0000 7fec3ffff640 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7c2d80 0x556cffbc1900 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6256da24defa718 server_cookie=9940412e2df00baf in_seq=0 out_seq=0 2024-05-14T15:34:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.688+0000 7fec1ffff640 10 client.27313 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.690+0000 7fec1ffff640 1 -- 172.21.15.5:0/1598099674 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cffb181e0 con 0x556cff7c2d80 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.690+0000 7fec1ffff640 10 client.27313 handle_command_reply: tid=0 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.691+0000 7fec4df7a740 1 client.27313 shutdown 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.692+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 msgr2=0x556cffbaf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.692+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 0x556cffbaf0e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec380114a0 tx=0x7fec380096c0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 shutdown_connections 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7c2d80 0x556cffbc1900 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbbb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7e34d0 0x556cffbaf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1598099674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7db2f0 0x556cffbaf0e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 >> 172.21.15.5:0/1598099674 conn(0x556cff7f8300 msgr2=0x556cffba4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.693+0000 7fec1ffff640 0 client.27313 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.695+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 shutdown_connections 2024-05-14T15:34:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.698+0000 7fec4df7a740 1 -- 172.21.15.5:0/1598099674 wait complete. 2024-05-14T15:34:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.698+0000 7fec4df7a740 20 client.27313 trim_cache size 0 max 16384 2024-05-14T15:34:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.728+0000 7fec4df7a740 1 Processor -- start 2024-05-14T15:34:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.728+0000 7fec4df7a740 1 -- start start 2024-05-14T15:34:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 0x556cffba5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff8041b0 0x556cffbc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4df7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec401cb310 con 0x556cff8041b0 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4df7a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec401c8be0 con 0x556cff7db2f0 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4df7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec401c6a70 con 0x556cff7e34d0 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4558b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 0x556cffba5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4558b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 0x556cffba5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49542/0 (socket says 172.21.15.5:49542) 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4558b640 1 -- 172.21.15.5:0/2640685075 learned_addr learned my addr 172.21.15.5:0/2640685075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4558b640 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff8041b0 0x556cffbc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.729+0000 7fec4558b640 1 -- 172.21.15.5:0/2640685075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 msgr2=0x556cffbc7100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:34:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.730+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.731+0000 7fec4558b640 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbc7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.731+0000 7fec4558b640 1 -- 172.21.15.5:0/2640685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff8041b0 msgr2=0x556cffbc7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.731+0000 7fec4558b640 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff8041b0 0x556cffbc7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.731+0000 7fec4558b640 1 -- 172.21.15.5:0/2640685075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cff7c2fb0 con 0x556cff7db2f0 2024-05-14T15:34:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.732+0000 7fec3ffff640 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:34:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.732+0000 7fec4558b640 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 0x556cffba5080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x556cffbb04e0 tx=0x7fec3800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb4e30a2278003cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.733+0000 7fec2a7fc640 1 -- 172.21.15.5:0/2640685075 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3801e070 con 0x556cff7db2f0 2024-05-14T15:34:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.733+0000 7fec2a7fc640 1 -- 172.21.15.5:0/2640685075 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3800fc30 con 0x556cff7db2f0 2024-05-14T15:34:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.733+0000 7fec2a7fc640 1 -- 172.21.15.5:0/2640685075 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3803a040 con 0x556cff7db2f0 2024-05-14T15:34:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.735+0000 7fec4df7a740 1 -- 172.21.15.5:0/2640685075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 msgr2=0x556cffba5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.735+0000 7fec4df7a740 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 0x556cffba5080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x556cffbb04e0 tx=0x7fec3800f730 comp rx=0 tx=0).stop 2024-05-14T15:34:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.736+0000 7fec4df7a740 1 -- 172.21.15.5:0/2640685075 shutdown_connections 2024-05-14T15:34:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.736+0000 7fec4df7a740 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.736+0000 7fec4df7a740 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff8041b0 0x556cffbc7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.736+0000 7fec4df7a740 1 --2- 172.21.15.5:0/2640685075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff7db2f0 0x556cffba5080 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.736+0000 7fec4df7a740 1 -- 172.21.15.5:0/2640685075 >> 172.21.15.5:0/2640685075 conn(0x556cff7f8300 msgr2=0x556cffba4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.738+0000 7fec4df7a740 1 -- 172.21.15.5:0/2640685075 shutdown_connections 2024-05-14T15:34:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.738+0000 7fec4df7a740 1 -- 172.21.15.5:0/2640685075 wait complete. 2024-05-14T15:34:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.738+0000 7fec4df7a740 1 -- start start 2024-05-14T15:34:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.738+0000 7fec4df7a740 1 Processor -- start 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffbb74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbb2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec401c70c0 con 0x556cff7db2f0 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec401c8510 con 0x556cff8041b0 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec401c7770 con 0x556cff7e34d0 2024-05-14T15:34:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4df7a740 10 client.0 _ll_register_callbacks cb 0x7fec38009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.739+0000 7fec4558b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffbb74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbb1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59940/0 (socket says 172.21.15.5:59940) 2024-05-14T15:34:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 -- 172.21.15.5:0/1383354251 learned_addr learned my addr 172.21.15.5:0/1383354251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 -- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 msgr2=0x556cffbb2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbb2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 -- 172.21.15.5:0/1383354251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 msgr2=0x556cffbb74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffbb74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.740+0000 7fec3ffff640 1 -- 172.21.15.5:0/1383354251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cffbb29a0 con 0x556cff7e34d0 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.741+0000 7fec3ffff640 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbb1f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009850 tx=0x7fec2c009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785f4c880fb77b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.741+0000 7fec1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.741+0000 7fec1ffff640 1 -- 172.21.15.5:0/1383354251 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0453d0 con 0x556cff7e34d0 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.741+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3800fc30 con 0x556cff7e34d0 2024-05-14T15:34:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.742+0000 7fec1ffff640 1 -- 172.21.15.5:0/1383354251 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fec2c045570 con 0x556cff7e34d0 2024-05-14T15:34:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.744+0000 7fec1ffff640 1 -- 172.21.15.5:0/1383354251 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fec2c055050 con 0x556cff7e34d0 2024-05-14T15:34:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.746+0000 7fec4df7a740 10 client.27316 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.746+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cffc287b0 con 0x556cff7e34d0 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec1ffff640 1 -- 172.21.15.5:0/1383354251 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3800fc30 con 0x556cff7e34d0 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec1ffff640 1 -- 172.21.15.5:0/1383354251 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fec2c045ce0 con 0x556cff7e34d0 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec4df7a740 10 client.27316 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec4df7a740 10 client.27316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec4df7a740 20 client.27316 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7981c0 0x556cffc2b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec4df7a740 4 client.27316 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:34:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.750+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cffb12050 con 0x556cff7981c0 2024-05-14T15:34:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.752+0000 7fec4558b640 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7981c0 0x556cffc2b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.759+0000 7fec4558b640 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7981c0 0x556cffc2b000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02484579a49c9a9 server_cookie=21cef516ad3e01fb in_seq=0 out_seq=0 2024-05-14T15:34:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:42.761+0000 7fec1ffff640 10 client.27316 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:43.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 927279 9.76 MB/sec execute 364 sec latency 205.044 ms 2024-05-14T15:34:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.640+0000 7fec1ffff640 1 -- 172.21.15.5:0/1383354251 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cffb12050 con 0x556cff7981c0 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.640+0000 7fec1ffff640 10 client.27316 handle_command_reply: tid=0 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.640+0000 7fec4df7a740 1 client.27316 shutdown 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.640+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 msgr2=0x556cffbb1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.640+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbb1f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009850 tx=0x7fec2c009540 comp rx=0 tx=0).stop 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 shutdown_connections 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556cff7981c0 0x556cffc2b000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cff7e34d0 0x556cffbb1f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556cff8041b0 0x556cffbb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 --2- 172.21.15.5:0/1383354251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cff7db2f0 0x556cffbb74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 >> 172.21.15.5:0/1383354251 conn(0x556cff7f8300 msgr2=0x556cffba88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec1ffff640 0 client.27316 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 shutdown_connections 2024-05-14T15:34:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 1 -- 172.21.15.5:0/1383354251 wait complete. 2024-05-14T15:34:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.641+0000 7fec4df7a740 20 client.27316 trim_cache size 0 max 16384 2024-05-14T15:34:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:34:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:34:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:34:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:34:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fec300b2b60 msgr2=0x7fec300b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fec300b2b60 0x7fec300b5070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec340a6030 tx=0x7fec340057f0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40103310 msgr2=0x7fec401b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40103310 0x7fec401b22e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fec3400d6a0 tx=0x7fec3400db60 comp rx=0 tx=0).stop 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 shutdown_connections 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fec300b2b60 0x7fec300b5070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.642+0000 7fec46fee640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec40109060 0x7fec4010cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.643+0000 7fec46fee640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec40108bc0 0x7fec4010c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.643+0000 7fec46fee640 1 --2- 172.21.15.5:0/3826309379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40103310 0x7fec401b22e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.643+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 >> 172.21.15.5:0/3826309379 conn(0x7fec400781b0 msgr2=0x7fec400ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.643+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 shutdown_connections 2024-05-14T15:34:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:43.643+0000 7fec46fee640 1 -- 172.21.15.5:0/3826309379 wait complete. 2024-05-14T15:34:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.987+0000 7faf59c27700 1 -- 172.21.15.5:0/2850675322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 msgr2=0x7faf541100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.987+0000 7faf59c27700 1 --2- 172.21.15.5:0/2850675322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf541100a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009fc0 tx=0x7faf4c00b330 comp rx=0 tx=0).stop 2024-05-14T15:34:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 -- 172.21.15.5:0/2850675322 shutdown_connections 2024-05-14T15:34:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 --2- 172.21.15.5:0/2850675322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf541100a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 --2- 172.21.15.5:0/2850675322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf541034c0 0x7faf541038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 --2- 172.21.15.5:0/2850675322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf54107df0 0x7faf54108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 -- 172.21.15.5:0/2850675322 >> 172.21.15.5:0/2850675322 conn(0x7faf540fedc0 msgr2=0x7faf541011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 -- 172.21.15.5:0/2850675322 shutdown_connections 2024-05-14T15:34:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.988+0000 7faf59c27700 1 -- 172.21.15.5:0/2850675322 wait complete. 2024-05-14T15:34:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.989+0000 7faf59c27700 1 Processor -- start 2024-05-14T15:34:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.989+0000 7faf59c27700 1 -- start start 2024-05-14T15:34:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.989+0000 7faf59c27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf541034c0 0x7faf54199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.989+0000 7faf59c27700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf54107df0 0x7faf5419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf59c27700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf5419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf59c27700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf541133c0 con 0x7faf541034c0 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf541034c0 0x7faf54199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf541034c0 0x7faf54199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47494/0 (socket says 172.21.15.5:47494) 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf537fe700 1 -- 172.21.15.5:0/1152576265 learned_addr learned my addr 172.21.15.5:0/1152576265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf52ffd700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf54107df0 0x7faf5419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf53fff700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf5419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf54113240 con 0x7faf54107df0 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.990+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf54113540 con 0x7faf54108740 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.991+0000 7faf53fff700 1 -- 172.21.15.5:0/1152576265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf54107df0 msgr2=0x7faf5419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.991+0000 7faf53fff700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf54107df0 0x7faf5419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.991+0000 7faf53fff700 1 -- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf541034c0 msgr2=0x7faf54199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.991+0000 7faf53fff700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf541034c0 0x7faf54199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.991+0000 7faf53fff700 1 -- 172.21.15.5:0/1152576265 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5419ebe0 con 0x7faf54108740 2024-05-14T15:34:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.991+0000 7faf53fff700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf5419e540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00b6c0 tx=0x7faf4c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.992+0000 7faf50ff9700 1 -- 172.21.15.5:0/1152576265 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c021070 con 0x7faf54108740 2024-05-14T15:34:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.992+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5419ee70 con 0x7faf54108740 2024-05-14T15:34:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.992+0000 7faf50ff9700 1 -- 172.21.15.5:0/1152576265 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c007ac0 con 0x7faf54108740 2024-05-14T15:34:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.992+0000 7faf50ff9700 1 -- 172.21.15.5:0/1152576265 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c005040 con 0x7faf54108740 2024-05-14T15:34:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.992+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5406b610 con 0x7faf54108740 2024-05-14T15:34:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.997+0000 7faf50ff9700 1 -- 172.21.15.5:0/1152576265 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7faf4c024070 con 0x7faf54108740 2024-05-14T15:34:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.997+0000 7faf6863bb80 1 Processor -- start 2024-05-14T15:34:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 -- start start 2024-05-14T15:34:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 0x55eed52267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed5226ce0 0x55eed522afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed522b4f0 0x55eed522b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eed51488e0 con 0x55eed50a8ec0 2024-05-14T15:34:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eed50e7840 con 0x55eed5226ce0 2024-05-14T15:34:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.998+0000 7faf6863bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eed51448a0 con 0x55eed522b4f0 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.999+0000 7faf50ff9700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7faf3c0b4d00 0x7faf3c0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.999+0000 7faf50ff9700 1 -- 172.21.15.5:0/1152576265 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7faf4c00f020 con 0x7faf54108740 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.999+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 0x55eed52267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.999+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 0x55eed52267a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47508/0 (socket says 172.21.15.5:47508) 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.999+0000 7faf52ffd700 1 -- 172.21.15.5:0/3561936538 learned_addr learned my addr 172.21.15.5:0/3561936538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:43.999+0000 7faf52ffd700 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed522b4f0 0x55eed522b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.000+0000 7faf53fff700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7faf3c0b4d00 0x7faf3c0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.000+0000 7faf52ffd700 1 -- 172.21.15.5:0/3561936538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed522b4f0 msgr2=0x55eed522b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.000+0000 7faf52ffd700 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed522b4f0 0x55eed522b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.000+0000 7faf52ffd700 1 -- 172.21.15.5:0/3561936538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed5226ce0 msgr2=0x55eed522afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:34:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.000+0000 7faf52ffd700 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed5226ce0 0x55eed522afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.000+0000 7faf52ffd700 1 -- 172.21.15.5:0/3561936538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed5095950 con 0x55eed50a8ec0 2024-05-14T15:34:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.001+0000 7faf53fff700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7faf3c0b4d00 0x7faf3c0b7210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0e10a0 tx=0x7faf4c009ac0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.001+0000 7faf52ffd700 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 0x55eed52267a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7faf480137e0 tx=0x7faf480096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726b4a1479eda509 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.002+0000 7faf39ffb700 1 -- 172.21.15.5:0/3561936538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48013e00 con 0x55eed50a8ec0 2024-05-14T15:34:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.002+0000 7faf39ffb700 1 -- 172.21.15.5:0/3561936538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf48004500 con 0x55eed50a8ec0 2024-05-14T15:34:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.002+0000 7faf39ffb700 1 -- 172.21.15.5:0/3561936538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48005020 con 0x55eed50a8ec0 2024-05-14T15:34:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.003+0000 7faf6863bb80 1 -- 172.21.15.5:0/3561936538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 msgr2=0x55eed52267a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.003+0000 7faf6863bb80 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 0x55eed52267a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7faf480137e0 tx=0x7faf480096b0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.005+0000 7faf6863bb80 1 -- 172.21.15.5:0/3561936538 shutdown_connections 2024-05-14T15:34:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.005+0000 7faf6863bb80 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed522b4f0 0x55eed522b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.005+0000 7faf6863bb80 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed50a8ec0 0x55eed52267a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.005+0000 7faf6863bb80 1 --2- 172.21.15.5:0/3561936538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed5226ce0 0x55eed522afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.005+0000 7faf6863bb80 1 -- 172.21.15.5:0/3561936538 >> 172.21.15.5:0/3561936538 conn(0x55eed50f0680 msgr2=0x55eed5221530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:44.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.010+0000 7faf6863bb80 1 -- 172.21.15.5:0/3561936538 shutdown_connections 2024-05-14T15:34:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.011+0000 7faf6863bb80 1 -- 172.21.15.5:0/3561936538 wait complete. 2024-05-14T15:34:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.011+0000 7faf6863bb80 1 -- start start 2024-05-14T15:34:44.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 Processor -- start 2024-05-14T15:34:44.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed52302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed52307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed5230d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eed5233440 con 0x55eed5226ce0 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eed50aecb0 con 0x55eed522b4f0 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eed52335c0 con 0x55eed50a8ec0 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.012+0000 7faf6863bb80 10 client.0 _ll_register_callbacks cb 0x55eed50e09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.013+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed52302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.013+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed52302a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59996/0 (socket says 172.21.15.5:59996) 2024-05-14T15:34:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.013+0000 7faf52ffd700 1 -- 172.21.15.5:0/214042662 learned_addr learned my addr 172.21.15.5:0/214042662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.013+0000 7faf52ffd700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed5230d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.014+0000 7faf537fe700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed52307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.014+0000 7faf52ffd700 1 -- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 msgr2=0x55eed5230d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.014+0000 7faf52ffd700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed5230d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.014+0000 7faf52ffd700 1 -- 172.21.15.5:0/214042662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 msgr2=0x55eed52307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.014+0000 7faf52ffd700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed52307e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.014+0000 7faf52ffd700 1 -- 172.21.15.5:0/214042662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed50e4710 con 0x55eed50a8ec0 2024-05-14T15:34:44.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.015+0000 7faf52ffd700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed52302a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf48004a40 tx=0x7faf48004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4e247817419790 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.015+0000 7faf32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.016+0000 7faf32ffd700 1 -- 172.21.15.5:0/214042662 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480052e0 con 0x55eed50a8ec0 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.016+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf48004500 con 0x55eed50a8ec0 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.016+0000 7faf32ffd700 1 -- 172.21.15.5:0/214042662 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7faf48005480 con 0x55eed50a8ec0 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.016+0000 7faf32ffd700 1 -- 172.21.15.5:0/214042662 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf48005680 con 0x55eed50a8ec0 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 10 client.27328 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:44.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eed5235070 con 0x55eed50a8ec0 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf32ffd700 1 -- 172.21.15.5:0/214042662 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf48004500 con 0x55eed50a8ec0 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf32ffd700 1 -- 172.21.15.5:0/214042662 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7faf48020470 con 0x55eed50a8ec0 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 10 client.27328 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 10 client.27328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 20 client.27328 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52376b0 0x55eed5239ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 4 client.27328 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:44.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.017+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed51562c0 con 0x55eed52376b0 2024-05-14T15:34:44.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.018+0000 7faf537fe700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52376b0 0x55eed5239ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.019+0000 7faf537fe700 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52376b0 0x55eed5239ad0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481c880571db72a8 server_cookie=5271c6448d59b877 in_seq=0 out_seq=0 2024-05-14T15:34:44.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.019+0000 7faf32ffd700 10 client.27328 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:44.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.023+0000 7faf32ffd700 1 -- 172.21.15.5:0/214042662 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eed51562c0 con 0x55eed52376b0 2024-05-14T15:34:44.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.023+0000 7faf32ffd700 10 client.27328 handle_command_reply: tid=0 2024-05-14T15:34:44.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.023+0000 7faf6863bb80 1 client.27328 shutdown 2024-05-14T15:34:44.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.024+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 msgr2=0x55eed52302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:44.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.024+0000 7faf6863bb80 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed52302a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faf48004a40 tx=0x7faf48004b60 comp rx=0 tx=0).stop 2024-05-14T15:34:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 shutdown_connections 2024-05-14T15:34:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf6863bb80 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52376b0 0x55eed5239ad0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf6863bb80 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed5230d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf6863bb80 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed52302a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf6863bb80 1 --2- 172.21.15.5:0/214042662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed52307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 >> 172.21.15.5:0/214042662 conn(0x55eed50f0680 msgr2=0x55eed522b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.027+0000 7faf32ffd700 0 client.27328 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.028+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 shutdown_connections 2024-05-14T15:34:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.028+0000 7faf6863bb80 1 -- 172.21.15.5:0/214042662 wait complete. 2024-05-14T15:34:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.028+0000 7faf6863bb80 20 client.27328 trim_cache size 0 max 16384 2024-05-14T15:34:44.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:43 smithi005 ceph-mon[31559]: pgmap v310: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-05-14T15:34:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.067+0000 7faf6863bb80 1 Processor -- start 2024-05-14T15:34:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.067+0000 7faf6863bb80 1 -- start start 2024-05-14T15:34:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed5225f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed522c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed525b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf6863bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf541b0d70 con 0x55eed5226ce0 2024-05-14T15:34:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf6863bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf541ae600 con 0x55eed522b4f0 2024-05-14T15:34:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf6863bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf541ac470 con 0x55eed50a8ec0 2024-05-14T15:34:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed5225f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed525b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed5225f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:60008/0 (socket says 172.21.15.5:60008) 2024-05-14T15:34:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.068+0000 7faf52ffd700 1 -- 172.21.15.5:0/1778948132 learned_addr learned my addr 172.21.15.5:0/1778948132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.069+0000 7faf537fe700 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed522c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.070+0000 7faf52ffd700 1 -- 172.21.15.5:0/1778948132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 msgr2=0x55eed525b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.070+0000 7faf52ffd700 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed525b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.070+0000 7faf52ffd700 1 -- 172.21.15.5:0/1778948132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 msgr2=0x55eed522c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.070+0000 7faf52ffd700 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed522c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.070+0000 7faf52ffd700 1 -- 172.21.15.5:0/1778948132 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed521db60 con 0x55eed50a8ec0 2024-05-14T15:34:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.070+0000 7faf52ffd700 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed5225f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf48013dd0 tx=0x7faf48000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfc59bbd8dd410d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.071+0000 7faf397fa700 1 -- 172.21.15.5:0/1778948132 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480052e0 con 0x55eed50a8ec0 2024-05-14T15:34:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.071+0000 7faf397fa700 1 -- 172.21.15.5:0/1778948132 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf48005480 con 0x55eed50a8ec0 2024-05-14T15:34:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.071+0000 7faf397fa700 1 -- 172.21.15.5:0/1778948132 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48020a60 con 0x55eed50a8ec0 2024-05-14T15:34:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 -- 172.21.15.5:0/1778948132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 msgr2=0x55eed5225f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed5225f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faf48013dd0 tx=0x7faf48000bc0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 -- 172.21.15.5:0/1778948132 shutdown_connections 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed522b4f0 0x55eed525b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed50a8ec0 0x55eed5225f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 --2- 172.21.15.5:0/1778948132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed5226ce0 0x55eed522c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 -- 172.21.15.5:0/1778948132 >> 172.21.15.5:0/1778948132 conn(0x55eed50f0680 msgr2=0x55eed5221520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.073+0000 7faf6863bb80 1 -- 172.21.15.5:0/1778948132 shutdown_connections 2024-05-14T15:34:44.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.074+0000 7faf6863bb80 1 -- 172.21.15.5:0/1778948132 wait complete. 2024-05-14T15:34:44.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.075+0000 7faf6863bb80 1 -- start start 2024-05-14T15:34:44.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.075+0000 7faf6863bb80 1 Processor -- start 2024-05-14T15:34:44.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.076+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed50a8ec0 0x55eed523a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.076+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed5226ce0 0x55eed523a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.077+0000 7faf6863bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 0x55eed523ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.077+0000 7faf6863bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf541acac0 con 0x55eed522b4f0 2024-05-14T15:34:44.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.077+0000 7faf6863bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf541adf30 con 0x55eed50a8ec0 2024-05-14T15:34:44.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.077+0000 7faf6863bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf541ad190 con 0x55eed5226ce0 2024-05-14T15:34:44.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.077+0000 7faf6863bb80 10 client.0 _ll_register_callbacks cb 0x55eed50e09a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:44.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.078+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 0x55eed523ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.078+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 0x55eed523ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47528/0 (socket says 172.21.15.5:47528) 2024-05-14T15:34:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.078+0000 7faf52ffd700 1 -- 172.21.15.5:0/4063166360 learned_addr learned my addr 172.21.15.5:0/4063166360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf537fe700 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed5226ce0 0x55eed523a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf52ffd700 1 -- 172.21.15.5:0/4063166360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed5226ce0 msgr2=0x55eed523a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf52ffd700 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed5226ce0 0x55eed523a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf52ffd700 1 -- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed50a8ec0 msgr2=0x55eed523a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf52ffd700 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed50a8ec0 0x55eed523a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf52ffd700 1 -- 172.21.15.5:0/4063166360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed52bacf0 con 0x55eed522b4f0 2024-05-14T15:34:44.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf52ffd700 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 0x55eed523ad50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faf48005650 tx=0x7faf48009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4447a31265de35 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf32ffd700 1 -- 172.21.15.5:0/4063166360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480052e0 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf32ffd700 1 -- 172.21.15.5:0/4063166360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7faf4801c040 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.079+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf48005480 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf32ffd700 1 -- 172.21.15.5:0/4063166360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faf48004a40 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf6863bb80 10 client.17742 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eed52bbf20 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf32ffd700 1 -- 172.21.15.5:0/4063166360 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf48017070 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf32ffd700 1 -- 172.21.15.5:0/4063166360 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7faf48036410 con 0x55eed522b4f0 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf6863bb80 10 client.17742 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf6863bb80 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf6863bb80 20 client.17742 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:44.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.080+0000 7faf6863bb80 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52bc5f0 0x55eed52bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:44.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.081+0000 7faf6863bb80 4 client.17742 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:34:44.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.081+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7faf48005480 con 0x55eed52bc5f0 2024-05-14T15:34:44.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.082+0000 7faf537fe700 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52bc5f0 0x55eed52bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.086+0000 7faf32ffd700 10 client.17742 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.086+0000 7faf537fe700 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52bc5f0 0x55eed52bea10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f417c32db7c2c1d0 server_cookie=7e5992725bd6b672 in_seq=0 out_seq=0 2024-05-14T15:34:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.088+0000 7faf32ffd700 1 -- 172.21.15.5:0/4063166360 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7faf48005480 con 0x55eed52bc5f0 2024-05-14T15:34:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.088+0000 7faf32ffd700 10 client.17742 handle_command_reply: tid=0 2024-05-14T15:34:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.088+0000 7faf6863bb80 1 client.17742 shutdown 2024-05-14T15:34:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.089+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 msgr2=0x55eed523ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.089+0000 7faf6863bb80 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 0x55eed523ad50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faf48005650 tx=0x7faf48009510 comp rx=0 tx=0).stop 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 shutdown_connections 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf6863bb80 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55eed52bc5f0 0x55eed52bea10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf6863bb80 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed522b4f0 0x55eed523ad50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf6863bb80 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eed50a8ec0 0x55eed523a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf32ffd700 0 client.17742 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf6863bb80 1 --2- 172.21.15.5:0/4063166360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eed5226ce0 0x55eed523a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.090+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 >> 172.21.15.5:0/4063166360 conn(0x55eed50f0680 msgr2=0x55eed5221520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:44.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.092+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 shutdown_connections 2024-05-14T15:34:44.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.092+0000 7faf6863bb80 1 -- 172.21.15.5:0/4063166360 wait complete. 2024-05-14T15:34:44.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.092+0000 7faf6863bb80 20 client.17742 trim_cache size 0 max 16384 2024-05-14T15:34:44.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:34:44.095 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:34:44.095 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "446024fd-eefc-4826-9886-54d196c34f55", 2024-05-14T15:34:44.095 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:34:44.095 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:34:44.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.095+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7faf3c0b4d00 msgr2=0x7faf3c0b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:44.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.095+0000 7faf59c27700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7faf3c0b4d00 0x7faf3c0b7210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0e10a0 tx=0x7faf4c009ac0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.095+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 msgr2=0x7faf5419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:44.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.095+0000 7faf59c27700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf5419e540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00b6c0 tx=0x7faf4c00cc30 comp rx=0 tx=0).stop 2024-05-14T15:34:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.096+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 shutdown_connections 2024-05-14T15:34:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.097+0000 7faf59c27700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7faf3c0b4d00 0x7faf3c0b7210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.097+0000 7faf59c27700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf54108740 0x7faf5419e540 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.097+0000 7faf59c27700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf541034c0 0x7faf54199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.097+0000 7faf59c27700 1 --2- 172.21.15.5:0/1152576265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf54107df0 0x7faf5419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:44.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.097+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 >> 172.21.15.5:0/1152576265 conn(0x7faf540fedc0 msgr2=0x7faf54107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.099+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 shutdown_connections 2024-05-14T15:34:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:44.100+0000 7faf59c27700 1 -- 172.21.15.5:0/1152576265 wait complete. 2024-05-14T15:34:44.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 927455 9.73 MB/sec execute 365 sec latency 302.542 ms 2024-05-14T15:34:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:43 smithi144 ceph-mon[35650]: pgmap v310: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-05-14T15:34:44.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:43 smithi193 ceph-mon[36001]: pgmap v310: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-05-14T15:34:44.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:34:45.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 928281 9.71 MB/sec execute 366 sec latency 50.733 ms 2024-05-14T15:34:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:45.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:44 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:34:45.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:44 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:45.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:44 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:34:45.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:44 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:45.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:44 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:34:45.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:44 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:45.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:44 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:34:45.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:44 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:45.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:44 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:34:45.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:44 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:45.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:44 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:34:45.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:44 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:34:46.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:45 smithi005 ceph-mon[31559]: pgmap v311: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-05-14T15:34:46.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:45 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:34:46.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 930701 9.72 MB/sec execute 367 sec latency 16.894 ms 2024-05-14T15:34:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:45 smithi144 ceph-mon[35650]: pgmap v311: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-05-14T15:34:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:45 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:34:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:45 smithi193 ceph-mon[36001]: pgmap v311: 129 pgs: 129 active+clean; 78 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-05-14T15:34:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:45 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:34:47.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 931601 9.71 MB/sec execute 368 sec latency 36.886 ms 2024-05-14T15:34:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.328+0000 7f7aa71d2700 1 -- 172.21.15.5:0/2464428353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 msgr2=0x7f7aa0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.328+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/2464428353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa0108220 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0099c0 tx=0x7f7a8c009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 -- 172.21.15.5:0/2464428353 shutdown_connections 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/2464428353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7aa0108760 0x7f7aa01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/2464428353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/2464428353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa0108220 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 -- 172.21.15.5:0/2464428353 >> 172.21.15.5:0/2464428353 conn(0x7f7aa00fef50 msgr2=0x7f7aa0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 -- 172.21.15.5:0/2464428353 shutdown_connections 2024-05-14T15:34:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 -- 172.21.15.5:0/2464428353 wait complete. 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.329+0000 7f7aa71d2700 1 Processor -- start 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 -- start start 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0116700 con 0x7f7aa0107e10 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0116580 con 0x7f7aa0108760 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa71d2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa0116880 con 0x7f7aa01034e0 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa4f6e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa4f6e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39490/0 (socket says 172.21.15.5:39490) 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36384/0 (socket says 172.21.15.5:36384) 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa4f6e700 1 -- 172.21.15.5:0/1500364175 learned_addr learned my addr 172.21.15.5:0/1500364175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa576f700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 -- 172.21.15.5:0/1500364175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 msgr2=0x7f7aa019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 -- 172.21.15.5:0/1500364175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7aa0108760 msgr2=0x7f7aa01a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7a9ffff700 1 -- 172.21.15.5:0/1500364175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa01a1ea0 con 0x7f7aa0107e10 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.330+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.331+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d570 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0098e0 tx=0x7f7a8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.331+0000 7f7a9dffb700 1 -- 172.21.15.5:0/1500364175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c026070 con 0x7f7aa0107e10 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.331+0000 7f7a9dffb700 1 -- 172.21.15.5:0/1500364175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c003c70 con 0x7f7aa0107e10 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.331+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa01a2130 con 0x7f7aa0107e10 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.331+0000 7f7a9dffb700 1 -- 172.21.15.5:0/1500364175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00f9c0 con 0x7f7aa0107e10 2024-05-14T15:34:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.331+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa01a2610 con 0x7f7aa0107e10 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.334+0000 7f7a9dffb700 1 -- 172.21.15.5:0/1500364175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7a8c029070 con 0x7f7aa0107e10 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7a9dffb700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7a900b29f0 0x7f7a900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7a9dffb700 1 -- 172.21.15.5:0/1500364175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f7a8c0e3190 con 0x7f7aa0107e10 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7a900b29f0 0x7f7a900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 Processor -- start 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 -- start start 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x559964714720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x559964719880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7a900b29f0 0x7f7a900b4f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a94011550 tx=0x7f7a94009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559964636860 con 0x559964714c60 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599645d57c0 con 0x559964719470 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.335+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559964632820 con 0x559964596e40 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x559964719880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x559964719880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53646/0 (socket says 172.21.15.5:53646) 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 -- 172.21.15.5:0/2848829920 learned_addr learned my addr 172.21.15.5:0/2848829920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36400/0 (socket says 172.21.15.5:36400) 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x559964714720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 -- 172.21.15.5:0/2848829920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 msgr2=0x559964714720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x559964714720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 -- 172.21.15.5:0/2848829920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 msgr2=0x559964718f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 -- 172.21.15.5:0/2848829920 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599645838d0 con 0x559964719470 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7aa576f700 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x559964719880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b560 tx=0x7f7a9800bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a701fcc64aef1bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7a8affd700 1 -- 172.21.15.5:0/2848829920 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9801a030 con 0x559964719470 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7a8affd700 1 -- 172.21.15.5:0/2848829920 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a9800b590 con 0x559964719470 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.336+0000 7f7a8affd700 1 -- 172.21.15.5:0/2848829920 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98016c30 con 0x559964719470 2024-05-14T15:34:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2848829920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 msgr2=0x559964719880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x559964719880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b560 tx=0x7f7a9800bb30 comp rx=0 tx=0).stop 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2848829920 shutdown_connections 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x559964719880 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x559964714720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2848829920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2848829920 >> 172.21.15.5:0/2848829920 conn(0x5599645de600 msgr2=0x55996470f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2848829920 shutdown_connections 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.337+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2848829920 wait complete. 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.338+0000 7f7ab5be6b80 1 -- start start 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.338+0000 7f7ab5be6b80 1 Processor -- start 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.338+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x55996471e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.338+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964714c60 0x55996471e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.338+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964719470 0x55996471eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559964721420 con 0x559964719470 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55996459cc30 con 0x559964596e40 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599647215a0 con 0x559964714c60 2024-05-14T15:34:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7aa4f6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x55996471e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7aa576f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964719470 0x55996471eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964714c60 0x55996471e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7aa4f6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x55996471e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53654/0 (socket says 172.21.15.5:53654) 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7ab5be6b80 10 client.0 _ll_register_callbacks cb 0x5599645ce920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7aa4f6e700 1 -- 172.21.15.5:0/577226836 learned_addr learned my addr 172.21.15.5:0/577226836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 -- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 msgr2=0x55996471e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x55996471e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 -- 172.21.15.5:0/577226836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964719470 msgr2=0x55996471eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964719470 0x55996471eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 -- 172.21.15.5:0/577226836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599645d2690 con 0x559964714c60 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7aa576f700 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964719470 0x55996471eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964714c60 0x55996471e660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0e04e0 tx=0x7f7a8c009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=556a59859c8e8f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a7ffff700 1 -- 172.21.15.5:0/577226836 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c003e20 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a7ffff700 1 -- 172.21.15.5:0/577226836 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f7a8c0b2c50 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9800b590 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.339+0000 7f7a7ffff700 1 -- 172.21.15.5:0/577226836 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c0aa800 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 10 client.27343 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559964721480 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7a7ffff700 1 -- 172.21.15.5:0/577226836 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a9800b590 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7a7ffff700 1 -- 172.21.15.5:0/577226836 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7a8c00fb60 con 0x559964714c60 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 10 client.27343 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 10 client.27343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 20 client.27343 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55996472e550 0x559964730970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 4 client.27343 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559964644240 con 0x55996472e550 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.340+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55996472e550 0x559964730970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.341+0000 7f7a7ffff700 10 client.27343 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.341+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55996472e550 0x559964730970 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0c32928368eb7a server_cookie=396f3db206baa9aa in_seq=0 out_seq=0 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.341+0000 7f7a7ffff700 1 -- 172.21.15.5:0/577226836 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559964644240 con 0x55996472e550 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.341+0000 7f7a7ffff700 10 client.27343 handle_command_reply: tid=0 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 client.27343 shutdown 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964714c60 msgr2=0x55996471e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964714c60 0x55996471e660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0e04e0 tx=0x7f7a8c009f70 comp rx=0 tx=0).stop 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 shutdown_connections 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55996472e550 0x559964730970 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964719470 0x55996471eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x55996471e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7a7ffff700 0 client.27343 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/577226836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964714c60 0x55996471e660 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 >> 172.21.15.5:0/577226836 conn(0x5599645de600 msgr2=0x559964713450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 shutdown_connections 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/577226836 wait complete. 2024-05-14T15:34:47.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.342+0000 7f7ab5be6b80 20 client.27343 trim_cache size 0 max 16384 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 Processor -- start 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 -- start start 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x55996470f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x55996472a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b5100 con 0x559964714c60 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b2990 con 0x559964719470 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b0800 con 0x559964596e40 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36410/0 (socket says 172.21.15.5:36410) 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 -- 172.21.15.5:0/3998115637 learned_addr learned my addr 172.21.15.5:0/3998115637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7aa576f700 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x55996472a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x55996470f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 -- 172.21.15.5:0/3998115637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 msgr2=0x55996470f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x55996470f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 -- 172.21.15.5:0/3998115637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 msgr2=0x55996472a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x55996472a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.364+0000 7f7a9ffff700 1 -- 172.21.15.5:0/3998115637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559964710250 con 0x559964714c60 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.365+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718d90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0aa090 tx=0x7f7a8c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dc6306f06188ff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.365+0000 7f7a8a7fc700 1 -- 172.21.15.5:0/3998115637 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0aa100 con 0x559964714c60 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.365+0000 7f7a8a7fc700 1 -- 172.21.15.5:0/3998115637 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c0ae4b0 con 0x559964714c60 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.365+0000 7f7a8a7fc700 1 -- 172.21.15.5:0/3998115637 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0a9360 con 0x559964714c60 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/3998115637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 msgr2=0x559964718d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718d90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0aa090 tx=0x7f7a8c009310 comp rx=0 tx=0).stop 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/3998115637 shutdown_connections 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964719470 0x55996472a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964596e40 0x55996470f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/3998115637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x559964718d90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/3998115637 >> 172.21.15.5:0/3998115637 conn(0x5599645de600 msgr2=0x559964713450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/3998115637 shutdown_connections 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/3998115637 wait complete. 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 -- start start 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.366+0000 7f7ab5be6b80 1 Processor -- start 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x559964724370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x5599647248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964719470 0x559964724df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b0e50 con 0x559964714c60 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b22c0 con 0x559964596e40 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa01b1520 con 0x559964719470 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7ab5be6b80 10 client.0 _ll_register_callbacks cb 0x5599645ce920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x5599647248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7aa4f6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x559964724370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x5599647248b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36416/0 (socket says 172.21.15.5:36416) 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 -- 172.21.15.5:0/2198041714 learned_addr learned my addr 172.21.15.5:0/2198041714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7aa576f700 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964719470 0x559964724df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 -- 172.21.15.5:0/2198041714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964719470 msgr2=0x559964724df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964719470 0x559964724df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 -- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 msgr2=0x559964724370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x559964724370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.367+0000 7f7a9ffff700 1 -- 172.21.15.5:0/2198041714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559964797ce0 con 0x559964714c60 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a9ffff700 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x5599647248b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000c00 tx=0x7f7a8c010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4897eb5f395cf785 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a7ffff700 1 -- 172.21.15.5:0/2198041714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c003e20 con 0x559964714c60 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a7ffff700 1 -- 172.21.15.5:0/2198041714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f7a8c0a9360 con 0x559964714c60 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8c0ae4b0 con 0x559964714c60 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a7ffff700 1 -- 172.21.15.5:0/2198041714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c0aa100 con 0x559964714c60 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 10 client.17760 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559964797ce0 con 0x559964714c60 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a7ffff700 1 -- 172.21.15.5:0/2198041714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c0ae4b0 con 0x559964714c60 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7a7ffff700 1 -- 172.21.15.5:0/2198041714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f7a8c0a5b70 con 0x559964714c60 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 10 client.17760 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 20 client.17760 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559964799080 0x55996479b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 4 client.17760 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.368+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55996463e2e0 con 0x559964799080 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.369+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559964799080 0x55996479b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.369+0000 7f7aa4f6e700 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559964799080 0x55996479b4a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f3c45337efdceb server_cookie=531a699809aca9c7 in_seq=0 out_seq=0 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.369+0000 7f7a7ffff700 10 client.17760 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7a7ffff700 1 -- 172.21.15.5:0/2198041714 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55996463e2e0 con 0x559964799080 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7a7ffff700 10 client.17760 handle_command_reply: tid=0 2024-05-14T15:34:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 client.17760 shutdown 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 msgr2=0x5599647248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x5599647248b0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000c00 tx=0x7f7a8c010f80 comp rx=0 tx=0).stop 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 shutdown_connections 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559964799080 0x55996479b4a0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559964719470 0x559964724df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559964596e40 0x559964724370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 --2- 172.21.15.5:0/2198041714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559964714c60 0x5599647248b0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7a7ffff700 0 client.17760 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 >> 172.21.15.5:0/2198041714 conn(0x5599645de600 msgr2=0x55996471c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 shutdown_connections 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.370+0000 7f7ab5be6b80 1 -- 172.21.15.5:0/2198041714 wait complete. 2024-05-14T15:34:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7ab5be6b80 20 client.17760 trim_cache size 0 max 16384 2024-05-14T15:34:47.395 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:34:47.395 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:34:47.395 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:34:47.395 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:34:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7a900b29f0 msgr2=0x7f7a900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7a900b29f0 0x7f7a900b4f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a94011550 tx=0x7f7a94009260 comp rx=0 tx=0).stop 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 msgr2=0x7f7aa019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d570 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0098e0 tx=0x7f7a8c004270 comp rx=0 tx=0).stop 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 shutdown_connections 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7a900b29f0 0x7f7a900b4f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7aa0108760 0x7f7aa01a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa01034e0 0x7f7aa019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 --2- 172.21.15.5:0/1500364175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa0107e10 0x7f7aa019d570 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 >> 172.21.15.5:0/1500364175 conn(0x7f7aa00fef50 msgr2=0x7f7aa0107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 shutdown_connections 2024-05-14T15:34:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:47.371+0000 7f7aa71d2700 1 -- 172.21.15.5:0/1500364175 wait complete. 2024-05-14T15:34:48.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 933782 9.71 MB/sec execute 369 sec latency 17.452 ms 2024-05-14T15:34:48.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:47 smithi005 ceph-mon[31559]: pgmap v312: 129 pgs: 129 active+clean; 82 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-05-14T15:34:48.209 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:34:48.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:34:48.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:47 smithi144 ceph-mon[35650]: pgmap v312: 129 pgs: 129 active+clean; 82 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-05-14T15:34:48.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:47 smithi193 ceph-mon[36001]: pgmap v312: 129 pgs: 129 active+clean; 82 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 255 op/s 2024-05-14T15:34:48.466 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:49.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 934999 9.71 MB/sec execute 370 sec latency 35.306 ms 2024-05-14T15:34:50.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:49 smithi005 ceph-mon[31559]: pgmap v313: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 435 op/s 2024-05-14T15:34:50.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 936880 9.70 MB/sec execute 371 sec latency 29.014 ms 2024-05-14T15:34:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:49 smithi144 ceph-mon[35650]: pgmap v313: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 435 op/s 2024-05-14T15:34:50.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:49 smithi193 ceph-mon[36001]: pgmap v313: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.1 MiB/s wr, 435 op/s 2024-05-14T15:34:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.327+0000 7f2950afd700 1 -- 172.21.15.5:0/1689544659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c107e10 msgr2=0x7f294c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.327+0000 7f2950afd700 1 --2- 172.21.15.5:0/1689544659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c107e10 0x7f294c108220 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f29400099c0 tx=0x7f2940009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 -- 172.21.15.5:0/1689544659 shutdown_connections 2024-05-14T15:34:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 --2- 172.21.15.5:0/1689544659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c108760 0x7f294c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 --2- 172.21.15.5:0/1689544659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c1034e0 0x7f294c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 --2- 172.21.15.5:0/1689544659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c107e10 0x7f294c108220 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 -- 172.21.15.5:0/1689544659 >> 172.21.15.5:0/1689544659 conn(0x7f294c0fef30 msgr2=0x7f294c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 -- 172.21.15.5:0/1689544659 shutdown_connections 2024-05-14T15:34:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.328+0000 7f2950afd700 1 -- 172.21.15.5:0/1689544659 wait complete. 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 Processor -- start 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 -- start start 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 0x7f294c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 0x7f294c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 0x7f294c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f294c113370 con 0x7f294c108760 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1131f0 con 0x7f294c107e10 2024-05-14T15:34:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2950afd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1134f0 con 0x7f294c1034e0 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f294ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 0x7f294c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f294a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 0x7f294c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2949d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 0x7f294c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f294ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 0x7f294c1a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36432/0 (socket says 172.21.15.5:36432) 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f294a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 0x7f294c19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:39564/0 (socket says 172.21.15.5:39564) 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f2949d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 0x7f294c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53696/0 (socket says 172.21.15.5:53696) 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.329+0000 7f294ad9d700 1 -- 172.21.15.5:0/2394302541 learned_addr learned my addr 172.21.15.5:0/2394302541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294a59c700 1 -- 172.21.15.5:0/2394302541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 msgr2=0x7f294c19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294a59c700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 0x7f294c19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294a59c700 1 -- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 msgr2=0x7f294c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294a59c700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 0x7f294c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294a59c700 1 -- 172.21.15.5:0/2394302541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c1a1ed0 con 0x7f294c1034e0 2024-05-14T15:34:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294ad9d700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 0x7f294c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f2949d9b700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 0x7f294c19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f294a59c700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 0x7f294c19d000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c00c9c0 con 0x7f294c1034e0 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c00cb60 con 0x7f294c1034e0 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c1a21c0 con 0x7f294c1034e0 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c018680 con 0x7f294c1034e0 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.330+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1a9ae0 con 0x7f294c1034e0 2024-05-14T15:34:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.331+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c0512c0 con 0x7f294c1034e0 2024-05-14T15:34:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.333+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f293c010430 con 0x7f294c1034e0 2024-05-14T15:34:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.334+0000 7f293b7fe700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f29340b29f0 0x7f29340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.334+0000 7f2949d9b700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f29340b29f0 0x7f29340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.334+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f293c0e2f30 con 0x7f294c1034e0 2024-05-14T15:34:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.334+0000 7f2949d9b700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f29340b29f0 0x7f29340b4f00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2940000c00 tx=0x7f294001a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.338+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f293c0aa4b0 con 0x7f294c1034e0 2024-05-14T15:34:50.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.619+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f294c02e5f0 con 0x7f294c1034e0 2024-05-14T15:34:50.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.620+0000 7f293b7fe700 1 -- 172.21.15.5:0/2394302541 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7f293c0a9db0 con 0x7f294c1034e0 2024-05-14T15:34:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:34:50.623 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:34:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f29340b29f0 msgr2=0x7f29340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f29340b29f0 0x7f29340b4f00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2940000c00 tx=0x7f294001a040 comp rx=0 tx=0).stop 2024-05-14T15:34:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 msgr2=0x7f294c19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 0x7f294c19d000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00ed00 comp rx=0 tx=0).stop 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 shutdown_connections 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f29340b29f0 0x7f29340b4f00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f294c108760 0x7f294c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f294c1034e0 0x7f294c19d000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 --2- 172.21.15.5:0/2394302541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c107e10 0x7f294c19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 >> 172.21.15.5:0/2394302541 conn(0x7f294c0fef30 msgr2=0x7f294c10ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 shutdown_connections 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:50.623+0000 7f2950afd700 1 -- 172.21.15.5:0/2394302541 wait complete. 2024-05-14T15:34:50.626 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:34:51.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 938396 9.71 MB/sec execute 372 sec latency 20.483 ms 2024-05-14T15:34:51.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:50 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2394302541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:51.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:34:51.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:50 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2394302541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:51.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:50 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2394302541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:34:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:52.051 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:51 smithi005 ceph-mon[31559]: pgmap v314: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 387 op/s 2024-05-14T15:34:52.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939838 9.70 MB/sec execute 373 sec latency 17.190 ms 2024-05-14T15:34:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:51 smithi144 ceph-mon[35650]: pgmap v314: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 387 op/s 2024-05-14T15:34:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:51 smithi193 ceph-mon[36001]: pgmap v314: 129 pgs: 129 active+clean; 85 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 387 op/s 2024-05-14T15:34:53.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 941719 9.70 MB/sec execute 374 sec latency 30.921 ms 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 -- 172.21.15.5:0/2857842780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88108740 msgr2=0x7fea881100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 --2- 172.21.15.5:0/2857842780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88108740 0x7fea881100a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fea8000b0d0 tx=0x7fea8000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 -- 172.21.15.5:0/2857842780 shutdown_connections 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 --2- 172.21.15.5:0/2857842780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88108740 0x7fea881100a0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 --2- 172.21.15.5:0/2857842780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea881034c0 0x7fea881038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 --2- 172.21.15.5:0/2857842780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea88107df0 0x7fea88108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 -- 172.21.15.5:0/2857842780 >> 172.21.15.5:0/2857842780 conn(0x7fea880fedc0 msgr2=0x7fea881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 -- 172.21.15.5:0/2857842780 shutdown_connections 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.394+0000 7fea8e48b700 1 -- 172.21.15.5:0/2857842780 wait complete. 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 Processor -- start 2024-05-14T15:34:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 -- start start 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 0x7fea8819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 0x7fea8819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea88108740 0x7fea88199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113370 con 0x7fea88107df0 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea881131f0 con 0x7fea881034c0 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8e48b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea881134f0 con 0x7fea88108740 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 0x7fea8819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea87fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 0x7fea8819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 0x7fea8819fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36456/0 (socket says 172.21.15.5:36456) 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea87fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 0x7fea8819f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:53710/0 (socket says 172.21.15.5:53710) 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea877fe700 1 -- 172.21.15.5:0/4010221982 learned_addr learned my addr 172.21.15.5:0/4010221982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.395+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea88108740 0x7fea88199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8ca28700 1 -- 172.21.15.5:0/4010221982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 msgr2=0x7fea8819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 0x7fea8819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8ca28700 1 -- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 msgr2=0x7fea8819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 0x7fea8819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8ca28700 1 -- 172.21.15.5:0/4010221982 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819a260 con 0x7fea88108740 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea87fff700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 0x7fea8819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea877fe700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 0x7fea8819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea88108740 0x7fea88199b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea8000b0d0 tx=0x7fea80007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea857fa700 1 -- 172.21.15.5:0/4010221982 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8001f070 con 0x7fea88108740 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819a4f0 con 0x7fea88108740 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea857fa700 1 -- 172.21.15.5:0/4010221982 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea80004c50 con 0x7fea88108740 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea857fa700 1 -- 172.21.15.5:0/4010221982 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8001b430 con 0x7fea88108740 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.396+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a6750 con 0x7fea88108740 2024-05-14T15:34:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.399+0000 7fea857fa700 1 -- 172.21.15.5:0/4010221982 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fea80022070 con 0x7fea88108740 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea857fa700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fea700b29f0 0x7fea700b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 Processor -- start 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 -- start start 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea87fff700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fea700b29f0 0x7fea700b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea857fa700 1 -- 172.21.15.5:0/4010221982 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fea8000f020 con 0x7fea88108740 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123c3be40 0x558123dbb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558123cdb860 con 0x558123dbbe70 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558123c7a7c0 con 0x558123dc0680 2024-05-14T15:34:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558123cd7820 con 0x558123c3be40 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea87fff700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fea700b29f0 0x7fea700b4f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea78011490 tx=0x7fea78009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36464/0 (socket says 172.21.15.5:36464) 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea877fe700 1 -- 172.21.15.5:0/4038464519 learned_addr learned my addr 172.21.15.5:0/4038464519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.400+0000 7fea87fff700 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123c3be40 0x558123dbb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea877fe700 1 -- 172.21.15.5:0/4038464519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123c3be40 msgr2=0x558123dbb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea877fe700 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123c3be40 0x558123dbb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea877fe700 1 -- 172.21.15.5:0/4038464519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 msgr2=0x558123dc0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea877fe700 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea877fe700 1 -- 172.21.15.5:0/4038464519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558123c288d0 con 0x558123dbbe70 2024-05-14T15:34:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea877fe700 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc0140 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0115f0 tx=0x7fea7c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6624a23845d67a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea6e7fc700 1 -- 172.21.15.5:0/4038464519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c004230 con 0x558123dbbe70 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea6e7fc700 1 -- 172.21.15.5:0/4038464519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c0043d0 con 0x558123dbbe70 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.401+0000 7fea6e7fc700 1 -- 172.21.15.5:0/4038464519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c005620 con 0x558123dbbe70 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4038464519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 msgr2=0x558123dc0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc0140 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0115f0 tx=0x7fea7c011900 comp rx=0 tx=0).stop 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4038464519 shutdown_connections 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123c3be40 0x558123dbb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4038464519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc0140 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4038464519 >> 172.21.15.5:0/4038464519 conn(0x558123c83600 msgr2=0x558123db44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4038464519 shutdown_connections 2024-05-14T15:34:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4038464519 wait complete. 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.402+0000 7fea9ce9fb80 1 -- start start 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 Processor -- start 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dc52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dbbe70 0x558123dc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558123dc8630 con 0x558123c3be40 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558123c41c30 con 0x558123dc0680 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558123dc87b0 con 0x558123dbbe70 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dc52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea9ce9fb80 10 client.0 _ll_register_callbacks cb 0x558123c73920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dc52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36478/0 (socket says 172.21.15.5:36478) 2024-05-14T15:34:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 -- 172.21.15.5:0/1065477591 learned_addr learned my addr 172.21.15.5:0/1065477591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea877fe700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dbbe70 0x558123dc5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea8ca28700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 -- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dbbe70 msgr2=0x558123dc5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dbbe70 0x558123dc5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 -- 172.21.15.5:0/1065477591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 msgr2=0x558123dc5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.403+0000 7fea87fff700 1 -- 172.21.15.5:0/1065477591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558123c77690 con 0x558123c3be40 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea87fff700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dc52d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fea78003ad0 tx=0x7fea780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4a7fabc2deabd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea677fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea677fe700 1 -- 172.21.15.5:0/1065477591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7801fc20 con 0x558123c3be40 2024-05-14T15:34:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea677fe700 1 -- 172.21.15.5:0/1065477591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fea7801fdc0 con 0x558123c3be40 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7c0043d0 con 0x558123c3be40 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea677fe700 1 -- 172.21.15.5:0/1065477591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea7802d6b0 con 0x558123c3be40 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 10 client.17778 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558123dd31d0 con 0x558123c3be40 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea677fe700 1 -- 172.21.15.5:0/1065477591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c0043d0 con 0x558123c3be40 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea677fe700 1 -- 172.21.15.5:0/1065477591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fea78026070 con 0x558123c3be40 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 10 client.17778 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 10 client.17778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 20 client.17778 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123dd5950 0x558123dd7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 4 client.17778 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.404+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558123ce9240 con 0x558123dd5950 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.405+0000 7fea877fe700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123dd5950 0x558123dd7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.405+0000 7fea677fe700 10 client.17778 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.405+0000 7fea877fe700 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123dd5950 0x558123dd7dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae10ee91fafc6bf1 server_cookie=971d92be50a64d1e in_seq=0 out_seq=0 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea677fe700 1 -- 172.21.15.5:0/1065477591 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558123ce9240 con 0x558123dd5950 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea677fe700 10 client.17778 handle_command_reply: tid=0 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 client.17778 shutdown 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 msgr2=0x558123dc52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dc52d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fea78003ad0 tx=0x7fea780047c0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 shutdown_connections 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123dd5950 0x558123dd7dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dc0680 0x558123dc5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dc52d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/1065477591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dbbe70 0x558123dc5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea677fe700 0 client.17778 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 >> 172.21.15.5:0/1065477591 conn(0x558123c83600 msgr2=0x558123dba660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 shutdown_connections 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/1065477591 wait complete. 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.406+0000 7fea9ce9fb80 20 client.17778 trim_cache size 0 max 16384 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.421+0000 7fea9ce9fb80 1 Processor -- start 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 -- start start 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dbff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dbbe70 0x558123dc45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123dc5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b0d80 con 0x558123c3be40 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ae610 con 0x558123dbbe70 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ac480 con 0x558123dc0680 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dbff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dbff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36490/0 (socket says 172.21.15.5:36490) 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 -- 172.21.15.5:0/4092053516 learned_addr learned my addr 172.21.15.5:0/4092053516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea877fe700 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dbbe70 0x558123dc45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea8ca28700 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123dc5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 -- 172.21.15.5:0/4092053516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 msgr2=0x558123dc5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123dc5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 -- 172.21.15.5:0/4092053516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dbbe70 msgr2=0x558123dc45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dbbe70 0x558123dc45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 -- 172.21.15.5:0/4092053516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558123dc3840 con 0x558123c3be40 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.422+0000 7fea87fff700 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dbff80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fea78028510 tx=0x7fea7801f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f689e5e9d75a6d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.423+0000 7fea6dffb700 1 -- 172.21.15.5:0/4092053516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7803b070 con 0x558123c3be40 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.423+0000 7fea6dffb700 1 -- 172.21.15.5:0/4092053516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78011e50 con 0x558123c3be40 2024-05-14T15:34:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.423+0000 7fea6dffb700 1 -- 172.21.15.5:0/4092053516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7802dcb0 con 0x558123c3be40 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.423+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4092053516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 msgr2=0x558123dbff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.423+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dbff80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fea78028510 tx=0x7fea7801f7a0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4092053516 shutdown_connections 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123dc5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123c3be40 0x558123dbff80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/4092053516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123dbbe70 0x558123dc45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4092053516 >> 172.21.15.5:0/4092053516 conn(0x558123c83600 msgr2=0x558123db3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4092053516 shutdown_connections 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/4092053516 wait complete. 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 -- start start 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.424+0000 7fea9ce9fb80 1 Processor -- start 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123c3be40 0x558123dc8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123e3e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea881acad0 con 0x558123dbbe70 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea881adf40 con 0x558123c3be40 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ad1a0 con 0x558123dc0680 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 10 client.0 _ll_register_callbacks cb 0x558123c73920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36500/0 (socket says 172.21.15.5:36500) 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea87fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123c3be40 0x558123dc8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 -- 172.21.15.5:0/134043198 learned_addr learned my addr 172.21.15.5:0/134043198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea8ca28700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123e3e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 -- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 msgr2=0x558123e3e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123e3e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 -- 172.21.15.5:0/134043198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123c3be40 msgr2=0x558123dc8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123c3be40 0x558123dc8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 -- 172.21.15.5:0/134043198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558123dc9760 con 0x558123dbbe70 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea8ca28700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123e3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea87fff700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123c3be40 0x558123dc8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea877fe700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc8f40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0115c0 tx=0x7fea7c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdc0ff633c98913 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea677fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea677fe700 1 -- 172.21.15.5:0/134043198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c035060 con 0x558123dbbe70 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea78011e50 con 0x558123dbbe70 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.425+0000 7fea677fe700 1 -- 172.21.15.5:0/134043198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fea7c005620 con 0x558123dbbe70 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea677fe700 1 -- 172.21.15.5:0/134043198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea7c03c050 con 0x558123dbbe70 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 10 client.17784 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558123e3f660 con 0x558123dbbe70 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea677fe700 1 -- 172.21.15.5:0/134043198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78011e50 con 0x558123dbbe70 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea677fe700 1 -- 172.21.15.5:0/134043198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fea7c026550 con 0x558123dbbe70 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 10 client.17784 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 10 client.17784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 20 client.17784 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123e3fac0 0x558123e41ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 4 client.17784 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558123ce32e0 con 0x558123e3fac0 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.426+0000 7fea87fff700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123e3fac0 0x558123e41ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea87fff700 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123e3fac0 0x558123e41ee0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b640e65ee4a7fc99 server_cookie=5188c4d98e458d9a in_seq=0 out_seq=0 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea677fe700 10 client.17784 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea677fe700 1 -- 172.21.15.5:0/134043198 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558123ce32e0 con 0x558123e3fac0 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea677fe700 10 client.17784 handle_command_reply: tid=0 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 client.17784 shutdown 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 msgr2=0x558123dc8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc8f40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0115c0 tx=0x7fea7c0092f0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 shutdown_connections 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558123dc0680 0x558123e3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558123e3fac0 0x558123e41ee0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558123c3be40 0x558123dc8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea677fe700 0 client.17784 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 --2- 172.21.15.5:0/134043198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558123dbbe70 0x558123dc8f40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 >> 172.21.15.5:0/134043198 conn(0x558123c83600 msgr2=0x558123db4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 shutdown_connections 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.427+0000 7fea9ce9fb80 1 -- 172.21.15.5:0/134043198 wait complete. 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea9ce9fb80 20 client.17784 trim_cache size 0 max 16384 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fea700b29f0 msgr2=0x7fea700b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fea700b29f0 0x7fea700b4f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea78011490 tx=0x7fea78009260 comp rx=0 tx=0).stop 2024-05-14T15:34:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea88108740 msgr2=0x7fea88199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea88108740 0x7fea88199b30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea8000b0d0 tx=0x7fea80007a00 comp rx=0 tx=0).stop 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 shutdown_connections 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fea700b29f0 0x7fea700b4f00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea88108740 0x7fea88199b30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea881034c0 0x7fea8819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 --2- 172.21.15.5:0/4010221982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88107df0 0x7fea8819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 >> 172.21.15.5:0/4010221982 conn(0x7fea880fedc0 msgr2=0x7fea88107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 shutdown_connections 2024-05-14T15:34:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:53.428+0000 7fea8e48b700 1 -- 172.21.15.5:0/4010221982 wait complete. 2024-05-14T15:34:54.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 943125 9.69 MB/sec execute 375 sec latency 29.917 ms 2024-05-14T15:34:54.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:53 smithi005 ceph-mon[31559]: pgmap v315: 129 pgs: 129 active+clean; 92 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 448 op/s 2024-05-14T15:34:54.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:34:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:53 smithi144 ceph-mon[35650]: pgmap v315: 129 pgs: 129 active+clean; 92 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 448 op/s 2024-05-14T15:34:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:53 smithi193 ceph-mon[36001]: pgmap v315: 129 pgs: 129 active+clean; 92 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.3 MiB/s wr, 448 op/s 2024-05-14T15:34:54.501 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:55.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 945224 9.70 MB/sec execute 376 sec latency 14.846 ms 2024-05-14T15:34:56.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 946611 9.69 MB/sec execute 377 sec latency 26.449 ms 2024-05-14T15:34:56.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:55 smithi005 ceph-mon[31559]: pgmap v316: 129 pgs: 129 active+clean; 92 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 385 op/s 2024-05-14T15:34:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:55 smithi144 ceph-mon[35650]: pgmap v316: 129 pgs: 129 active+clean; 92 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 385 op/s 2024-05-14T15:34:56.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:55 smithi193 ceph-mon[36001]: pgmap v316: 129 pgs: 129 active+clean; 92 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 385 op/s 2024-05-14T15:34:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 -- 172.21.15.5:0/83439776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c0109870 msgr2=0x7f65c01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 --2- 172.21.15.5:0/83439776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c0109870 0x7f65c01111d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f65b800b0d0 tx=0x7f65b800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 -- 172.21.15.5:0/83439776 shutdown_connections 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 --2- 172.21.15.5:0/83439776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c0109870 0x7f65c01111d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 --2- 172.21.15.5:0/83439776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c01045f0 0x7f65c0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 --2- 172.21.15.5:0/83439776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c0108f20 0x7f65c0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 -- 172.21.15.5:0/83439776 >> 172.21.15.5:0/83439776 conn(0x7f65c00fffe0 msgr2=0x7f65c0102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 -- 172.21.15.5:0/83439776 shutdown_connections 2024-05-14T15:34:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.341+0000 7f65c858b700 1 -- 172.21.15.5:0/83439776 wait complete. 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 Processor -- start 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 -- start start 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 0x7f65c0075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c0108f20 0x7f65c0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0109870 0x7f65c0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01144a0 con 0x7f65c01045f0 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0114320 con 0x7f65c0109870 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c858b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0114620 con 0x7f65c0108f20 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c6327700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 0x7f65c0075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c6327700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 0x7f65c0075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36778/0 (socket says 172.21.15.5:36778) 2024-05-14T15:34:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.342+0000 7f65c6327700 1 -- 172.21.15.5:0/2207427444 learned_addr learned my addr 172.21.15.5:0/2207427444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c5b26700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c0108f20 0x7f65c0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6b28700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0109870 0x7f65c0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6327700 1 -- 172.21.15.5:0/2207427444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c0108f20 msgr2=0x7f65c0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6327700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c0108f20 0x7f65c0076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6327700 1 -- 172.21.15.5:0/2207427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0109870 msgr2=0x7f65c0076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6327700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0109870 0x7f65c0076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6327700 1 -- 172.21.15.5:0/2207427444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c01a9140 con 0x7f65c01045f0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6b28700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0109870 0x7f65c0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c6327700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 0x7f65c0075ac0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00e9a0 tx=0x7f65bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65b77fe700 1 -- 172.21.15.5:0/2207427444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00ca10 con 0x7f65c01045f0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65b77fe700 1 -- 172.21.15.5:0/2207427444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65bc00cbb0 con 0x7f65c01045f0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c01a9430 con 0x7f65c01045f0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65b77fe700 1 -- 172.21.15.5:0/2207427444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc018680 con 0x7f65c01045f0 2024-05-14T15:34:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.343+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c01a9970 con 0x7f65c01045f0 2024-05-14T15:34:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.346+0000 7f65b77fe700 1 -- 172.21.15.5:0/2207427444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f65bc010430 con 0x7f65c01045f0 2024-05-14T15:34:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.347+0000 7f65b77fe700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65ac0b29f0 0x7f65ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.347+0000 7f65c5b26700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65ac0b29f0 0x7f65ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.347+0000 7f65b77fe700 1 -- 172.21.15.5:0/2207427444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f65bc0e2c00 con 0x7f65c01045f0 2024-05-14T15:34:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.347+0000 7f65d6f9fb80 1 Processor -- start 2024-05-14T15:34:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.347+0000 7f65c5b26700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65ac0b29f0 0x7f65ac0b4f00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65c0108960 tx=0x7f65b0009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 -- start start 2024-05-14T15:34:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f5d6e7860 con 0x559f5d7ca470 2024-05-14T15:34:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f5d6867c0 con 0x559f5d7c5c60 2024-05-14T15:34:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f5d6e3820 con 0x559f5d647e40 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65c6b28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65c6327700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65c5b26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65c6b28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36788/0 (socket says 172.21.15.5:36788) 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65c6327700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51264/0 (socket says 172.21.15.5:51264) 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.348+0000 7f65c6b28700 1 -- 172.21.15.5:0/3226856713 learned_addr learned my addr 172.21.15.5:0/3226856713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65c6327700 1 -- 172.21.15.5:0/3226856713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 msgr2=0x559f5d7c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65c6327700 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65c6327700 1 -- 172.21.15.5:0/3226856713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 msgr2=0x559f5d7ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65c6327700 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65c6327700 1 -- 172.21.15.5:0/3226856713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5d6348d0 con 0x559f5d647e40 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65c6327700 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7c5720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0dff50 tx=0x7f65bc0b2300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f47881ac554e332 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65b4ff9700 1 -- 172.21.15.5:0/3226856713 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0a5050 con 0x559f5d647e40 2024-05-14T15:34:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65b4ff9700 1 -- 172.21.15.5:0/3226856713 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65bc0aa1e0 con 0x559f5d647e40 2024-05-14T15:34:56.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.349+0000 7f65b4ff9700 1 -- 172.21.15.5:0/3226856713 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0b24b0 con 0x559f5d647e40 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3226856713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 msgr2=0x559f5d7c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7c5720 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0dff50 tx=0x7f65bc0b2300 comp rx=0 tx=0).stop 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3226856713 shutdown_connections 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7c5720 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3226856713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3226856713 >> 172.21.15.5:0/3226856713 conn(0x559f5d68f600 msgr2=0x559f5d7c04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3226856713 shutdown_connections 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.350+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3226856713 wait complete. 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 -- start start 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 Processor -- start 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 0x559f5d7cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7ca470 0x559f5d7cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f5d7d2420 con 0x559f5d647e40 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f5d64dc30 con 0x559f5d7c5c60 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f5d7d25a0 con 0x559f5d7ca470 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65d6f9fb80 10 client.0 _ll_register_callbacks cb 0x559f5d67f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65c6327700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 0x559f5d7cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65c6327700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 0x559f5d7cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36802/0 (socket says 172.21.15.5:36802) 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65c6327700 1 -- 172.21.15.5:0/2178264841 learned_addr learned my addr 172.21.15.5:0/2178264841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65c5b26700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.351+0000 7f65c6b28700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7ca470 0x559f5d7cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65c6327700 1 -- 172.21.15.5:0/2178264841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7ca470 msgr2=0x559f5d7cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65c6327700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7ca470 0x559f5d7cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65c6327700 1 -- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 msgr2=0x559f5d7cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65c6327700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65c6327700 1 -- 172.21.15.5:0/2178264841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5d683690 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65c6327700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 0x559f5d7cf1e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0dffb0 tx=0x7f65bc0a9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cda115feded84ff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65a9ffb700 1 -- 172.21.15.5:0/2178264841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0ade90 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65a9ffb700 1 -- 172.21.15.5:0/2178264841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f65bc0b26c0 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65bc0aa1e0 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65a9ffb700 1 -- 172.21.15.5:0/2178264841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65bc0a7a30 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65d6f9fb80 10 client.17796 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f5d7dd080 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65a9ffb700 1 -- 172.21.15.5:0/2178264841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65bc0aa1e0 con 0x559f5d647e40 2024-05-14T15:34:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.352+0000 7f65a9ffb700 1 -- 172.21.15.5:0/2178264841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f65bc010720 con 0x559f5d647e40 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65d6f9fb80 10 client.17796 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65d6f9fb80 10 client.17796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65d6f9fb80 20 client.17796 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d7e1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65d6f9fb80 4 client.17796 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f5d6f5240 con 0x559f5d7df550 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65c5b26700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d7e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65c5b26700 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d7e1970 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ead967aea2ea06f9 server_cookie=5c0cd3242bc21b12 in_seq=0 out_seq=0 2024-05-14T15:34:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.353+0000 7f65a9ffb700 10 client.17796 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65a9ffb700 1 -- 172.21.15.5:0/2178264841 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f5d6f5240 con 0x559f5d7df550 2024-05-14T15:34:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65a9ffb700 10 client.17796 handle_command_reply: tid=0 2024-05-14T15:34:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 client.17796 shutdown 2024-05-14T15:34:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 msgr2=0x559f5d7cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 0x559f5d7cf1e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0dffb0 tx=0x7f65bc0a9f70 comp rx=0 tx=0).stop 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 shutdown_connections 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d7e1970 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65a9ffb700 0 client.17796 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7ca470 0x559f5d7cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d647e40 0x559f5d7cf1e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/2178264841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.354+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 >> 172.21.15.5:0/2178264841 conn(0x559f5d68f600 msgr2=0x559f5d7c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.355+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 shutdown_connections 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.355+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/2178264841 wait complete. 2024-05-14T15:34:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.355+0000 7f65d6f9fb80 20 client.17796 trim_cache size 0 max 16384 2024-05-14T15:34:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 Processor -- start 2024-05-14T15:34:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 -- start start 2024-05-14T15:34:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01b51b0 con 0x559f5d7ca470 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01b2a40 con 0x559f5d7c5c60 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01b08b0 con 0x559f5d647e40 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65c6b28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65c6b28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7bfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36808/0 (socket says 172.21.15.5:36808) 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65c6b28700 1 -- 172.21.15.5:0/681224326 learned_addr learned my addr 172.21.15.5:0/681224326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65c6327700 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.370+0000 7f65c5b26700 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65c6b28700 1 -- 172.21.15.5:0/681224326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 msgr2=0x559f5d7ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65c6b28700 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65c6b28700 1 -- 172.21.15.5:0/681224326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 msgr2=0x559f5d7ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65c6b28700 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65c6b28700 1 -- 172.21.15.5:0/681224326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5d7c6130 con 0x559f5d7ca470 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65c6b28700 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7bfbe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009310 tx=0x7f65b8009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8563dc74a903c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65abfff700 1 -- 172.21.15.5:0/681224326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8009460 con 0x559f5d7ca470 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65abfff700 1 -- 172.21.15.5:0/681224326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b80075c0 con 0x559f5d7ca470 2024-05-14T15:34:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.371+0000 7f65abfff700 1 -- 172.21.15.5:0/681224326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8005280 con 0x559f5d7ca470 2024-05-14T15:34:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/681224326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 msgr2=0x559f5d7bfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7bfbe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009310 tx=0x7f65b8009340 comp rx=0 tx=0).stop 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/681224326 shutdown_connections 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7bfbe0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d647e40 0x559f5d7ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/681224326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d7c5c60 0x559f5d7ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/681224326 >> 172.21.15.5:0/681224326 conn(0x559f5d68f600 msgr2=0x559f5d7cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/681224326 shutdown_connections 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/681224326 wait complete. 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 -- start start 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.372+0000 7f65d6f9fb80 1 Processor -- start 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 0x559f5d7e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7c5c60 0x559f5d7e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01b0f00 con 0x559f5d7ca470 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01b2370 con 0x559f5d647e40 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c01b15d0 con 0x559f5d7c5c60 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65d6f9fb80 10 client.0 _ll_register_callbacks cb 0x559f5d67f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c6327700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 0x559f5d7e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c6b28700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c6327700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 0x559f5d7e4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:33302/0 (socket says 172.21.15.5:33302) 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c6327700 1 -- 172.21.15.5:0/3343955828 learned_addr learned my addr 172.21.15.5:0/3343955828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c5b26700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7c5c60 0x559f5d7e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c5b26700 1 -- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 msgr2=0x559f5d7e4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c5b26700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 0x559f5d7e4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c5b26700 1 -- 172.21.15.5:0/3343955828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 msgr2=0x559f5d7e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c5b26700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c5b26700 1 -- 172.21.15.5:0/3343955828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5d7e4000 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c6327700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 0x559f5d7e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.373+0000 7f65c6b28700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65c5b26700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7c5c60 0x559f5d7e3370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f65b0011ae0 tx=0x7f65b00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2854713ade5a5713 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65a9ffb700 1 -- 172.21.15.5:0/3343955828 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0003cb0 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65a9ffb700 1 -- 172.21.15.5:0/3343955828 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f65b002a410 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b80075c0 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65a9ffb700 1 -- 172.21.15.5:0/3343955828 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65b003a050 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65d6f9fb80 10 client.27391 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f5d8489f0 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.374+0000 7f65a9ffb700 1 -- 172.21.15.5:0/3343955828 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b80075c0 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65a9ffb700 1 -- 172.21.15.5:0/3343955828 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f65b001e9b0 con 0x559f5d7c5c60 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 10 client.27391 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 10 client.27391 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 20 client.27391 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d84b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 4 client.27391 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f5d6ef2e0 con 0x559f5d7df550 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65c6327700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d84b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65c6327700 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d84b0c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=537618100e7480d5 server_cookie=ad00ea297a1af37 in_seq=0 out_seq=0 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65a9ffb700 10 client.27391 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65a9ffb700 1 -- 172.21.15.5:0/3343955828 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f5d6ef2e0 con 0x559f5d7df550 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65a9ffb700 10 client.27391 handle_command_reply: tid=0 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.375+0000 7f65d6f9fb80 1 client.27391 shutdown 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7c5c60 msgr2=0x559f5d7e3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7c5c60 0x559f5d7e3370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f65b0011ae0 tx=0x7f65b00047c0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 shutdown_connections 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x559f5d7df550 0x559f5d84b0c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f5d7ca470 0x559f5d7e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65a9ffb700 0 client.27391 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:34:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f5d647e40 0x559f5d7e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 --2- 172.21.15.5:0/3343955828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f5d7c5c60 0x559f5d7e3370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 >> 172.21.15.5:0/3343955828 conn(0x559f5d68f600 msgr2=0x559f5d7bfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 shutdown_connections 2024-05-14T15:34:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 1 -- 172.21.15.5:0/3343955828 wait complete. 2024-05-14T15:34:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65d6f9fb80 20 client.27391 trim_cache size 0 max 16384 2024-05-14T15:34:56.382 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65ac0b29f0 msgr2=0x7f65ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65c858b700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65ac0b29f0 0x7f65ac0b4f00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65c0108960 tx=0x7f65b0009260 comp rx=0 tx=0).stop 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 msgr2=0x7f65c0075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65c858b700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 0x7f65c0075ac0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00e9a0 tx=0x7f65bc00ecb0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.376+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 shutdown_connections 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f65ac0b29f0 0x7f65ac0b4f00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0109870 0x7f65c0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65c01045f0 0x7f65c0075ac0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 --2- 172.21.15.5:0/2207427444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c0108f20 0x7f65c0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 >> 172.21.15.5:0/2207427444 conn(0x7f65c00fffe0 msgr2=0x7f65c0106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 shutdown_connections 2024-05-14T15:34:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:56.377+0000 7f65c858b700 1 -- 172.21.15.5:0/2207427444 wait complete. 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 -- 172.21.15.5:0/751296299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0103950 msgr2=0x7f8fb010f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/751296299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0103950 0x7f8fb010f6c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009ca0 tx=0x7f8fa802a8c0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 -- 172.21.15.5:0/751296299 shutdown_connections 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/751296299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0103950 0x7f8fb010f6c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/751296299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0109040 0x7f8fb01032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/751296299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fb0108ba0 0x7f8fb0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 -- 172.21.15.5:0/751296299 >> 172.21.15.5:0/751296299 conn(0x7f8fb00779e0 msgr2=0x7f8fb0077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 -- 172.21.15.5:0/751296299 shutdown_connections 2024-05-14T15:34:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.758+0000 7f8fb72f2640 1 -- 172.21.15.5:0/751296299 wait complete. 2024-05-14T15:34:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.759+0000 7f8fb72f2640 1 Processor -- start 2024-05-14T15:34:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.759+0000 7f8fb72f2640 1 -- start start 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb72f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fb0103950 0x7f8fb01a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb72f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 0x7f8fb01a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb72f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0109040 0x7f8fb019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb72f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0112d90 con 0x7f8fb0108ba0 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb72f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0112c10 con 0x7f8fb0109040 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb72f2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0112f10 con 0x7f8fb0103950 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 0x7f8fb01a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 0x7f8fb01a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36842/0 (socket says 172.21.15.5:36842) 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 -- 172.21.15.5:0/791443720 learned_addr learned my addr 172.21.15.5:0/791443720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb588f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0109040 0x7f8fb019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb508e640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fb0103950 0x7f8fb01a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 -- 172.21.15.5:0/791443720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fb0103950 msgr2=0x7f8fb01a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fb0103950 0x7f8fb01a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 -- 172.21.15.5:0/791443720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0109040 msgr2=0x7f8fb019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0109040 0x7f8fb019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 -- 172.21.15.5:0/791443720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb019ad90 con 0x7f8fb0108ba0 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fb488d640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 0x7f8fb01a0a20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.760+0000 7f8fa67fc640 1 -- 172.21.15.5:0/791443720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000ccc0 con 0x7f8fb0108ba0 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.761+0000 7f8fa67fc640 1 -- 172.21.15.5:0/791443720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa00035f0 con 0x7f8fb0108ba0 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.761+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb019b020 con 0x7f8fb0108ba0 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.761+0000 7f8fa67fc640 1 -- 172.21.15.5:0/791443720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000ce60 con 0x7f8fb0108ba0 2024-05-14T15:34:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.761+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb00ff1d0 con 0x7f8fb0108ba0 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.763+0000 7f8fa67fc640 1 -- 172.21.15.5:0/791443720 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8fa0003aa0 con 0x7f8fb0108ba0 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.763+0000 7f8fbe1a6740 1 Processor -- start 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 -- start start 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 0x55dc7c870be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c871120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4af4d0 0x55dc7c871660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c7d65d0 con 0x55dc7c4af4d0 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c78f890 con 0x55dc7c4a72f0 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c7d2590 con 0x55dc7c4d01b0 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb488d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4af4d0 0x55dc7c871660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 0x55dc7c870be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 0x55dc7c870be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51318/0 (socket says 172.21.15.5:51318) 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb488d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4af4d0 0x55dc7c871660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36852/0 (socket says 172.21.15.5:36852) 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 -- 172.21.15.5:0/4114007451 learned_addr learned my addr 172.21.15.5:0/4114007451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 -- 172.21.15.5:0/4114007451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 msgr2=0x55dc7c871120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c871120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 -- 172.21.15.5:0/4114007451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4af4d0 msgr2=0x55dc7c871660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fa67fc640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f9c0b2950 0x7f8f9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4af4d0 0x55dc7c871660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.764+0000 7f8fb588f640 1 -- 172.21.15.5:0/4114007451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7c4e8210 con 0x55dc7c4d01b0 2024-05-14T15:34:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.765+0000 7f8fa67fc640 1 -- 172.21.15.5:0/791443720 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f8fa00e2d10 con 0x7f8fb0108ba0 2024-05-14T15:34:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.765+0000 7f8fb588f640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f9c0b2950 0x7f8f9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.765+0000 7f8fb588f640 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 0x55dc7c870be0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8fa802f410 tx=0x7f8fa802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3342cfea81b5d12a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.765+0000 7f8f9b7fe640 1 -- 172.21.15.5:0/4114007451 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8031920 con 0x55dc7c4d01b0 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.765+0000 7f8f9b7fe640 1 -- 172.21.15.5:0/4114007451 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa8036030 con 0x55dc7c4d01b0 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.765+0000 7f8f9b7fe640 1 -- 172.21.15.5:0/4114007451 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8031c10 con 0x55dc7c4d01b0 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/4114007451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 msgr2=0x55dc7c870be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 0x55dc7c870be0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8fa802f410 tx=0x7f8fa802f440 comp rx=0 tx=0).stop 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fb588f640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f9c0b2950 0x7f8f9c0b4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fa804a000 tx=0x7f8fa804b000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/4114007451 shutdown_connections 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4af4d0 0x55dc7c871660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c871120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/4114007451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4d01b0 0x55dc7c870be0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/4114007451 >> 172.21.15.5:0/4114007451 conn(0x55dc7c4c4300 msgr2=0x55dc7c86d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/4114007451 shutdown_connections 2024-05-14T15:34:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/4114007451 wait complete. 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.766+0000 7f8fbe1a6740 1 -- start start 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 Processor -- start 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c87b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 0x55dc7c887070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c87db90 con 0x55dc7c4d01b0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c793370 con 0x55dc7c4af4d0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dc7c87dd10 con 0x55dc7c4a72f0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fbe1a6740 10 client.0 _ll_register_callbacks cb 0x55dc7c4e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fb588f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 0x55dc7c887070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fb588f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 0x55dc7c887070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36868/0 (socket says 172.21.15.5:36868) 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fb588f640 1 -- 172.21.15.5:0/555782529 learned_addr learned my addr 172.21.15.5:0/555782529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fb488d640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c87b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.767+0000 7f8fb508e640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb588f640 1 -- 172.21.15.5:0/555782529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 msgr2=0x55dc7c87b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb588f640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb588f640 1 -- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 msgr2=0x55dc7c87b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb588f640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c87b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb588f640 1 -- 172.21.15.5:0/555782529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7c5b4620 con 0x55dc7c4d01b0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb488d640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c87b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fb588f640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 0x55dc7c887070 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8004540 tx=0x7f8fa802fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30fb05c49cc1bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8f98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8f98ff9640 1 -- 172.21.15.5:0/555782529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8031c10 con 0x55dc7c4d01b0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8f98ff9640 1 -- 172.21.15.5:0/555782529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f8fa804c050 con 0x55dc7c4d01b0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa8036030 con 0x55dc7c4d01b0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8f98ff9640 1 -- 172.21.15.5:0/555782529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fa8068050 con 0x55dc7c4d01b0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fbe1a6740 10 client.17814 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc7c87db90 con 0x55dc7c4d01b0 2024-05-14T15:34:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.768+0000 7f8f98ff9640 1 -- 172.21.15.5:0/555782529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fa8036030 con 0x55dc7c4d01b0 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8f98ff9640 1 -- 172.21.15.5:0/555782529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8fa80305c0 con 0x55dc7c4d01b0 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fbe1a6740 10 client.17814 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fbe1a6740 10 client.17814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fbe1a6740 20 client.17814 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c48ed80 0x55dc7c88d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fbe1a6740 4 client.17814 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7c7e41e0 con 0x55dc7c48ed80 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fb508e640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c48ed80 0x55dc7c88d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8f98ff9640 10 client.17814 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.769+0000 7f8fb508e640 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c48ed80 0x55dc7c88d050 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a43592bee37562 server_cookie=ec380ab1f5aff1e4 in_seq=0 out_seq=0 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.770+0000 7f8f98ff9640 1 -- 172.21.15.5:0/555782529 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc7c7e41e0 con 0x55dc7c48ed80 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.770+0000 7f8f98ff9640 10 client.17814 handle_command_reply: tid=0 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.770+0000 7f8fbe1a6740 1 client.17814 shutdown 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 msgr2=0x55dc7c887070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 0x55dc7c887070 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8004540 tx=0x7f8fa802fdf0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 shutdown_connections 2024-05-14T15:34:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c48ed80 0x55dc7c88d050 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c4d01b0 0x55dc7c887070 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c87b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8f98ff9640 0 client.17814 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/555782529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 >> 172.21.15.5:0/555782529 conn(0x55dc7c4c4300 msgr2=0x55dc7c8709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 shutdown_connections 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/555782529 wait complete. 2024-05-14T15:34:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.771+0000 7f8fbe1a6740 20 client.17814 trim_cache size 0 max 16384 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.786+0000 7f8fbe1a6740 1 Processor -- start 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.786+0000 7f8fbe1a6740 1 -- start start 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.786+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c8757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01b5f20 con 0x55dc7c48ed80 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01b37f0 con 0x55dc7c4a72f0 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01b1660 con 0x55dc7c4af4d0 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36880/0 (socket says 172.21.15.5:36880) 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 -- 172.21.15.5:0/776924086 learned_addr learned my addr 172.21.15.5:0/776924086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb488d640 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c8757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb588f640 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 -- 172.21.15.5:0/776924086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4af4d0 msgr2=0x55dc7c8927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8927d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 -- 172.21.15.5:0/776924086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 msgr2=0x55dc7c8757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c8757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 -- 172.21.15.5:0/776924086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7c8794e0 con 0x55dc7c48ed80 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8fb508e640 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87abe0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x55dc7c86e6d0 tx=0x7f8fac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b9f6b54501f405 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8f9affd640 1 -- 172.21.15.5:0/776924086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac010e60 con 0x55dc7c48ed80 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8f9affd640 1 -- 172.21.15.5:0/776924086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fac021070 con 0x55dc7c48ed80 2024-05-14T15:34:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.787+0000 7f8f9affd640 1 -- 172.21.15.5:0/776924086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0038e0 con 0x55dc7c48ed80 2024-05-14T15:34:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/776924086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 msgr2=0x55dc7c87abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87abe0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x55dc7c86e6d0 tx=0x7f8fac010800 comp rx=0 tx=0).stop 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/776924086 shutdown_connections 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4a72f0 0x55dc7c8757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/776924086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87abe0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/776924086 >> 172.21.15.5:0/776924086 conn(0x55dc7c4c4300 msgr2=0x55dc7c87a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/776924086 shutdown_connections 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.788+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/776924086 wait complete. 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 -- start start 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 Processor -- start 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01b1cb0 con 0x55dc7c48ed80 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01b3120 con 0x55dc7c4af4d0 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01b2380 con 0x55dc7c4a72f0 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fbe1a6740 10 client.0 _ll_register_callbacks cb 0x7f8fa8049020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fb508e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fb508e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36890/0 (socket says 172.21.15.5:36890) 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fb508e640 1 -- 172.21.15.5:0/24595487 learned_addr learned my addr 172.21.15.5:0/24595487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fb588f640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.789+0000 7f8fb488d640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fb508e640 1 -- 172.21.15.5:0/24595487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 msgr2=0x55dc7c87e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fb508e640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fb508e640 1 -- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 msgr2=0x55dc7c8ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fb508e640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8ed340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fb508e640 1 -- 172.21.15.5:0/24595487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7c8ed8b0 con 0x55dc7c48ed80 2024-05-14T15:34:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fb508e640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87e3c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8fac010f50 tx=0x7f8fac010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e8871086562f19 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8f98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8f98ff9640 1 -- 172.21.15.5:0/24595487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac01c3d0 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8f98ff9640 1 -- 172.21.15.5:0/24595487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f8fac01c570 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fac021070 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8f98ff9640 1 -- 172.21.15.5:0/24595487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fac038050 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fbe1a6740 10 client.17820 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dc7c8f4be0 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.790+0000 7f8f98ff9640 1 -- 172.21.15.5:0/24595487 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fac021070 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8f98ff9640 1 -- 172.21.15.5:0/24595487 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8fac02d3f0 con 0x55dc7c48ed80 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fbe1a6740 10 client.17820 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fbe1a6740 10 client.17820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fbe1a6740 20 client.17820 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c4d01b0 0x55dc7c8f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fbe1a6740 4 client.17820 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:34:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc7c7de050 con 0x55dc7c4d01b0 2024-05-14T15:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fb488d640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c4d01b0 0x55dc7c8f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8fb488d640 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c4d01b0 0x55dc7c8f7360 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb5857d85cb92b1 server_cookie=8e5ddead30174295 in_seq=0 out_seq=0 2024-05-14T15:34:56.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:56.791+0000 7f8f98ff9640 10 client.17820 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.121+0000 7f8f98ff9640 1 -- 172.21.15.5:0/24595487 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc7c7de050 con 0x55dc7c4d01b0 2024-05-14T15:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.121+0000 7f8f98ff9640 10 client.17820 handle_command_reply: tid=0 2024-05-14T15:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.121+0000 7f8fbe1a6740 1 client.17820 shutdown 2024-05-14T15:34:57.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.121+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 msgr2=0x55dc7c87e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.121+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87e3c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8fac010f50 tx=0x7f8fac010f80 comp rx=0 tx=0).stop 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 shutdown_connections 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55dc7c4d01b0 0x55dc7c8f7360 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dc7c4a72f0 0x55dc7c87e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dc7c4af4d0 0x55dc7c8ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 --2- 172.21.15.5:0/24595487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dc7c48ed80 0x55dc7c87e3c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 >> 172.21.15.5:0/24595487 conn(0x55dc7c4c4300 msgr2=0x55dc7c870970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8f98ff9640 0 client.17820 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 shutdown_connections 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 1 -- 172.21.15.5:0/24595487 wait complete. 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.122+0000 7f8fbe1a6740 20 client.17820 trim_cache size 0 max 16384 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f9c0b2950 msgr2=0x7f8f9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f9c0b2950 0x7f8f9c0b4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8fa804a000 tx=0x7f8fa804b000 comp rx=0 tx=0).stop 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 msgr2=0x7f8fb01a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 0x7f8fb01a0a20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000eeb0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 shutdown_connections 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f9c0b2950 0x7f8f9c0b4e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0109040 0x7f8fb019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb0108ba0 0x7f8fb01a0a20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 --2- 172.21.15.5:0/791443720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fb0103950 0x7f8fb01a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 >> 172.21.15.5:0/791443720 conn(0x7f8fb00779e0 msgr2=0x7f8fb01020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 shutdown_connections 2024-05-14T15:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:34:57.123+0000 7f8fb72f2640 1 -- 172.21.15.5:0/791443720 wait complete. 2024-05-14T15:34:57.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 948455 9.69 MB/sec execute 378 sec latency 252.928 ms 2024-05-14T15:34:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:34:58.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:57 smithi005 ceph-mon[31559]: pgmap v317: 129 pgs: 129 active+clean; 95 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 413 op/s 2024-05-14T15:34:58.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 949390 9.68 MB/sec execute 379 sec latency 18.898 ms 2024-05-14T15:34:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:57 smithi144 ceph-mon[35650]: pgmap v317: 129 pgs: 129 active+clean; 95 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 413 op/s 2024-05-14T15:34:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:57 smithi193 ceph-mon[36001]: pgmap v317: 129 pgs: 129 active+clean; 95 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 413 op/s 2024-05-14T15:34:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3412394805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c1034c0 msgr2=0x7f4c1c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3412394805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c1038d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4c180099c0 tx=0x7f4c18009cd0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3412394805 shutdown_connections 2024-05-14T15:34:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3412394805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c108740 0x7f4c1c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3412394805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c1038d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3412394805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c1c107df0 0x7f4c1c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3412394805 >> 172.21.15.5:0/3412394805 conn(0x7f4c1c0fedf0 msgr2=0x7f4c1c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3412394805 shutdown_connections 2024-05-14T15:34:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.149+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3412394805 wait complete. 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 Processor -- start 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 -- start start 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c107df0 0x7f4c1c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c1c108740 0x7f4c1c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c113390 con 0x7f4c1c107df0 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c113210 con 0x7f4c1c108740 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c22f4d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c113510 con 0x7f4c1c1034c0 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c107df0 0x7f4c1c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.150+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:51380/0 (socket says 172.21.15.5:51380) 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3912428773 learned_addr learned my addr 172.21.15.5:0/3912428773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c214ea700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c1c108740 0x7f4c1c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3912428773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c1c108740 msgr2=0x7f4c1c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c1c108740 0x7f4c1c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c107df0 msgr2=0x7f4c1c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c107df0 0x7f4c1c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3912428773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c19fce0 con 0x7f4c1c1034c0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c13fff700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c107df0 0x7f4c1c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c19ae10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c18006010 tx=0x7f4c18005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c11ffb700 1 -- 172.21.15.5:0/3912428773 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18026070 con 0x7f4c1c1034c0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c11ffb700 1 -- 172.21.15.5:0/3912428773 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c18005680 con 0x7f4c1c1034c0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1c19ff70 con 0x7f4c1c1034c0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.151+0000 7f4c11ffb700 1 -- 172.21.15.5:0/3912428773 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800fb20 con 0x7f4c1c1034c0 2024-05-14T15:34:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.152+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1c1a0450 con 0x7f4c1c1034c0 2024-05-14T15:34:59.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 951531 9.68 MB/sec execute 380 sec latency 57.854 ms 2024-05-14T15:34:59.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.154+0000 7f4c11ffb700 1 -- 172.21.15.5:0/3912428773 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4c18029070 con 0x7f4c1c1034c0 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.155+0000 7f4c11ffb700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4c040b29f0 0x7f4c040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.155+0000 7f4c13fff700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4c040b29f0 0x7f4c040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.155+0000 7f4c11ffb700 1 -- 172.21.15.5:0/3912428773 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f4c180e3180 con 0x7f4c1c1034c0 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.155+0000 7f4c31961b80 1 Processor -- start 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 -- start start 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c13fff700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4c040b29f0 0x7f4c040b4f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c1028c0 tx=0x7f4c0c009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 0x55de677b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677aee10 0x55de677af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de676d0940 con 0x55de677aee10 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55de67635da0 con 0x55de67630e40 2024-05-14T15:34:59.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c31961b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55de676cc900 con 0x55de677b4d80 2024-05-14T15:34:59.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c214ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677aee10 0x55de677af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 0x55de677b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c214ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677aee10 0x55de677af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36918/0 (socket says 172.21.15.5:36918) 2024-05-14T15:34:59.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 0x55de677b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:33410/0 (socket says 172.21.15.5:33410) 2024-05-14T15:34:59.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c214ea700 1 -- 172.21.15.5:0/3560032475 learned_addr learned my addr 172.21.15.5:0/3560032475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:59.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c13fff700 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3560032475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 msgr2=0x55de677ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3560032475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677aee10 msgr2=0x55de677af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677aee10 0x55de677af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.156+0000 7f4c20ce9700 1 -- 172.21.15.5:0/3560032475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de6761d8d0 con 0x55de67630e40 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.157+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 0x55de677b4840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c180e04d0 tx=0x7f4c180a99b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eeea67ef0317bde server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.157+0000 7f4c02ffd700 1 -- 172.21.15.5:0/3560032475 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800f460 con 0x55de67630e40 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.157+0000 7f4c02ffd700 1 -- 172.21.15.5:0/3560032475 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c180ae160 con 0x55de67630e40 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.157+0000 7f4c02ffd700 1 -- 172.21.15.5:0/3560032475 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c180a8f10 con 0x55de67630e40 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 -- 172.21.15.5:0/3560032475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 msgr2=0x55de677b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 0x55de677b4840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c180e04d0 tx=0x7f4c180a99b0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 -- 172.21.15.5:0/3560032475 shutdown_connections 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677aee10 0x55de677af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de67630e40 0x55de677b4840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 --2- 172.21.15.5:0/3560032475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 -- 172.21.15.5:0/3560032475 >> 172.21.15.5:0/3560032475 conn(0x55de67678630 msgr2=0x55de677a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 -- 172.21.15.5:0/3560032475 shutdown_connections 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 -- 172.21.15.5:0/3560032475 wait complete. 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.158+0000 7f4c31961b80 1 -- start start 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 Processor -- start 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de67630e40 0x55de677b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 0x55de677b9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de677bb5e0 con 0x55de677b4d80 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55de67689ed0 con 0x55de677aee10 2024-05-14T15:34:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55de677bb760 con 0x55de67630e40 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c31961b80 10 client.0 _ll_register_callbacks cb 0x55de67668950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c214ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 0x55de677b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c214ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 0x55de677b9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36924/0 (socket says 172.21.15.5:36924) 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c214ea700 1 -- 172.21.15.5:0/440353706 learned_addr learned my addr 172.21.15.5:0/440353706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.159+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de67630e40 0x55de677b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c13fff700 1 -- 172.21.15.5:0/440353706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de67630e40 msgr2=0x55de677b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c13fff700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de67630e40 0x55de677b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c13fff700 1 -- 172.21.15.5:0/440353706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 msgr2=0x55de677b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c13fff700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 0x55de677b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c13fff700 1 -- 172.21.15.5:0/440353706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de67687760 con 0x55de677aee10 2024-05-14T15:34:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c214ea700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 0x55de677b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c13fff700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677b8ac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55de677ab1f0 tx=0x7f4c0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269434a67dc91680 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4bfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4bfbfff700 1 -- 172.21.15.5:0/440353706 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c01e800 con 0x55de677aee10 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c180ae160 con 0x55de677aee10 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4bfbfff700 1 -- 172.21.15.5:0/440353706 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f4c0c01e9a0 con 0x55de677aee10 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4bfbfff700 1 -- 172.21.15.5:0/440353706 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c0c031050 con 0x55de677aee10 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c31961b80 10 client.27446 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.160+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55de677c62a0 con 0x55de677aee10 2024-05-14T15:34:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4bfbfff700 1 -- 172.21.15.5:0/440353706 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c180ae160 con 0x55de677aee10 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4bfbfff700 1 -- 172.21.15.5:0/440353706 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4c0c026020 con 0x55de677aee10 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c31961b80 10 client.27446 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c31961b80 10 client.27446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c31961b80 20 client.27446 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c31961b80 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de677c8830 0x55de677cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c31961b80 4 client.27446 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de676de320 con 0x55de677c8830 2024-05-14T15:34:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.161+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de677c8830 0x55de677cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.162+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de677c8830 0x55de677cac50 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309812dcdd9d4007 server_cookie=4aee8b133a203ff3 in_seq=0 out_seq=0 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.162+0000 7f4bfbfff700 10 client.27446 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.162+0000 7f4bfbfff700 1 -- 172.21.15.5:0/440353706 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de676de320 con 0x55de677c8830 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.162+0000 7f4bfbfff700 10 client.27446 handle_command_reply: tid=0 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.162+0000 7f4c31961b80 1 client.27446 shutdown 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 msgr2=0x55de677b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677b8ac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55de677ab1f0 tx=0x7f4c0c0047c0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 shutdown_connections 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de677c8830 0x55de677cac50 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de677b4d80 0x55de677b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de67630e40 0x55de677b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4bfbfff700 0 client.27446 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 --2- 172.21.15.5:0/440353706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677b8ac0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 >> 172.21.15.5:0/440353706 conn(0x55de67678630 msgr2=0x55de677ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 shutdown_connections 2024-05-14T15:34:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 1 -- 172.21.15.5:0/440353706 wait complete. 2024-05-14T15:34:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.163+0000 7f4c31961b80 20 client.27446 trim_cache size 0 max 16384 2024-05-14T15:34:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 Processor -- start 2024-05-14T15:34:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 -- start start 2024-05-14T15:34:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677aee10 0x55de677b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677b4d80 0x55de677b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c1b2f80 con 0x55de67630e40 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c1b0810 con 0x55de677b4d80 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c31961b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c1ae680 con 0x55de677aee10 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36940/0 (socket says 172.21.15.5:36940) 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 -- 172.21.15.5:0/314180338 learned_addr learned my addr 172.21.15.5:0/314180338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c214ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677b4d80 0x55de677b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c13fff700 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677aee10 0x55de677b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 -- 172.21.15.5:0/314180338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677aee10 msgr2=0x55de677b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677aee10 0x55de677b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 -- 172.21.15.5:0/314180338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677b4d80 msgr2=0x55de677b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.180+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677b4d80 0x55de677b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.181+0000 7f4c20ce9700 1 -- 172.21.15.5:0/314180338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de677a1890 con 0x55de67630e40 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.181+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677af220 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4c180e0ad0 tx=0x7f4c1800f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab7a7bcf3edf9c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.181+0000 7f4c027fc700 1 -- 172.21.15.5:0/314180338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c180c3020 con 0x55de67630e40 2024-05-14T15:34:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.181+0000 7f4c027fc700 1 -- 172.21.15.5:0/314180338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c180b2a30 con 0x55de67630e40 2024-05-14T15:34:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.181+0000 7f4c027fc700 1 -- 172.21.15.5:0/314180338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c180a8f10 con 0x55de67630e40 2024-05-14T15:34:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 -- 172.21.15.5:0/314180338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 msgr2=0x55de677af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677af220 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4c180e0ad0 tx=0x7f4c1800f5f0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 -- 172.21.15.5:0/314180338 shutdown_connections 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677b4d80 0x55de677b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677af220 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 --2- 172.21.15.5:0/314180338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677aee10 0x55de677b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 -- 172.21.15.5:0/314180338 >> 172.21.15.5:0/314180338 conn(0x55de67678630 msgr2=0x55de677b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 -- 172.21.15.5:0/314180338 shutdown_connections 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 -- 172.21.15.5:0/314180338 wait complete. 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.182+0000 7f4c31961b80 1 -- start start 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 Processor -- start 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677c09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c1aecd0 con 0x55de67630e40 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c1b0140 con 0x55de677aee10 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c1af3a0 con 0x55de677b4d80 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c31961b80 10 client.0 _ll_register_callbacks cb 0x55de67668950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677c09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c20ce9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677c09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36958/0 (socket says 172.21.15.5:36958) 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c13fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c20ce9700 1 -- 172.21.15.5:0/2025850038 learned_addr learned my addr 172.21.15.5:0/2025850038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.183+0000 7f4c214ea700 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c20ce9700 1 -- 172.21.15.5:0/2025850038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 msgr2=0x55de677cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c20ce9700 1 -- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 msgr2=0x55de677c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:34:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c20ce9700 1 -- 172.21.15.5:0/2025850038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de677cf940 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c20ce9700 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677c09b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4c180a74b0 tx=0x7f4c180a6920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fafdfa6e84ae67e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4bfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4bfbfff700 1 -- 172.21.15.5:0/2025850038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c180c3070 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c180b2a30 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4bfbfff700 1 -- 172.21.15.5:0/2025850038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7f4c180a94a0 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4bfbfff700 1 -- 172.21.15.5:0/2025850038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c180a96c0 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c31961b80 10 client.17838 fetch_fsmap learned FSMap version 32 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.184+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55de67832720 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4bfbfff700 1 -- 172.21.15.5:0/2025850038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c180b2a30 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4bfbfff700 1 -- 172.21.15.5:0/2025850038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4c180af0d0 con 0x55de67630e40 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c31961b80 10 client.17838 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c31961b80 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:34:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c31961b80 20 client.17838 populate_metadata read hostname 'smithi005' 2024-05-14T15:34:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c31961b80 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de678329f0 0x55de67834e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:34:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c31961b80 4 client.17838 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:34:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de676d83c0 con 0x55de678329f0 2024-05-14T15:34:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4c13fff700 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de678329f0 0x55de67834e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:34:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.185+0000 7f4bfbfff700 10 client.17838 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c13fff700 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de678329f0 0x55de67834e10 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeff423cb4b1e246 server_cookie=35a6f2d3f086f19 in_seq=0 out_seq=0 2024-05-14T15:34:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4bfbfff700 1 -- 172.21.15.5:0/2025850038 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de676d83c0 con 0x55de678329f0 2024-05-14T15:34:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4bfbfff700 10 client.17838 handle_command_reply: tid=0 2024-05-14T15:34:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 client.17838 shutdown 2024-05-14T15:34:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 msgr2=0x55de677c09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677c09b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4c180a74b0 tx=0x7f4c180a6920 comp rx=0 tx=0).stop 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 shutdown_connections 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55de678329f0 0x55de67834e10 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4bfbfff700 0 client.17838 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55de677b4d80 0x55de677cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de67630e40 0x55de677c09b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 --2- 172.21.15.5:0/2025850038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de677aee10 0x55de677c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 >> 172.21.15.5:0/2025850038 conn(0x55de67678630 msgr2=0x55de677a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 shutdown_connections 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.186+0000 7f4c31961b80 1 -- 172.21.15.5:0/2025850038 wait complete. 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c31961b80 20 client.17838 trim_cache size 0 max 16384 2024-05-14T15:34:59.192 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:34:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4c040b29f0 msgr2=0x7f4c040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4c040b29f0 0x7f4c040b4f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c1028c0 tx=0x7f4c0c009260 comp rx=0 tx=0).stop 2024-05-14T15:34:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 msgr2=0x7f4c1c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:34:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c19ae10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c18006010 tx=0x7f4c18005280 comp rx=0 tx=0).stop 2024-05-14T15:34:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 shutdown_connections 2024-05-14T15:34:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4c040b29f0 0x7f4c040b4f00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c1c108740 0x7f4c1c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c1c1034c0 0x7f4c1c19ae10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 --2- 172.21.15.5:0/3912428773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c107df0 0x7f4c1c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:34:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.187+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 >> 172.21.15.5:0/3912428773 conn(0x7f4c1c0fedf0 msgr2=0x7f4c1c1075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:34:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.188+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 shutdown_connections 2024-05-14T15:34:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:34:59.188+0000 7f4c22f4d700 1 -- 172.21.15.5:0/3912428773 wait complete. 2024-05-14T15:35:00.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 952856 9.68 MB/sec execute 381 sec latency 42.486 ms 2024-05-14T15:35:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:34:59 smithi144 ceph-mon[35650]: pgmap v318: 129 pgs: 129 active+clean; 99 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 335 op/s 2024-05-14T15:35:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:34:59 smithi193 ceph-mon[36001]: pgmap v318: 129 pgs: 129 active+clean; 99 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 335 op/s 2024-05-14T15:35:00.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:34:59 smithi005 ceph-mon[31559]: pgmap v318: 129 pgs: 129 active+clean; 99 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 335 op/s 2024-05-14T15:35:00.903 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:35:00.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:35:01.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 954461 9.67 MB/sec execute 382 sec latency 72.817 ms 2024-05-14T15:35:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:02.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 954493 9.65 MB/sec execute 383 sec latency 87.974 ms 2024-05-14T15:35:02.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:01 smithi144 ceph-mon[35650]: pgmap v319: 129 pgs: 129 active+clean; 99 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-14T15:35:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:01 smithi193 ceph-mon[36001]: pgmap v319: 129 pgs: 129 active+clean; 99 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-14T15:35:02.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:01 smithi005 ceph-mon[31559]: pgmap v319: 129 pgs: 129 active+clean; 99 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-14T15:35:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 -- 172.21.15.5:0/3176837018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a8107df0 msgr2=0x7fa9a8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 --2- 172.21.15.5:0/3176837018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a8107df0 0x7fa9a8108200 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa9900099c0 tx=0x7fa990009cd0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 -- 172.21.15.5:0/3176837018 shutdown_connections 2024-05-14T15:35:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 --2- 172.21.15.5:0/3176837018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 0x7fa9a81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 --2- 172.21.15.5:0/3176837018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9a81034c0 0x7fa9a81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 --2- 172.21.15.5:0/3176837018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a8107df0 0x7fa9a8108200 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 -- 172.21.15.5:0/3176837018 >> 172.21.15.5:0/3176837018 conn(0x7fa9a80fedc0 msgr2=0x7fa9a81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:03.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 -- 172.21.15.5:0/3176837018 shutdown_connections 2024-05-14T15:35:03.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.152+0000 7fa9ae623700 1 -- 172.21.15.5:0/3176837018 wait complete. 2024-05-14T15:35:03.155 INFO:tasks.workunit.client.0.smithi005.stdout: 1 956047 9.65 MB/sec execute 384 sec latency 15.255 ms 2024-05-14T15:35:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 Processor -- start 2024-05-14T15:35:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 -- start start 2024-05-14T15:35:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 0x7fa9a80ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9a8107df0 0x7fa9a80ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 0x7fa9a80ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8113350 con 0x7fa9a81034c0 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81131d0 con 0x7fa9a8108740 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9ae623700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81134d0 con 0x7fa9a8107df0 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 0x7fa9a80ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 0x7fa9a80ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36978/0 (socket says 172.21.15.5:36978) 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2224337644 learned_addr learned my addr 172.21.15.5:0/2224337644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 0x7fa9a80ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.153+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9a8107df0 0x7fa9a80ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2224337644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9a8107df0 msgr2=0x7fa9a80ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9a8107df0 0x7fa9a80ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2224337644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 msgr2=0x7fa9a80ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 0x7fa9a80ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2224337644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a81a9200 con 0x7fa9a81034c0 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 0x7fa9a80ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 0x7fa9a80ff2a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa99800d980 tx=0x7fa99800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a5ffb700 1 -- 172.21.15.5:0/2224337644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998016070 con 0x7fa9a81034c0 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a5ffb700 1 -- 172.21.15.5:0/2224337644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa998004830 con 0x7fa9a81034c0 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a81a94f0 con 0x7fa9a81034c0 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9a5ffb700 1 -- 172.21.15.5:0/2224337644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998005540 con 0x7fa9a81034c0 2024-05-14T15:35:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.154+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a81a9a00 con 0x7fa9a81034c0 2024-05-14T15:35:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.157+0000 7fa9a5ffb700 1 -- 172.21.15.5:0/2224337644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa998004220 con 0x7fa9a81034c0 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.158+0000 7fa9a5ffb700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9940b29f0 0x7fa9940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.158+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9940b29f0 0x7fa9940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.158+0000 7fa9bd037b80 1 Processor -- start 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.158+0000 7fa9a5ffb700 1 -- 172.21.15.5:0/2224337644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa9980e27e0 con 0x7fa9a81034c0 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.158+0000 7fa9bd037b80 1 -- start start 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb2d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb27950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb27e90 0x55aceeb282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9940b29f0 0x7fa9940b4f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa990000c00 tx=0x7fa99001a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb2d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aceea499c0 con 0x55acee9a9ec0 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa99ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb27950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb2d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36986/0 (socket says 172.21.15.5:36986) 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2527921166 learned_addr learned my addr 172.21.15.5:0/2527921166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55acee9aee20 con 0x55aceeb27e90 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aceea45980 con 0x55aceeb2de00 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb27e90 0x55aceeb282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2527921166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 msgr2=0x55aceeb27950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb27950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2527921166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb27e90 msgr2=0x55aceeb282a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb27e90 0x55aceeb282a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2527921166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acee996950 con 0x55acee9a9ec0 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.159+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb27e90 0x55aceeb282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.160+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb2d8c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa9980a5740 tx=0x7fa9980a8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842a1cd740f0ae42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.160+0000 7fa99e7fc700 1 -- 172.21.15.5:0/2527921166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980ada70 con 0x55acee9a9ec0 2024-05-14T15:35:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.160+0000 7fa99e7fc700 1 -- 172.21.15.5:0/2527921166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9980b2520 con 0x55acee9a9ec0 2024-05-14T15:35:03.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.160+0000 7fa99e7fc700 1 -- 172.21.15.5:0/2527921166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99808fbe0 con 0x55acee9a9ec0 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 msgr2=0x55aceeb2d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb2d8c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa9980a5740 tx=0x7fa9980a8f60 comp rx=0 tx=0).stop 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 shutdown_connections 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb27e90 0x55aceeb282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb2d8c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2527921166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 >> 172.21.15.5:0/2527921166 conn(0x55acee9f16b0 msgr2=0x55aceeb22a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 shutdown_connections 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2527921166 wait complete. 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.161+0000 7fa9bd037b80 1 -- start start 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 Processor -- start 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb31ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb32010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aceeb34840 con 0x55acee9a9ec0 2024-05-14T15:35:03.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aceea02f50 con 0x55aceeb2de00 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aceeb349c0 con 0x55aceeb27e90 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9bd037b80 10 client.0 _ll_register_callbacks cb 0x55acee9e19d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa99ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb31ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb31590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36998/0 (socket says 172.21.15.5:36998) 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2210595886 learned_addr learned my addr 172.21.15.5:0/2210595886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb32010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2210595886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 msgr2=0x55aceeb31ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb31ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 msgr2=0x55aceeb32010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb32010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.162+0000 7fa9a7fff700 1 -- 172.21.15.5:0/2210595886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceea007e0 con 0x55acee9a9ec0 2024-05-14T15:35:03.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb31590 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa9980046a0 tx=0x7fa9980a8e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d87e5e0f3ebec2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9877fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9877fe700 1 -- 172.21.15.5:0/2210595886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980a5950 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9877fe700 1 -- 172.21.15.5:0/2210595886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa9980910d0 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9980b2520 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9877fe700 1 -- 172.21.15.5:0/2210595886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9980a5af0 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 10 client.17850 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aceeb3f490 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9877fe700 1 -- 172.21.15.5:0/2210595886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9980b2520 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9877fe700 1 -- 172.21.15.5:0/2210595886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa9980e2040 con 0x55acee9a9ec0 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 10 client.17850 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:03.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 20 client.17850 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.163+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceeb41960 0x55aceeb43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.164+0000 7fa9bd037b80 4 client.17850 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.164+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aceea573a0 con 0x55aceeb41960 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.164+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceeb41960 0x55aceeb43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.164+0000 7fa9877fe700 10 client.17850 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.164+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceeb41960 0x55aceeb43d80 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67886c493cda5a8b server_cookie=e669f2bed1db650c in_seq=0 out_seq=0 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.165+0000 7fa9877fe700 1 -- 172.21.15.5:0/2210595886 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aceea573a0 con 0x55aceeb41960 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.165+0000 7fa9877fe700 10 client.17850 handle_command_reply: tid=0 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.165+0000 7fa9bd037b80 1 client.17850 shutdown 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 msgr2=0x55aceeb31590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb31590 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa9980046a0 tx=0x7fa9980a8e30 comp rx=0 tx=0).stop 2024-05-14T15:35:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 shutdown_connections 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceeb41960 0x55aceeb43d80 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb32010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb31590 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2210595886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb31ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 >> 172.21.15.5:0/2210595886 conn(0x55acee9f16b0 msgr2=0x55aceeb26940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9877fe700 0 client.17850 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 shutdown_connections 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2210595886 wait complete. 2024-05-14T15:35:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.166+0000 7fa9bd037b80 20 client.17850 trim_cache size 0 max 16384 2024-05-14T15:35:03.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 Processor -- start 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 -- start start 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acee9a9ec0 0x55aceeb27950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 0x55aceeb308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb3dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b5220 con 0x55aceeb27e90 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b2ab0 con 0x55acee9a9ec0 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b0920 con 0x55aceeb2de00 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa99ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 0x55aceeb308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa99ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 0x55aceeb308b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37008/0 (socket says 172.21.15.5:37008) 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa99ffff700 1 -- 172.21.15.5:0/2374194431 learned_addr learned my addr 172.21.15.5:0/2374194431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.188+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acee9a9ec0 0x55aceeb27950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb3dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99ffff700 1 -- 172.21.15.5:0/2374194431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 msgr2=0x55aceeb3dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb3dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99ffff700 1 -- 172.21.15.5:0/2374194431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acee9a9ec0 msgr2=0x55aceeb27950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acee9a9ec0 0x55aceeb27950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99ffff700 1 -- 172.21.15.5:0/2374194431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aceeb23550 con 0x55aceeb27e90 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acee9a9ec0 0x55aceeb27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99ffff700 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 0x55aceeb308b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8076ab0 tx=0x7fa990009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713d490498162533 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99dffb700 1 -- 172.21.15.5:0/2374194431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990003da0 con 0x55aceeb27e90 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99dffb700 1 -- 172.21.15.5:0/2374194431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99000bc40 con 0x55aceeb27e90 2024-05-14T15:35:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.189+0000 7fa99dffb700 1 -- 172.21.15.5:0/2374194431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9900337d0 con 0x55aceeb27e90 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2374194431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 msgr2=0x55aceeb308b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 0x55aceeb308b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8076ab0 tx=0x7fa990009cd0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2374194431 shutdown_connections 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb2de00 0x55aceeb3dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acee9a9ec0 0x55aceeb27950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/2374194431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aceeb27e90 0x55aceeb308b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2374194431 >> 172.21.15.5:0/2374194431 conn(0x55acee9f16b0 msgr2=0x55aceeb1cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2374194431 shutdown_connections 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.190+0000 7fa9bd037b80 1 -- 172.21.15.5:0/2374194431 wait complete. 2024-05-14T15:35:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 -- start start 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 Processor -- start 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb46be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb43fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b0f70 con 0x55acee9a9ec0 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b23e0 con 0x55aceeb2de00 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b1640 con 0x55aceeb27e90 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9bd037b80 10 client.0 _ll_register_callbacks cb 0x55acee9e19d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb46be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9a7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb46be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37016/0 (socket says 172.21.15.5:37016) 2024-05-14T15:35:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9a7fff700 1 -- 172.21.15.5:0/1622541251 learned_addr learned my addr 172.21.15.5:0/1622541251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.191+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa99ffff700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb43fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9a7fff700 1 -- 172.21.15.5:0/1622541251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 msgr2=0x55aceeb43fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb43fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9a7fff700 1 -- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 msgr2=0x55aceeb444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9a7fff700 1 -- 172.21.15.5:0/1622541251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceeb44a30 con 0x55acee9a9ec0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9acbc0700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9a7fff700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb46be0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa99800d950 tx=0x7fa998007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cfc9192bfc3ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9877fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9877fe700 1 -- 172.21.15.5:0/1622541251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980a9920 con 0x55acee9a9ec0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9877fe700 1 -- 172.21.15.5:0/1622541251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9546+0+0 (secure 0 0 0) 0x7fa99808fbe0 con 0x55acee9a9ec0 2024-05-14T15:35:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99000bc40 con 0x55acee9a9ec0 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9877fe700 1 -- 172.21.15.5:0/1622541251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa9980b7050 con 0x55acee9a9ec0 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9bd037b80 10 client.17856 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.192+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aceebabad0 con 0x55acee9a9ec0 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9877fe700 1 -- 172.21.15.5:0/1622541251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99000bc40 con 0x55acee9a9ec0 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9877fe700 1 -- 172.21.15.5:0/1622541251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fa9980e2040 con 0x55acee9a9ec0 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9bd037b80 10 client.17856 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9bd037b80 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9bd037b80 20 client.17856 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceebabf40 0x55aceebae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9bd037b80 4 client.17856 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aceea51440 con 0x55aceebabf40 2024-05-14T15:35:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.193+0000 7fa99ffff700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceebabf40 0x55aceebae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.194+0000 7fa9877fe700 10 client.17856 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:03.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.194+0000 7fa99ffff700 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceebabf40 0x55aceebae360 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb067bd8e142a4c2 server_cookie=4eb88604adec6225 in_seq=0 out_seq=0 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.195+0000 7fa9877fe700 1 -- 172.21.15.5:0/1622541251 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aceea51440 con 0x55aceebabf40 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.195+0000 7fa9877fe700 10 client.17856 handle_command_reply: tid=0 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.195+0000 7fa9bd037b80 1 client.17856 shutdown 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.195+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 msgr2=0x55aceeb46be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.195+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb46be0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa99800d950 tx=0x7fa998007640 comp rx=0 tx=0).stop 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 shutdown_connections 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55aceebabf40 0x55aceebae360 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aceeb2de00 0x55aceeb444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acee9a9ec0 0x55aceeb46be0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9877fe700 0 client.17856 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 --2- 172.21.15.5:0/1622541251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceeb27e90 0x55aceeb43fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 >> 172.21.15.5:0/1622541251 conn(0x55acee9f16b0 msgr2=0x55aceeb1cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 shutdown_connections 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 1 -- 172.21.15.5:0/1622541251 wait complete. 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9bd037b80 20 client.17856 trim_cache size 0 max 16384 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "87ad84ae-bd49-4333-a11b-86793d4c5752", 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:35:03.202 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9940b29f0 msgr2=0x7fa9940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9ae623700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9940b29f0 0x7fa9940b4f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa990000c00 tx=0x7fa99001a040 comp rx=0 tx=0).stop 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 msgr2=0x7fa9a80ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.196+0000 7fa9ae623700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 0x7fa9a80ff2a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa99800d980 tx=0x7fa99800dc90 comp rx=0 tx=0).stop 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 shutdown_connections 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa9940b29f0 0x7fa9940b4f00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9a8108740 0x7fa9a80ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9a81034c0 0x7fa9a80ff2a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 --2- 172.21.15.5:0/2224337644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9a8107df0 0x7fa9a80ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 >> 172.21.15.5:0/2224337644 conn(0x7fa9a80fedc0 msgr2=0x7fa9a806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 shutdown_connections 2024-05-14T15:35:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:03.197+0000 7fa9ae623700 1 -- 172.21.15.5:0/2224337644 wait complete. 2024-05-14T15:35:03.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:35:04.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 957523 9.64 MB/sec execute 385 sec latency 19.608 ms 2024-05-14T15:35:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:03 smithi144 ceph-mon[35650]: pgmap v320: 129 pgs: 129 active+clean; 88 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.0 MiB/s wr, 314 op/s 2024-05-14T15:35:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:03 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:35:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:03 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:03 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:35:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:03 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:03 smithi193 ceph-mon[36001]: pgmap v320: 129 pgs: 129 active+clean; 88 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.0 MiB/s wr, 314 op/s 2024-05-14T15:35:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:03 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:35:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:03 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:03 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:35:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:03 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:04.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:03 smithi005 ceph-mon[31559]: pgmap v320: 129 pgs: 129 active+clean; 88 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.0 MiB/s wr, 314 op/s 2024-05-14T15:35:04.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:03 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:35:04.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:03 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:04.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:03 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:35:04.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:03 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:05.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 959497 9.65 MB/sec execute 386 sec latency 27.383 ms 2024-05-14T15:35:06.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5b67fc700 1 -- 172.21.15.5:0/2366353518 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8017400 con 0x7ff5b8103ba0 2024-05-14T15:35:06.044 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:05 smithi005 ceph-mon[31559]: pgmap v321: 129 pgs: 129 active+clean; 88 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 253 op/s 2024-05-14T15:35:06.044 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:05 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:35:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2366353518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103ba0 msgr2=0x7ff5b81080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2366353518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103ba0 0x7ff5b81080d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80098f0 tx=0x7ff5a8009c00 comp rx=0 tx=0).stop 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2366353518 shutdown_connections 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2366353518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b81086a0 0x7ff5b8108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2366353518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b8103250 0x7ff5b8103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2366353518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103ba0 0x7ff5b81080d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.042+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2366353518 >> 172.21.15.5:0/2366353518 conn(0x7ff5b80fedc0 msgr2=0x7ff5b81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.043+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2366353518 shutdown_connections 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.043+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2366353518 wait complete. 2024-05-14T15:35:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 Processor -- start 2024-05-14T15:35:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 -- start start 2024-05-14T15:35:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 0x7ff5b8075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b8103ba0 0x7ff5b806e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b81086a0 0x7ff5b806ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b811aac0 con 0x7ff5b8103ba0 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b811a940 con 0x7ff5b81086a0 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.044+0000 7ff5bddbd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b811ac40 con 0x7ff5b8103250 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 0x7ff5b8075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 0x7ff5b8075420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50756/0 (socket says 172.21.15.5:50756) 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b77fe700 1 -- 172.21.15.5:0/2279898633 learned_addr learned my addr 172.21.15.5:0/2279898633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b8103ba0 0x7ff5b806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/2279898633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 msgr2=0x7ff5b8075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 0x7ff5b8075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/2279898633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b81086a0 msgr2=0x7ff5b806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b81086a0 0x7ff5b806ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/2279898633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b8072f50 con 0x7ff5b8103ba0 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.045+0000 7ff5b77fe700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 0x7ff5b8075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.046+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b8103ba0 0x7ff5b806e550 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009230 tx=0x7ff5a8009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.049+0000 7ff5b4ff9700 1 -- 172.21.15.5:0/2279898633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80051f0 con 0x7ff5b8103ba0 2024-05-14T15:35:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.049+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b80731e0 con 0x7ff5b8103ba0 2024-05-14T15:35:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.049+0000 7ff5b4ff9700 1 -- 172.21.15.5:0/2279898633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a800b560 con 0x7ff5b8103ba0 2024-05-14T15:35:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.049+0000 7ff5b4ff9700 1 -- 172.21.15.5:0/2279898633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8003ac0 con 0x7ff5b8103ba0 2024-05-14T15:35:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.050+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b8073670 con 0x7ff5b8103ba0 2024-05-14T15:35:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.057+0000 7ff5cc7d1b80 1 Processor -- start 2024-05-14T15:35:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 -- start start 2024-05-14T15:35:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650bb3ea860 con 0x5650bb4cd470 2024-05-14T15:35:06.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650bb3897c0 con 0x5650bb4c8c60 2024-05-14T15:35:06.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.058+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650bb3e6820 con 0x5650bb34ae40 2024-05-14T15:35:06.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.060+0000 7ff5b7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.060+0000 7ff5b7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50772/0 (socket says 172.21.15.5:50772) 2024-05-14T15:35:06.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.060+0000 7ff5b7fff700 1 -- 172.21.15.5:0/897628106 learned_addr learned my addr 172.21.15.5:0/897628106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:06.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.060+0000 7ff5b4ff9700 1 -- 172.21.15.5:0/2279898633 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff5a800ba10 con 0x7ff5b8103ba0 2024-05-14T15:35:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.060+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.061+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/897628106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 msgr2=0x5650bb4c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.061+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.061+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/897628106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 msgr2=0x5650bb4ccf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.061+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4ccf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.061+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/897628106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bb3378d0 con 0x5650bb4cd470 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.061+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4cd880 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80e0c30 tx=0x7ff5a80e9a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b067b34f8ebe459e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.062+0000 7ff5b4ff9700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5a00b2c00 0x7ff5a00b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.063+0000 7ff5b4ff9700 1 -- 172.21.15.5:0/2279898633 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7ff5a80e39b0 con 0x7ff5b8103ba0 2024-05-14T15:35:06.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.064+0000 7ff59dffb700 1 -- 172.21.15.5:0/897628106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80ee630 con 0x5650bb4cd470 2024-05-14T15:35:06.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.064+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5a00b2c00 0x7ff5a00b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.064+0000 7ff59dffb700 1 -- 172.21.15.5:0/897628106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a80f3020 con 0x5650bb4cd470 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/897628106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 msgr2=0x5650bb4cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4cd880 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80e0c30 tx=0x7ff5a80e9a60 comp rx=0 tx=0).stop 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5a00b2c00 0x7ff5a00b5110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0095a0 tx=0x7ff5ac009510 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/897628106 shutdown_connections 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4cd880 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/897628106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.065+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/897628106 >> 172.21.15.5:0/897628106 conn(0x5650bb392600 msgr2=0x5650bb4c34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:06.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.068+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/897628106 shutdown_connections 2024-05-14T15:35:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.070+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/897628106 wait complete. 2024-05-14T15:35:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.070+0000 7ff5cc7d1b80 1 -- start start 2024-05-14T15:35:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.070+0000 7ff5cc7d1b80 1 Processor -- start 2024-05-14T15:35:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 0x5650bb4d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650bb4d51e0 con 0x5650bb4cd470 2024-05-14T15:35:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650bb350c30 con 0x5650bb34ae40 2024-05-14T15:35:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650bb4d5360 con 0x5650bb4c8c60 2024-05-14T15:35:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.073+0000 7ff5cc7d1b80 10 client.0 _ll_register_callbacks cb 0x5650bb382920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.075+0000 7ff5b6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 0x5650bb4d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.075+0000 7ff5b6ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 0x5650bb4d2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50782/0 (socket says 172.21.15.5:50782) 2024-05-14T15:35:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.075+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/1528612171 learned_addr learned my addr 172.21.15.5:0/1528612171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.076+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.076+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 msgr2=0x5650bb4d4c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.076+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d4c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.076+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/1528612171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 msgr2=0x5650bb4d26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.076+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.076+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/1528612171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bb386690 con 0x5650bb4c8c60 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 0x5650bb4d2010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80aa7e0 tx=0x7ff5a80e0c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4e55d44491f235 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff596ffd700 1 -- 172.21.15.5:0/1528612171 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80f2030 con 0x5650bb4c8c60 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a80f3020 con 0x5650bb4c8c60 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff596ffd700 1 -- 172.21.15.5:0/1528612171 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7ff5a80a6e40 con 0x5650bb4c8c60 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff596ffd700 1 -- 172.21.15.5:0/1528612171 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff5a80a6710 con 0x5650bb4c8c60 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.078+0000 7ff5cc7d1b80 10 client.27457 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.079+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650bb4d51e0 con 0x5650bb4c8c60 2024-05-14T15:35:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.082+0000 7ff596ffd700 1 -- 172.21.15.5:0/1528612171 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a80f3020 con 0x5650bb4c8c60 2024-05-14T15:35:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff596ffd700 1 -- 172.21.15.5:0/1528612171 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff5a80e0eb0 con 0x5650bb4c8c60 2024-05-14T15:35:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff5cc7d1b80 10 client.27457 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff5cc7d1b80 10 client.27457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff5cc7d1b80 20 client.27457 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb4e1ff0 0x5650bb4e4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff5cc7d1b80 4 client.27457 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.083+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650bb3f8240 con 0x5650bb4e1ff0 2024-05-14T15:35:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.085+0000 7ff5b77fe700 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb4e1ff0 0x5650bb4e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.091+0000 7ff5b77fe700 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb4e1ff0 0x5650bb4e4440 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1a321c9338c6d1 server_cookie=ae96d41fa6282fe1 in_seq=0 out_seq=0 2024-05-14T15:35:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.091+0000 7ff596ffd700 10 client.27457 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.093+0000 7ff596ffd700 1 -- 172.21.15.5:0/1528612171 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650bb3f8240 con 0x5650bb4e1ff0 2024-05-14T15:35:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.093+0000 7ff596ffd700 10 client.27457 handle_command_reply: tid=0 2024-05-14T15:35:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.093+0000 7ff5cc7d1b80 1 client.27457 shutdown 2024-05-14T15:35:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.097+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 msgr2=0x5650bb4d2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.097+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 0x5650bb4d2010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80aa7e0 tx=0x7ff5a80e0c90 comp rx=0 tx=0).stop 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.100+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 shutdown_connections 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.100+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb4e1ff0 0x5650bb4e4440 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.100+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.100+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.100+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/1528612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4c8c60 0x5650bb4d2010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.100+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 >> 172.21.15.5:0/1528612171 conn(0x5650bb392600 msgr2=0x5650bb4c7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.101+0000 7ff596ffd700 0 client.27457 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.103+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 shutdown_connections 2024-05-14T15:35:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.103+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/1528612171 wait complete. 2024-05-14T15:35:06.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.103+0000 7ff5cc7d1b80 20 client.27457 trim_cache size 0 max 16384 2024-05-14T15:35:06.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.145+0000 7ff5cc7d1b80 1 Processor -- start 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.145+0000 7ff5cc7d1b80 1 -- start start 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.146+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.146+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4c8c60 0x5650bb4de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.146+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 0x5650bb4e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.146+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81ca720 con 0x5650bb4c8c60 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.146+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81c7fe0 con 0x5650bb34ae40 2024-05-14T15:35:06.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.146+0000 7ff5cc7d1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b806bef0 con 0x5650bb4cd470 2024-05-14T15:35:06.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.148+0000 7ff5b77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.148+0000 7ff5b7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 0x5650bb4e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.148+0000 7ff5b6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4c8c60 0x5650bb4de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.148+0000 7ff5b7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 0x5650bb4e9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50788/0 (socket says 172.21.15.5:50788) 2024-05-14T15:35:06.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.148+0000 7ff5b7fff700 1 -- 172.21.15.5:0/861634268 learned_addr learned my addr 172.21.15.5:0/861634268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.149+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/861634268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 msgr2=0x5650bb4e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.149+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 0x5650bb4e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.149+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/861634268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 msgr2=0x5650bb4d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.149+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.149+0000 7ff5b6ffd700 1 -- 172.21.15.5:0/861634268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bb4cbfb0 con 0x5650bb4c8c60 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.149+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 0x5650bb4e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:35:06.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.150+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4c8c60 0x5650bb4de280 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000c00 tx=0x7ff5a800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4802567004a8d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:06.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.150+0000 7ff59d7fa700 1 -- 172.21.15.5:0/861634268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80f2020 con 0x5650bb4c8c60 2024-05-14T15:35:06.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.150+0000 7ff59d7fa700 1 -- 172.21.15.5:0/861634268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a80cd110 con 0x5650bb4c8c60 2024-05-14T15:35:06.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.150+0000 7ff59d7fa700 1 -- 172.21.15.5:0/861634268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80a9fc0 con 0x5650bb4c8c60 2024-05-14T15:35:06.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.152+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/861634268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4c8c60 msgr2=0x5650bb4de280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.152+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4c8c60 0x5650bb4de280 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000c00 tx=0x7ff5a800bd00 comp rx=0 tx=0).stop 2024-05-14T15:35:06.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 960695 9.64 MB/sec execute 387 sec latency 30.334 ms 2024-05-14T15:35:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.154+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/861634268 shutdown_connections 2024-05-14T15:35:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.154+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb4cd470 0x5650bb4e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.154+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb34ae40 0x5650bb4d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.155+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/861634268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4c8c60 0x5650bb4de280 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.155+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/861634268 >> 172.21.15.5:0/861634268 conn(0x5650bb392600 msgr2=0x5650bb4d18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.157+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/861634268 shutdown_connections 2024-05-14T15:35:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.159+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/861634268 wait complete. 2024-05-14T15:35:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.159+0000 7ff5cc7d1b80 1 -- start start 2024-05-14T15:35:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.160+0000 7ff5cc7d1b80 1 Processor -- start 2024-05-14T15:35:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.160+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4da7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.161+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4dad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.161+0000 7ff5cc7d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.161+0000 7ff5b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4da7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.161+0000 7ff5b77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4da7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:50802/0 (socket says 172.21.15.5:50802) 2024-05-14T15:35:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.161+0000 7ff5b77fe700 1 -- 172.21.15.5:0/350631079 learned_addr learned my addr 172.21.15.5:0/350631079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.161+0000 7ff5b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.162+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b806fa80 con 0x5650bb4cd470 2024-05-14T15:35:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.162+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8112950 con 0x5650bb4c8c60 2024-05-14T15:35:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.162+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.162+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8111bb0 con 0x5650bb34ae40 2024-05-14T15:35:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.162+0000 7ff5cc7d1b80 10 client.0 _ll_register_callbacks cb 0x5650bb382920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.162+0000 7ff5b7fff700 1 -- 172.21.15.5:0/350631079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 msgr2=0x5650bb4da7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4da7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff5b7fff700 1 -- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 msgr2=0x5650bb4dad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4dad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff5b7fff700 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bb4d84d0 con 0x5650bb4cd470 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff5b7fff700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d7d30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005fd0 tx=0x7ff5ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825ddae2b6257e3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff5b6ffd700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4dad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.163+0000 7ff596ffd700 1 -- 172.21.15.5:0/350631079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac01f8f0 con 0x5650bb4cd470 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.164+0000 7ff596ffd700 1 -- 172.21.15.5:0/350631079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7ff5ac01fa90 con 0x5650bb4cd470 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.164+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a80cd110 con 0x5650bb4cd470 2024-05-14T15:35:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.165+0000 7ff596ffd700 1 -- 172.21.15.5:0/350631079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff5ac01fc90 con 0x5650bb4cd470 2024-05-14T15:35:06.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.167+0000 7ff5cc7d1b80 10 client.17874 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:06.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.167+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650bb54c370 con 0x5650bb4cd470 2024-05-14T15:35:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.168+0000 7ff596ffd700 1 -- 172.21.15.5:0/350631079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a80cd110 con 0x5650bb4cd470 2024-05-14T15:35:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.168+0000 7ff596ffd700 1 -- 172.21.15.5:0/350631079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7ff5ac027020 con 0x5650bb4cd470 2024-05-14T15:35:06.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.169+0000 7ff5cc7d1b80 10 client.17874 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:06.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.169+0000 7ff5cc7d1b80 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.169+0000 7ff5cc7d1b80 20 client.17874 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.169+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb54c9e0 0x5650bb54ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.169+0000 7ff5cc7d1b80 4 client.17874 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:35:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.169+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650bb3f22e0 con 0x5650bb54c9e0 2024-05-14T15:35:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.171+0000 7ff5b77fe700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb54c9e0 0x5650bb54ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.174+0000 7ff5b77fe700 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb54c9e0 0x5650bb54ee00 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39834693cd10e113 server_cookie=cdadd29267033950 in_seq=0 out_seq=0 2024-05-14T15:35:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.174+0000 7ff596ffd700 10 client.17874 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff596ffd700 1 -- 172.21.15.5:0/350631079 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5650bb3f22e0 con 0x5650bb54c9e0 2024-05-14T15:35:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff596ffd700 10 client.17874 handle_command_reply: tid=0 2024-05-14T15:35:06.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 client.17874 shutdown 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 msgr2=0x5650bb4d7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d7d30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac005fd0 tx=0x7ff5ac004750 comp rx=0 tx=0).stop 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 shutdown_connections 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5650bb54c9e0 0x5650bb54ee00 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650bb4cd470 0x5650bb4d7d30 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650bb34ae40 0x5650bb4da7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 --2- 172.21.15.5:0/350631079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650bb4c8c60 0x5650bb4dad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 >> 172.21.15.5:0/350631079 conn(0x5650bb392600 msgr2=0x5650bb4cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.177+0000 7ff596ffd700 0 client.17874 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.180+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 shutdown_connections 2024-05-14T15:35:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.180+0000 7ff5cc7d1b80 1 -- 172.21.15.5:0/350631079 wait complete. 2024-05-14T15:35:06.183 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:35:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:35:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:35:06.183 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.181+0000 7ff5cc7d1b80 20 client.17874 trim_cache size 0 max 16384 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5a00b2c00 msgr2=0x7ff5a00b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5a00b2c00 0x7ff5a00b5110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0095a0 tx=0x7ff5ac009510 comp rx=0 tx=0).stop 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b8103ba0 msgr2=0x7ff5b806e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b8103ba0 0x7ff5b806e550 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009230 tx=0x7ff5a8009480 comp rx=0 tx=0).stop 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 shutdown_connections 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff5a00b2c00 0x7ff5a00b5110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b81086a0 0x7ff5b806ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5b8103250 0x7ff5b8075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 --2- 172.21.15.5:0/2279898633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b8103ba0 0x7ff5b806e550 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 >> 172.21.15.5:0/2279898633 conn(0x7ff5b80fedc0 msgr2=0x7ff5b8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:06.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.182+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 shutdown_connections 2024-05-14T15:35:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:06.183+0000 7ff5bddbd700 1 -- 172.21.15.5:0/2279898633 wait complete. 2024-05-14T15:35:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:05 smithi144 ceph-mon[35650]: pgmap v321: 129 pgs: 129 active+clean; 88 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 253 op/s 2024-05-14T15:35:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:05 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:35:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:05 smithi193 ceph-mon[36001]: pgmap v321: 129 pgs: 129 active+clean; 88 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 253 op/s 2024-05-14T15:35:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:05 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:35:07.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 961177 9.62 MB/sec execute 388 sec latency 45.164 ms 2024-05-14T15:35:07.207 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:06 smithi005 ceph-mon[31559]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T15:35:07.259 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:35:07.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:35:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:06 smithi144 ceph-mon[35650]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T15:35:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:06 smithi193 ceph-mon[36001]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T15:35:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:08.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 962063 9.61 MB/sec execute 389 sec latency 78.641 ms 2024-05-14T15:35:08.188 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:07 smithi005 ceph-mon[31559]: pgmap v323: 129 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 99 active+clean; 94 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-05-14T15:35:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:07 smithi144 ceph-mon[35650]: pgmap v323: 129 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 99 active+clean; 94 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-05-14T15:35:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:07 smithi193 ceph-mon[36001]: pgmap v323: 129 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 99 active+clean; 94 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-05-14T15:35:09.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 962625 9.59 MB/sec execute 390 sec latency 257.108 ms 2024-05-14T15:35:10.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:09 smithi005 ceph-mon[31559]: pgmap v324: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 95 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-05-14T15:35:10.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 962920 9.57 MB/sec execute 391 sec latency 113.654 ms 2024-05-14T15:35:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:09 smithi144 ceph-mon[35650]: pgmap v324: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 95 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-05-14T15:35:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:09 smithi193 ceph-mon[36001]: pgmap v324: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 95 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-05-14T15:35:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.300+0000 7fb72c53f640 1 -- 172.21.15.5:0/2584046089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410f950 msgr2=0x7fb7241170f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.300+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2584046089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410f950 0x7fb7241170f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb714009370 tx=0x7fb71402b650 comp rx=0 tx=0).stop 2024-05-14T15:35:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.305+0000 7fb72c53f640 1 -- 172.21.15.5:0/2584046089 shutdown_connections 2024-05-14T15:35:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.305+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2584046089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410f950 0x7fb7241170f0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.305+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2584046089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410ee90 0x7fb72410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.305+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2584046089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb72410a900 0x7fb72410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.305+0000 7fb72c53f640 1 -- 172.21.15.5:0/2584046089 >> 172.21.15.5:0/2584046089 conn(0x7fb72406edb0 msgr2=0x7fb72406f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.308+0000 7fb72c53f640 1 -- 172.21.15.5:0/2584046089 shutdown_connections 2024-05-14T15:35:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.311+0000 7fb72c53f640 1 -- 172.21.15.5:0/2584046089 wait complete. 2024-05-14T15:35:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.312+0000 7fb72c53f640 1 Processor -- start 2024-05-14T15:35:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.312+0000 7fb72c53f640 1 -- start start 2024-05-14T15:35:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.312+0000 7fb72c53f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb72410a900 0x7fb72411ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72c53f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410ee90 0x7fb72411b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72c53f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 0x7fb72411b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72c53f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb724119140 con 0x7fb72410ee90 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72c53f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb724118fc0 con 0x7fb72410f950 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72c53f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241192c0 con 0x7fb72410a900 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72aadc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 0x7fb72411b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72aadc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 0x7fb72411b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57854/0 (socket says 172.21.15.5:57854) 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.313+0000 7fb72aadc640 1 -- 172.21.15.5:0/2257394885 learned_addr learned my addr 172.21.15.5:0/2257394885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.314+0000 7fb729ada640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410ee90 0x7fb72411b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.315+0000 7fb72aadc640 1 -- 172.21.15.5:0/2257394885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb72410a900 msgr2=0x7fb72411ac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T15:35:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.315+0000 7fb72aadc640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb72410a900 0x7fb72411ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.315+0000 7fb72aadc640 1 -- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410ee90 msgr2=0x7fb72411b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.315+0000 7fb72aadc640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410ee90 0x7fb72411b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.315+0000 7fb72aadc640 1 -- 172.21.15.5:0/2257394885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7241be0f0 con 0x7fb72410f950 2024-05-14T15:35:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.319+0000 7fb72aadc640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 0x7fb72411b710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb714006010 tx=0x7fb7140315d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.320+0000 7fb71b7fe640 1 -- 172.21.15.5:0/2257394885 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714040070 con 0x7fb72410f950 2024-05-14T15:35:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.320+0000 7fb71b7fe640 1 -- 172.21.15.5:0/2257394885 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71403c460 con 0x7fb72410f950 2024-05-14T15:35:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.320+0000 7fb71b7fe640 1 -- 172.21.15.5:0/2257394885 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714040070 con 0x7fb72410f950 2024-05-14T15:35:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.321+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7241be320 con 0x7fb72410f950 2024-05-14T15:35:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.321+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241be790 con 0x7fb72410f950 2024-05-14T15:35:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.337+0000 7fb71b7fe640 1 -- 172.21.15.5:0/2257394885 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb714043070 con 0x7fb72410f950 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.343+0000 7fb7333f8740 1 Processor -- start 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 -- start start 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 0x5609dc8b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc4f34d0 0x5609dc8b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609dc81a5d0 con 0x5609dc5141b0 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb729ada640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 0x5609dc8b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb729ada640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 0x5609dc8b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56960/0 (socket says 172.21.15.5:56960) 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb729ada640 1 -- 172.21.15.5:0/1720704501 learned_addr learned my addr 172.21.15.5:0/1720704501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609dc7d3890 con 0x5609dc4eb2f0 2024-05-14T15:35:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.344+0000 7fb7333f8740 1 -- 172.21.15.5:0/1720704501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5609dc816590 con 0x5609dc4f34d0 2024-05-14T15:35:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.345+0000 7fb71b7fe640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb7100b2e20 0x7fb7100b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.346+0000 7fb71b7fe640 1 -- 172.21.15.5:0/2257394885 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fb714031900 con 0x7fb72410f950 2024-05-14T15:35:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.348+0000 7fb729ada640 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc4f34d0 0x5609dc8b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.348+0000 7fb729ada640 1 -- 172.21.15.5:0/1720704501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc4f34d0 msgr2=0x5609dc8b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.348+0000 7fb729ada640 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc4f34d0 0x5609dc8b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.348+0000 7fb729ada640 1 -- 172.21.15.5:0/1720704501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 msgr2=0x5609dc8b5120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:35:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.348+0000 7fb729ada640 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8b5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.348+0000 7fb729ada640 1 -- 172.21.15.5:0/1720704501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dc52c210 con 0x5609dc5141b0 2024-05-14T15:35:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.351+0000 7fb729ada640 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 0x5609dc8b4be0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb71c015bd0 tx=0x7fb71c009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbe6f401ffa8c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.351+0000 7fb72aadc640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb7100b2e20 0x7fb7100b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.352+0000 7fb718bfe640 1 -- 172.21.15.5:0/1720704501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c015110 con 0x5609dc5141b0 2024-05-14T15:35:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.352+0000 7fb718bfe640 1 -- 172.21.15.5:0/1720704501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c003e20 con 0x5609dc5141b0 2024-05-14T15:35:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.352+0000 7fb718bfe640 1 -- 172.21.15.5:0/1720704501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0043c0 con 0x5609dc5141b0 2024-05-14T15:35:10.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.353+0000 7fb7333f8740 1 -- 172.21.15.5:0/1720704501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 msgr2=0x5609dc8b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.353+0000 7fb7333f8740 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 0x5609dc8b4be0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb71c015bd0 tx=0x7fb71c009720 comp rx=0 tx=0).stop 2024-05-14T15:35:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.354+0000 7fb7333f8740 1 -- 172.21.15.5:0/1720704501 shutdown_connections 2024-05-14T15:35:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.354+0000 7fb7333f8740 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc4f34d0 0x5609dc8b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.354+0000 7fb7333f8740 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.354+0000 7fb7333f8740 1 --2- 172.21.15.5:0/1720704501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc5141b0 0x5609dc8b4be0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.354+0000 7fb7333f8740 1 -- 172.21.15.5:0/1720704501 >> 172.21.15.5:0/1720704501 conn(0x5609dc508300 msgr2=0x5609dc8b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.355+0000 7fb72aadc640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb7100b2e20 0x7fb7100b5330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7140c9cd0 tx=0x7fb714033f00 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.357+0000 7fb7333f8740 1 -- 172.21.15.5:0/1720704501 shutdown_connections 2024-05-14T15:35:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.360+0000 7fb7333f8740 1 -- 172.21.15.5:0/1720704501 wait complete. 2024-05-14T15:35:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.361+0000 7fb7333f8740 1 -- start start 2024-05-14T15:35:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.361+0000 7fb7333f8740 1 Processor -- start 2024-05-14T15:35:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.362+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8bf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.362+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4f34d0 0x5609dc8bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.362+0000 7fb729ada640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.362+0000 7fb729ada640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8bf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57866/0 (socket says 172.21.15.5:57866) 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.362+0000 7fb729ada640 1 -- 172.21.15.5:0/2940631465 learned_addr learned my addr 172.21.15.5:0/2940631465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.363+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.363+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609dc8c1cb0 con 0x5609dc4f34d0 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.363+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609dc7d7370 con 0x5609dc4eb2f0 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.363+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5609dc8c1e30 con 0x5609dc5141b0 2024-05-14T15:35:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.363+0000 7fb7333f8740 10 client.0 _ll_register_callbacks cb 0x5609dc52ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.365+0000 7fb72a2db640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4f34d0 0x5609dc8bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.366+0000 7fb729ada640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.369+0000 7fb72a2db640 1 -- 172.21.15.5:0/2940631465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 msgr2=0x5609dc8c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.369+0000 7fb72a2db640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.369+0000 7fb72a2db640 1 -- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 msgr2=0x5609dc8bf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.369+0000 7fb72a2db640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8bf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.369+0000 7fb72a2db640 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dc5f8620 con 0x5609dc4f34d0 2024-05-14T15:35:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.370+0000 7fb72a2db640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4f34d0 0x5609dc8bf7c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x5609dc8b2320 tx=0x7fb720009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e8cded44ad89a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.370+0000 7fb70dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.371+0000 7fb70dffb640 1 -- 172.21.15.5:0/2940631465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720011420 con 0x5609dc4f34d0 2024-05-14T15:35:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.371+0000 7fb70dffb640 1 -- 172.21.15.5:0/2940631465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fb7200115c0 con 0x5609dc4f34d0 2024-05-14T15:35:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.371+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71c003e20 con 0x5609dc4f34d0 2024-05-14T15:35:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.379+0000 7fb70dffb640 1 -- 172.21.15.5:0/2940631465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb720011800 con 0x5609dc4f34d0 2024-05-14T15:35:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.379+0000 7fb7333f8740 10 client.17886 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.379+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5609dc8ce160 con 0x5609dc4f34d0 2024-05-14T15:35:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.383+0000 7fb70dffb640 1 -- 172.21.15.5:0/2940631465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c003e20 con 0x5609dc4f34d0 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.384+0000 7fb70dffb640 1 -- 172.21.15.5:0/2940631465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb720010a40 con 0x5609dc4f34d0 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.385+0000 7fb7333f8740 10 client.17886 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.385+0000 7fb7333f8740 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.386+0000 7fb7333f8740 20 client.17886 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.386+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4d2d80 0x5609dc8d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.386+0000 7fb7333f8740 4 client.17886 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.386+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609dc8281e0 con 0x5609dc4d2d80 2024-05-14T15:35:10.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.386+0000 7fb729ada640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4d2d80 0x5609dc8d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.396+0000 7fb729ada640 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4d2d80 0x5609dc8d16b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b874efe128b2a16 server_cookie=978b4e9b178070cb in_seq=0 out_seq=0 2024-05-14T15:35:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.397+0000 7fb70dffb640 10 client.17886 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.401+0000 7fb70dffb640 1 -- 172.21.15.5:0/2940631465 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609dc8281e0 con 0x5609dc4d2d80 2024-05-14T15:35:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.401+0000 7fb70dffb640 10 client.17886 handle_command_reply: tid=0 2024-05-14T15:35:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.402+0000 7fb7333f8740 1 client.17886 shutdown 2024-05-14T15:35:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.403+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4f34d0 msgr2=0x5609dc8bf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.403+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4f34d0 0x5609dc8bf7c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x5609dc8b2320 tx=0x7fb720009490 comp rx=0 tx=0).stop 2024-05-14T15:35:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 shutdown_connections 2024-05-14T15:35:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4d2d80 0x5609dc8d16b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4f34d0 0x5609dc8bf7c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb7333f8740 1 --2- 172.21.15.5:0/2940631465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4eb2f0 0x5609dc8bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 >> 172.21.15.5:0/2940631465 conn(0x5609dc508300 msgr2=0x5609dc8b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.406+0000 7fb70dffb640 0 client.17886 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.410+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 shutdown_connections 2024-05-14T15:35:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.412+0000 7fb7333f8740 1 -- 172.21.15.5:0/2940631465 wait complete. 2024-05-14T15:35:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.413+0000 7fb7333f8740 20 client.17886 trim_cache size 0 max 16384 2024-05-14T15:35:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.444+0000 7fb7333f8740 1 Processor -- start 2024-05-14T15:35:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.444+0000 7fb7333f8740 1 -- start start 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.445+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.445+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.445+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.445+0000 7fb7333f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241cb3f0 con 0x5609dc4eb2f0 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.446+0000 7fb72a2db640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.446+0000 7fb72a2db640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8b98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57872/0 (socket says 172.21.15.5:57872) 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.446+0000 7fb72a2db640 1 -- 172.21.15.5:0/26036855 learned_addr learned my addr 172.21.15.5:0/26036855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.446+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241c8cc0 con 0x5609dc4f34d0 2024-05-14T15:35:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.446+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241c6be0 con 0x5609dc5141b0 2024-05-14T15:35:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.448+0000 7fb729ada640 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.448+0000 7fb729ada640 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.449+0000 7fb729ada640 1 -- 172.21.15.5:0/26036855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 msgr2=0x5609dc8b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.449+0000 7fb729ada640 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.449+0000 7fb729ada640 1 -- 172.21.15.5:0/26036855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 msgr2=0x5609dc8c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.449+0000 7fb729ada640 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.449+0000 7fb729ada640 1 -- 172.21.15.5:0/26036855 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dc8bf4b0 con 0x5609dc5141b0 2024-05-14T15:35:10.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.450+0000 7fb72a2db640 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.451+0000 7fb729ada640 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c50a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb71c015ba0 tx=0x7fb71c026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8633ab393bca34e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.452+0000 7fb70ffff640 1 -- 172.21.15.5:0/26036855 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c010d70 con 0x5609dc5141b0 2024-05-14T15:35:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.453+0000 7fb70ffff640 1 -- 172.21.15.5:0/26036855 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c003e20 con 0x5609dc5141b0 2024-05-14T15:35:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.453+0000 7fb70ffff640 1 -- 172.21.15.5:0/26036855 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c02faa0 con 0x5609dc5141b0 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 msgr2=0x5609dc8c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c50a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb71c015ba0 tx=0x7fb71c026040 comp rx=0 tx=0).stop 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 shutdown_connections 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c50a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 --2- 172.21.15.5:0/26036855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.454+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 >> 172.21.15.5:0/26036855 conn(0x5609dc508300 msgr2=0x5609dc8bd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.455+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 shutdown_connections 2024-05-14T15:35:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.455+0000 7fb7333f8740 1 -- 172.21.15.5:0/26036855 wait complete. 2024-05-14T15:35:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.455+0000 7fb7333f8740 1 -- start start 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 Processor -- start 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241c71a0 con 0x5609dc4eb2f0 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241c85f0 con 0x5609dc4f34d0 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241c7850 con 0x5609dc5141b0 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.456+0000 7fb7333f8740 10 client.0 _ll_register_callbacks cb 0x7fb720006df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.458+0000 7fb72a2db640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.458+0000 7fb72a2db640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57884/0 (socket says 172.21.15.5:57884) 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.458+0000 7fb72a2db640 1 -- 172.21.15.5:0/3602593728 learned_addr learned my addr 172.21.15.5:0/3602593728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:10.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.458+0000 7fb729ada640 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.459+0000 7fb72a2db640 1 -- 172.21.15.5:0/3602593728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 msgr2=0x5609dc8c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.459+0000 7fb72a2db640 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.459+0000 7fb72a2db640 1 -- 172.21.15.5:0/3602593728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 msgr2=0x5609dc8cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.459+0000 7fb72a2db640 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.459+0000 7fb72a2db640 1 -- 172.21.15.5:0/3602593728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dc8c6ef0 con 0x5609dc4f34d0 2024-05-14T15:35:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.460+0000 7fb72a2db640 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8c6470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5609dc7c36e0 tx=0x7fb72000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9e2876d26e628c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.461+0000 7fb70dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:35:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.461+0000 7fb70dffb640 1 -- 172.21.15.5:0/3602593728 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720011ba0 con 0x5609dc4f34d0 2024-05-14T15:35:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.461+0000 7fb70dffb640 1 -- 172.21.15.5:0/3602593728 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fb720011d40 con 0x5609dc4f34d0 2024-05-14T15:35:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.463+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71c003e20 con 0x5609dc4f34d0 2024-05-14T15:35:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.468+0000 7fb70dffb640 1 -- 172.21.15.5:0/3602593728 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb720011630 con 0x5609dc4f34d0 2024-05-14T15:35:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.470+0000 7fb7333f8740 10 client.27491 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.470+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5609dc939630 con 0x5609dc4f34d0 2024-05-14T15:35:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.475+0000 7fb70dffb640 1 -- 172.21.15.5:0/3602593728 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71c003e20 con 0x5609dc4f34d0 2024-05-14T15:35:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.475+0000 7fb70dffb640 1 -- 172.21.15.5:0/3602593728 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb720010600 con 0x5609dc4f34d0 2024-05-14T15:35:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.476+0000 7fb7333f8740 10 client.27491 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.476+0000 7fb7333f8740 10 client.27491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.476+0000 7fb7333f8740 20 client.27491 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.476+0000 7fb7333f8740 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4a81c0 0x5609dc93bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.477+0000 7fb7333f8740 4 client.27491 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:35:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.477+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609dc822050 con 0x5609dc4a81c0 2024-05-14T15:35:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.478+0000 7fb729ada640 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4a81c0 0x5609dc93bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.482+0000 7fb729ada640 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4a81c0 0x5609dc93bda0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7eec22d9878135 server_cookie=e3b79319c35a4d96 in_seq=0 out_seq=0 2024-05-14T15:35:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.482+0000 7fb70dffb640 10 client.27491 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:10.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.529+0000 7f1576520700 1 -- 172.21.15.5:0/1506645249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 msgr2=0x7f1570108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.529+0000 7f1576520700 1 --2- 172.21.15.5:0/1506645249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f1570108b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1564009990 tx=0x7f1564009ca0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.543+0000 7f1576520700 1 -- 172.21.15.5:0/1506645249 shutdown_connections 2024-05-14T15:35:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.543+0000 7f1576520700 1 --2- 172.21.15.5:0/1506645249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f1570108b90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.543+0000 7f1576520700 1 --2- 172.21.15.5:0/1506645249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103270 0x7f1570103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.543+0000 7f1576520700 1 --2- 172.21.15.5:0/1506645249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1570103bc0 0x7f15701080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.543+0000 7f1576520700 1 -- 172.21.15.5:0/1506645249 >> 172.21.15.5:0/1506645249 conn(0x7f15700fef30 msgr2=0x7f1570101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.543+0000 7f1576520700 1 -- 172.21.15.5:0/1506645249 shutdown_connections 2024-05-14T15:35:10.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.545+0000 7f1576520700 1 -- 172.21.15.5:0/1506645249 wait complete. 2024-05-14T15:35:10.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 Processor -- start 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 -- start start 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1570103270 0x7f1570075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103bc0 0x7f157006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f157006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f157011aa90 con 0x7f1570103bc0 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f157011a910 con 0x7f15701086c0 2024-05-14T15:35:10.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.546+0000 7f1576520700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f157011ac10 con 0x7f1570103270 2024-05-14T15:35:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f156ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1570103270 0x7f1570075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f1574abd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f157006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f156f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103bc0 0x7f157006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f156f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103bc0 0x7f157006e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57020/0 (socket says 172.21.15.5:57020) 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f156f7fe700 1 -- 172.21.15.5:0/2728655280 learned_addr learned my addr 172.21.15.5:0/2728655280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f1574abd700 1 -- 172.21.15.5:0/2728655280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1570103270 msgr2=0x7f1570075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f1574abd700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1570103270 0x7f1570075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f1574abd700 1 -- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103bc0 msgr2=0x7f157006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.552+0000 7f1574abd700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103bc0 0x7f157006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.553+0000 7f1574abd700 1 -- 172.21.15.5:0/2728655280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1570072eb0 con 0x7f15701086c0 2024-05-14T15:35:10.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.553+0000 7f1574abd700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f157006eab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1564009990 tx=0x7f15640057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.553+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564027070 con 0x7f15701086c0 2024-05-14T15:35:10.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.553+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15700730e0 con 0x7f15701086c0 2024-05-14T15:35:10.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.553+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1570073610 con 0x7f15701086c0 2024-05-14T15:35:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.554+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15640095c0 con 0x7f15701086c0 2024-05-14T15:35:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.554+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564022e00 con 0x7f15701086c0 2024-05-14T15:35:10.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.555+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c005330 con 0x7f15701086c0 2024-05-14T15:35:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.558+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1564009d10 con 0x7f15701086c0 2024-05-14T15:35:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.559+0000 7f156d7fa700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15580b2af0 0x7f15580b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.560+0000 7f156ffff700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15580b2af0 0x7f15580b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.560+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7f15640e37c0 con 0x7f15701086c0 2024-05-14T15:35:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.563+0000 7f156ffff700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15580b2af0 0x7f15580b5000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1568009550 tx=0x7f15680091b0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:10.566+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1564013070 con 0x7f15701086c0 2024-05-14T15:35:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb70dffb640 1 -- 172.21.15.5:0/3602593728 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609dc822050 con 0x5609dc4a81c0 2024-05-14T15:35:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb70dffb640 10 client.27491 handle_command_reply: tid=0 2024-05-14T15:35:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 client.27491 shutdown 2024-05-14T15:35:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 msgr2=0x5609dc8c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8c6470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5609dc7c36e0 tx=0x7fb72000bf80 comp rx=0 tx=0).stop 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 shutdown_connections 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5609dc4a81c0 0x5609dc93bda0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609dc5141b0 0x5609dc8c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609dc4f34d0 0x5609dc8c6470 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 --2- 172.21.15.5:0/3602593728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609dc4eb2f0 0x5609dc8cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 >> 172.21.15.5:0/3602593728 conn(0x5609dc508300 msgr2=0x5609dc8b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb70dffb640 0 client.27491 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 shutdown_connections 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.628+0000 7fb7333f8740 1 -- 172.21.15.5:0/3602593728 wait complete. 2024-05-14T15:35:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.629+0000 7fb7333f8740 20 client.27491 trim_cache size 0 max 16384 2024-05-14T15:35:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:35:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:35:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:35:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:35:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.630+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb7100b2e20 msgr2=0x7fb7100b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.630+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb7100b2e20 0x7fb7100b5330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7140c9cd0 tx=0x7fb714033f00 comp rx=0 tx=0).stop 2024-05-14T15:35:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 msgr2=0x7fb72411b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 0x7fb72411b710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb714006010 tx=0x7fb7140315d0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 shutdown_connections 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb7100b2e20 0x7fb7100b5330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb72410f950 0x7fb72411b710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb72410ee90 0x7fb72411b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 --2- 172.21.15.5:0/2257394885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb72410a900 0x7fb72411ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 >> 172.21.15.5:0/2257394885 conn(0x7fb72406edb0 msgr2=0x7fb72410ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 shutdown_connections 2024-05-14T15:35:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:10.631+0000 7fb72c53f640 1 -- 172.21.15.5:0/2257394885 wait complete. 2024-05-14T15:35:11.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.098+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f155c005620 con 0x7f15701086c0 2024-05-14T15:35:11.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.101+0000 7f156d7fa700 1 -- 172.21.15.5:0/2728655280 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11864 (secure 0 0 0) 0x7f15640aad60 con 0x7f15701086c0 2024-05-14T15:35:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:35:11.103 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:30:54.824948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/1180616641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1180616641},{"type":"v1","addr":"172.21.15.144:6835","nonce":1180616641}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:35:11.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.105+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15580b2af0 msgr2=0x7f15580b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:11.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.105+0000 7f1576520700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15580b2af0 0x7f15580b5000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1568009550 tx=0x7f15680091b0 comp rx=0 tx=0).stop 2024-05-14T15:35:11.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 msgr2=0x7f157006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:11.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f157006eab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1564009990 tx=0x7f15640057c0 comp rx=0 tx=0).stop 2024-05-14T15:35:11.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 shutdown_connections 2024-05-14T15:35:11.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f15580b2af0 0x7f15580b5000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:11.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15701086c0 0x7f157006eab0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:11.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1570103270 0x7f1570075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:11.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 --2- 172.21.15.5:0/2728655280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1570103bc0 0x7f157006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:11.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.106+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 >> 172.21.15.5:0/2728655280 conn(0x7f15700fef30 msgr2=0x7f1570106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:11.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.109+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 shutdown_connections 2024-05-14T15:35:11.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:11.111+0000 7f1576520700 1 -- 172.21.15.5:0/2728655280 wait complete. 2024-05-14T15:35:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-05-14T15:35:11.154 INFO:tasks.workunit.client.0.smithi005.stdout: 1 963223 9.55 MB/sec execute 392 sec latency 96.077 ms 2024-05-14T15:35:11.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:35:12.155 INFO:tasks.workunit.client.0.smithi005.stdout: 1 963645 9.53 MB/sec execute 393 sec latency 52.497 ms 2024-05-14T15:35:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:11 smithi144 ceph-mon[35650]: pgmap v325: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 95 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-05-14T15:35:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:11 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2728655280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:35:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:11 smithi193 ceph-mon[36001]: pgmap v325: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 95 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-05-14T15:35:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:11 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2728655280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:35:12.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:11 smithi005 ceph-mon[31559]: pgmap v325: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 95 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-05-14T15:35:12.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:11 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2728655280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:35:12.339 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:13.155 INFO:tasks.workunit.client.0.smithi005.stdout: 1 964729 9.52 MB/sec execute 394 sec latency 74.909 ms 2024-05-14T15:35:14.155 INFO:tasks.workunit.client.0.smithi005.stdout: 1 966388 9.52 MB/sec execute 395 sec latency 51.445 ms 2024-05-14T15:35:14.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:13 smithi005 ceph-mon[31559]: pgmap v326: 129 pgs: 9 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 96 active+clean; 83 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-14T15:35:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:13 smithi144 ceph-mon[35650]: pgmap v326: 129 pgs: 9 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 96 active+clean; 83 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-14T15:35:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:13 smithi193 ceph-mon[36001]: pgmap v326: 129 pgs: 9 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 96 active+clean; 83 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-14T15:35:15.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 967184 9.51 MB/sec execute 396 sec latency 54.504 ms 2024-05-14T15:35:16.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 969375 9.51 MB/sec execute 397 sec latency 19.490 ms 2024-05-14T15:35:16.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:15 smithi005 ceph-mon[31559]: pgmap v327: 129 pgs: 9 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 96 active+clean; 83 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-14T15:35:16.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 -- 172.21.15.5:0/4058836510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a410a3e0 msgr2=0x7fe6a410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/4058836510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a410a3e0 0x7fe6a410a7f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe698009990 tx=0x7fe698009ca0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 -- 172.21.15.5:0/4058836510 shutdown_connections 2024-05-14T15:35:16.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/4058836510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4103130 0x7fe6a4111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/4058836510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a410a3e0 0x7fe6a410a7f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/4058836510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a41027e0 0x7fe6a4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 -- 172.21.15.5:0/4058836510 >> 172.21.15.5:0/4058836510 conn(0x7fe6a40fe6f0 msgr2=0x7fe6a4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:16.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 -- 172.21.15.5:0/4058836510 shutdown_connections 2024-05-14T15:35:16.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 -- 172.21.15.5:0/4058836510 wait complete. 2024-05-14T15:35:16.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.190+0000 7fe6ab0df700 1 Processor -- start 2024-05-14T15:35:16.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 -- start start 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 0x7fe6a41039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4103130 0x7fe6a4103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a410a3e0 0x7fe6a4104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4114480 con 0x7fe6a41027e0 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4114300 con 0x7fe6a4103130 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6ab0df700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4114600 con 0x7fe6a410a3e0 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 0x7fe6a41039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 0x7fe6a41039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46232/0 (socket says 172.21.15.5:46232) 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 -- 172.21.15.5:0/3931337391 learned_addr learned my addr 172.21.15.5:0/3931337391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4103130 0x7fe6a4103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a410a3e0 0x7fe6a4104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 -- 172.21.15.5:0/3931337391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a410a3e0 msgr2=0x7fe6a4104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a410a3e0 0x7fe6a4104440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 -- 172.21.15.5:0/3931337391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4103130 msgr2=0x7fe6a4103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4103130 0x7fe6a4103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.191+0000 7fe6a8e7b700 1 -- 172.21.15.5:0/3931337391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41a91f0 con 0x7fe6a41027e0 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.192+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 0x7fe6a41039c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe698000c00 tx=0x7fe69800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.192+0000 7fe6a1ffb700 1 -- 172.21.15.5:0/3931337391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698027070 con 0x7fe6a41027e0 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.192+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41a9480 con 0x7fe6a41027e0 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.192+0000 7fe6a1ffb700 1 -- 172.21.15.5:0/3931337391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe698022470 con 0x7fe6a41027e0 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.192+0000 7fe6a1ffb700 1 -- 172.21.15.5:0/3931337391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698017650 con 0x7fe6a41027e0 2024-05-14T15:35:16.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.192+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a9930 con 0x7fe6a41027e0 2024-05-14T15:35:16.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.195+0000 7fe6a1ffb700 1 -- 172.21.15.5:0/3931337391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe69802a070 con 0x7fe6a41027e0 2024-05-14T15:35:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6a1ffb700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe6940b29f0 0x7fe6940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 Processor -- start 2024-05-14T15:35:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe6940b29f0 0x7fe6940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6a1ffb700 1 -- 172.21.15.5:0/3931337391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fe6980e3790 con 0x7fe6a41027e0 2024-05-14T15:35:16.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 -- start start 2024-05-14T15:35:16.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a1477ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a1477f450 0x558a14783720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14784070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe6940b29f0 0x7fe6940b4f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4100370 tx=0x7fe690009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a14698860 con 0x558a1477f450 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558a146377c0 con 0x558a145f8e40 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.196+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a14694820 con 0x558a14783c60 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a1477f450 0x558a14783720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14784070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a1477ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14784070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59270/0 (socket says 172.21.15.5:59270) 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a1477ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55508/0 (socket says 172.21.15.5:55508) 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 -- 172.21.15.5:0/3715367242 learned_addr learned my addr 172.21.15.5:0/3715367242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a1477f450 0x558a14783720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46246/0 (socket says 172.21.15.5:46246) 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 -- 172.21.15.5:0/3715367242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 msgr2=0x558a1477ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a1477ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 -- 172.21.15.5:0/3715367242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a1477f450 msgr2=0x558a14783720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a1477f450 0x558a14783720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 -- 172.21.15.5:0/3715367242 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a145e58d0 con 0x558a14783c60 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14784070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe69c011680 tx=0x7fe69c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442c2ab14e9e36b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe68effd700 1 -- 172.21.15.5:0/3715367242 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c019030 con 0x558a14783c60 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe68effd700 1 -- 172.21.15.5:0/3715367242 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe69c0092e0 con 0x558a14783c60 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.197+0000 7fe68effd700 1 -- 172.21.15.5:0/3715367242 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c007500 con 0x558a14783c60 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3715367242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 msgr2=0x558a14784070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14784070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe69c011680 tx=0x7fe69c009920 comp rx=0 tx=0).stop 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3715367242 shutdown_connections 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14784070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a1477ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3715367242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a1477f450 0x558a14783720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3715367242 >> 172.21.15.5:0/3715367242 conn(0x558a14640600 msgr2=0x558a147714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3715367242 shutdown_connections 2024-05-14T15:35:16.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.198+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3715367242 wait complete. 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.199+0000 7fe6b9af3b80 1 -- start start 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.199+0000 7fe6b9af3b80 1 Processor -- start 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.199+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a145f8e40 0x558a14788980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.199+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a14788ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14789400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a1478bbb0 con 0x558a145f8e40 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558a145fec30 con 0x558a1477f450 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a1478bd30 con 0x558a14783c60 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a145f8e40 0x558a14788980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a14788ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a14788ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55522/0 (socket says 172.21.15.5:55522) 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a3fff700 1 -- 172.21.15.5:0/3597131173 learned_addr learned my addr 172.21.15.5:0/3597131173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a145f8e40 0x558a14788980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46256/0 (socket says 172.21.15.5:46256) 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14789400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6b9af3b80 10 client.0 _ll_register_callbacks cb 0x558a14630920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 -- 172.21.15.5:0/3597131173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 msgr2=0x558a14788ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a14788ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 -- 172.21.15.5:0/3597131173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a145f8e40 msgr2=0x558a14788980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a145f8e40 0x558a14788980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 -- 172.21.15.5:0/3597131173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a14634690 con 0x558a14783c60 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a14788ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe6a967c700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14789400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe69c011680 tx=0x7fe69c003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41fd19c4975a4a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.200+0000 7fe683fff700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:35:16.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.201+0000 7fe683fff700 1 -- 172.21.15.5:0/3597131173 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c004cd0 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.201+0000 7fe683fff700 1 -- 172.21.15.5:0/3597131173 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fe69c007500 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.201+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69c0092e0 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.201+0000 7fe683fff700 1 -- 172.21.15.5:0/3597131173 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe69c030050 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.201+0000 7fe6b9af3b80 10 client.27493 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a147967b0 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe683fff700 1 -- 172.21.15.5:0/3597131173 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe69c0092e0 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe683fff700 1 -- 172.21.15.5:0/3597131173 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe69c005050 con 0x558a14783c60 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 10 client.27493 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 10 client.27493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 20 client.27493 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14798d30 0x558a1479b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 4 client.27493 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:16.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.202+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a146a6240 con 0x558a14798d30 2024-05-14T15:35:16.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.203+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14798d30 0x558a1479b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.203+0000 7fe683fff700 10 client.27493 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.203+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14798d30 0x558a1479b150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d62ec1ddd6bc105c server_cookie=47b66b5593a9175f in_seq=0 out_seq=0 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe683fff700 1 -- 172.21.15.5:0/3597131173 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a146a6240 con 0x558a14798d30 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe683fff700 10 client.27493 handle_command_reply: tid=0 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 client.27493 shutdown 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 msgr2=0x558a14789400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14789400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe69c011680 tx=0x7fe69c003c30 comp rx=0 tx=0).stop 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 shutdown_connections 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a14783c60 0x558a14789400 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a145f8e40 0x558a14788980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14798d30 0x558a1479b150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/3597131173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a14788ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 >> 172.21.15.5:0/3597131173 conn(0x558a14640600 msgr2=0x558a14775450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.204+0000 7fe683fff700 0 client.27493 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:35:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.205+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 shutdown_connections 2024-05-14T15:35:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.206+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/3597131173 wait complete. 2024-05-14T15:35:16.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.206+0000 7fe6b9af3b80 20 client.27493 trim_cache size 0 max 16384 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 Processor -- start 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 -- start start 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a14771f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 0x558a14771610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a14794fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41b0f20 con 0x558a14783c60 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41ae7b0 con 0x558a145f8e40 2024-05-14T15:35:16.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41ac640 con 0x558a1477f450 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 0x558a14771610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a967c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a14794fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 0x558a14771610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59284/0 (socket says 172.21.15.5:59284) 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 -- 172.21.15.5:0/1472415384 learned_addr learned my addr 172.21.15.5:0/1472415384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a14771f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 -- 172.21.15.5:0/1472415384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 msgr2=0x558a14771f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a14771f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 -- 172.21.15.5:0/1472415384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 msgr2=0x558a14794fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a14794fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.222+0000 7fe6a3fff700 1 -- 172.21.15.5:0/1472415384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1478b6c0 con 0x558a1477f450 2024-05-14T15:35:16.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.223+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 0x558a14771610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe6900042f0 tx=0x7fe690014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7038950ce4a2781d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:16.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.223+0000 7fe68e7fc700 1 -- 172.21.15.5:0/1472415384 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69002c070 con 0x558a1477f450 2024-05-14T15:35:16.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.223+0000 7fe68e7fc700 1 -- 172.21.15.5:0/1472415384 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe690003c90 con 0x558a1477f450 2024-05-14T15:35:16.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.223+0000 7fe68e7fc700 1 -- 172.21.15.5:0/1472415384 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690027660 con 0x558a1477f450 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/1472415384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 msgr2=0x558a14771610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 0x558a14771610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe6900042f0 tx=0x7fe690014000 comp rx=0 tx=0).stop 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/1472415384 shutdown_connections 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a14794fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a145f8e40 0x558a14771f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/1472415384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a1477f450 0x558a14771610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/1472415384 >> 172.21.15.5:0/1472415384 conn(0x558a14640600 msgr2=0x558a14783520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/1472415384 shutdown_connections 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.224+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/1472415384 wait complete. 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 -- start start 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 Processor -- start 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 0x558a1479caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a1479d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a1479d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41acc90 con 0x558a14783c60 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41ae0e0 con 0x558a1477f450 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41ad340 con 0x558a145f8e40 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 0x558a1479caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a1479d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6b9af3b80 10 client.0 _ll_register_callbacks cb 0x558a14630920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6a967c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a1479d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a1479d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55542/0 (socket says 172.21.15.5:55542) 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6a8e7b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 0x558a1479caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59292/0 (socket says 172.21.15.5:59292) 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.225+0000 7fe6a3fff700 1 -- 172.21.15.5:0/211766892 learned_addr learned my addr 172.21.15.5:0/211766892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a3fff700 1 -- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 msgr2=0x558a1479caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 0x558a1479caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a3fff700 1 -- 172.21.15.5:0/211766892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 msgr2=0x558a1479d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a1479d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a3fff700 1 -- 172.21.15.5:0/211766892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a14801400 con 0x558a1477f450 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 0x558a1479caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a967c700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a1479d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6a3fff700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a1479d030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6900046e0 tx=0x7fe69001d8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64130a2b2bb6a053 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe683fff700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe683fff700 1 -- 172.21.15.5:0/211766892 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69002c040 con 0x558a1477f450 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe683fff700 1 -- 172.21.15.5:0/211766892 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fe690003e40 con 0x558a1477f450 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.226+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe690003c90 con 0x558a1477f450 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe683fff700 1 -- 172.21.15.5:0/211766892 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe690005030 con 0x558a1477f450 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 10 client.27515 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a14802660 con 0x558a1477f450 2024-05-14T15:35:16.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe683fff700 1 -- 172.21.15.5:0/211766892 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe690003c90 con 0x558a1477f450 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe683fff700 1 -- 172.21.15.5:0/211766892 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fe690004c80 con 0x558a1477f450 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 10 client.27515 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 10 client.27515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 20 client.27515 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14802c60 0x558a14805080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 4 client.27515 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:35:16.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.227+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a146a02e0 con 0x558a14802c60 2024-05-14T15:35:16.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.228+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14802c60 0x558a14805080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:16.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6a8e7b700 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14802c60 0x558a14805080 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1ccc181cc388b36 server_cookie=5b7bff4ee57e9e46 in_seq=0 out_seq=0 2024-05-14T15:35:16.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe683fff700 10 client.27515 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:35:16.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe683fff700 1 -- 172.21.15.5:0/211766892 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a146a02e0 con 0x558a14802c60 2024-05-14T15:35:16.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe683fff700 10 client.27515 handle_command_reply: tid=0 2024-05-14T15:35:16.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 client.27515 shutdown 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 msgr2=0x558a1479d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a1479d030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6900046e0 tx=0x7fe69001d8c0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 shutdown_connections 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a14783c60 0x558a1479d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a145f8e40 0x558a1479caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x558a14802c60 0x558a14805080 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 --2- 172.21.15.5:0/211766892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a1477f450 0x558a1479d030 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe683fff700 0 client.27515 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.229+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 >> 172.21.15.5:0/211766892 conn(0x558a14640600 msgr2=0x558a14783520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.230+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 shutdown_connections 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.230+0000 7fe6b9af3b80 1 -- 172.21.15.5:0/211766892 wait complete. 2024-05-14T15:35:16.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.230+0000 7fe6b9af3b80 20 client.27515 trim_cache size 0 max 16384 2024-05-14T15:35:16.235 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:35:16.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe6940b29f0 msgr2=0x7fe6940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe6940b29f0 0x7fe6940b4f00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4100370 tx=0x7fe690009260 comp rx=0 tx=0).stop 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 msgr2=0x7fe6a41039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 0x7fe6a41039c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe698000c00 tx=0x7fe69800bfd0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 shutdown_connections 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe6940b29f0 0x7fe6940b4f00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a410a3e0 0x7fe6a4104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6a41027e0 0x7fe6a41039c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 --2- 172.21.15.5:0/3931337391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6a4103130 0x7fe6a4103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 >> 172.21.15.5:0/3931337391 conn(0x7fe6a40fe6f0 msgr2=0x7fe6a4100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 shutdown_connections 2024-05-14T15:35:16.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:16.231+0000 7fe6ab0df700 1 -- 172.21.15.5:0/3931337391 wait complete. 2024-05-14T15:35:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:15 smithi144 ceph-mon[35650]: pgmap v327: 129 pgs: 9 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 96 active+clean; 83 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-14T15:35:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:15 smithi193 ceph-mon[36001]: pgmap v327: 129 pgs: 9 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 96 active+clean; 83 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 256 op/s 2024-05-14T15:35:17.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:35:17.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 970606 9.51 MB/sec execute 398 sec latency 20.316 ms 2024-05-14T15:35:17.291 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:18.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 972532 9.50 MB/sec execute 399 sec latency 31.214 ms 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: pgmap v328: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 82 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 243 op/s 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:17 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: pgmap v328: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 82 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 243 op/s 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:17 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: pgmap v328: 129 pgs: 8 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 97 active+clean; 82 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 243 op/s 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:18.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:17 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:19.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 973971 9.51 MB/sec execute 400 sec latency 28.542 ms 2024-05-14T15:35:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.283+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/1697302473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d01045d0 msgr2=0x7fb5d01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.283+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/1697302473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d01045d0 0x7fb5d01049e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00b0d0 tx=0x7fb5cc00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/1697302473 shutdown_connections 2024-05-14T15:35:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/1697302473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5d0109850 0x7fb5d01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/1697302473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d01045d0 0x7fb5d01049e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/1697302473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d0108f00 0x7fb5d0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/1697302473 >> 172.21.15.5:0/1697302473 conn(0x7fb5d00ffea0 msgr2=0x7fb5d01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/1697302473 shutdown_connections 2024-05-14T15:35:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/1697302473 wait complete. 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 Processor -- start 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.284+0000 7fb5d7c5f700 1 -- start start 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d7c5f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 0x7fb5d019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d7c5f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5d0108f00 0x7fb5d019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d7c5f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0109850 0x7fb5d019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d7c5f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0114480 con 0x7fb5d0109850 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d7c5f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0114300 con 0x7fb5d01045d0 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d7c5f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0114600 con 0x7fb5d0108f00 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 0x7fb5d019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 0x7fb5d019ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55556/0 (socket says 172.21.15.5:55556) 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 -- 172.21.15.5:0/2995991332 learned_addr learned my addr 172.21.15.5:0/2995991332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d61fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0109850 0x7fb5d019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5d0108f00 0x7fb5d019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 -- 172.21.15.5:0/2995991332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5d0108f00 msgr2=0x7fb5d019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5d0108f00 0x7fb5d019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 -- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0109850 msgr2=0x7fb5d019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0109850 0x7fb5d019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 -- 172.21.15.5:0/2995991332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d019fd40 con 0x7fb5d01045d0 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.285+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 0x7fb5d019ae70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc007a30 tx=0x7fb5cc007a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.286+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/2995991332 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc01f070 con 0x7fb5d01045d0 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.286+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d019ffd0 con 0x7fb5d01045d0 2024-05-14T15:35:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.286+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/2995991332 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5cc004c50 con 0x7fb5d01045d0 2024-05-14T15:35:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.286+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/2995991332 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc01b690 con 0x7fb5d01045d0 2024-05-14T15:35:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.286+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01a04b0 con 0x7fb5d01045d0 2024-05-14T15:35:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.289+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/2995991332 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb5cc022070 con 0x7fb5d01045d0 2024-05-14T15:35:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.289+0000 7fb5c6ffd700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb5bc0b29f0 0x7fb5bc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.289+0000 7fb5e6673b80 1 Processor -- start 2024-05-14T15:35:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.289+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb5bc0b29f0 0x7fb5bc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/2995991332 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fb5cc00f020 con 0x7fb5d01045d0 2024-05-14T15:35:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 -- start start 2024-05-14T15:35:19.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x556502804840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x5565027fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556502720940 con 0x5565027fee10 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556502685da0 con 0x556502804d80 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55650271c900 con 0x556502680e40 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb5bc0b29f0 0x7fb5bc0b4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009680 tx=0x7fb5c0009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x5565027fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d61fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x5565027fe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55558/0 (socket says 172.21.15.5:55558) 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 -- 172.21.15.5:0/3761656976 learned_addr learned my addr 172.21.15.5:0/3761656976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x556502804840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 -- 172.21.15.5:0/3761656976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 msgr2=0x556502804840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x556502804840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 -- 172.21.15.5:0/3761656976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 msgr2=0x5565027ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d51fa700 1 -- 172.21.15.5:0/3761656976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650266d8d0 con 0x556502804d80 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.290+0000 7fb5d61fc700 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.291+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x5565027fe8d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0011cf0 tx=0x7fb5c0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0324c5e0bcbf3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.291+0000 7fb5bbfff700 1 -- 172.21.15.5:0/3761656976 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000eb00 con 0x556502804d80 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.291+0000 7fb5bbfff700 1 -- 172.21.15.5:0/3761656976 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c000eca0 con 0x556502804d80 2024-05-14T15:35:19.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.291+0000 7fb5bbfff700 1 -- 172.21.15.5:0/3761656976 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0025630 con 0x556502804d80 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 -- 172.21.15.5:0/3761656976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 msgr2=0x5565027fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x5565027fe8d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0011cf0 tx=0x7fb5c0014000 comp rx=0 tx=0).stop 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 -- 172.21.15.5:0/3761656976 shutdown_connections 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x556502804840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/3761656976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x5565027fe8d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 -- 172.21.15.5:0/3761656976 >> 172.21.15.5:0/3761656976 conn(0x5565026c8630 msgr2=0x5565027f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 -- 172.21.15.5:0/3761656976 shutdown_connections 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 -- 172.21.15.5:0/3761656976 wait complete. 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.292+0000 7fb5e6673b80 1 -- start start 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 Processor -- start 2024-05-14T15:35:19.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x5565028084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565028089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x556502808f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55650280b820 con 0x5565027fee10 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565026d9ed0 con 0x556502804d80 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55650280b9a0 con 0x556502680e40 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5e6673b80 10 client.0 _ll_register_callbacks cb 0x5565026b8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5d61fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x556502808f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5d51fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565028089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5d59fb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x5565028084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5d61fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x556502808f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55568/0 (socket says 172.21.15.5:55568) 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.293+0000 7fb5d61fc700 1 -- 172.21.15.5:0/55868573 learned_addr learned my addr 172.21.15.5:0/55868573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5d61fc700 1 -- 172.21.15.5:0/55868573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 msgr2=0x5565028084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5d61fc700 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x5565028084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5d61fc700 1 -- 172.21.15.5:0/55868573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 msgr2=0x5565028089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5d61fc700 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565028089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5d61fc700 1 -- 172.21.15.5:0/55868573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565026d7760 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5d61fc700 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x556502808f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8013b10 tx=0x7fb5c8013e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6221fc1541d349f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5b97fa700 1 -- 172.21.15.5:0/55868573 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8004520 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5b97fa700 1 -- 172.21.15.5:0/55868573 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fb5c800e820 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c000eca0 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5b97fa700 1 -- 172.21.15.5:0/55868573 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb5c802b050 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5e6673b80 10 client.27527 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5565028164b0 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.294+0000 7fb5b97fa700 1 -- 172.21.15.5:0/55868573 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c000eca0 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5b97fa700 1 -- 172.21.15.5:0/55868573 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c801d400 con 0x556502804d80 2024-05-14T15:35:19.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5e6673b80 10 client.27527 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5e6673b80 10 client.27527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5e6673b80 20 client.27527 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556502818af0 0x55650281af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5e6673b80 4 client.27527 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55650272e320 con 0x556502818af0 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556502818af0 0x55650281af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5b97fa700 10 client.27527 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:35:19.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.295+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556502818af0 0x55650281af10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abe3f535d662411e server_cookie=5713aeafbe15eecb in_seq=0 out_seq=0 2024-05-14T15:35:19.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.296+0000 7fb5b97fa700 1 -- 172.21.15.5:0/55868573 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55650272e320 con 0x556502818af0 2024-05-14T15:35:19.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.296+0000 7fb5b97fa700 10 client.27527 handle_command_reply: tid=0 2024-05-14T15:35:19.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.296+0000 7fb5e6673b80 1 client.27527 shutdown 2024-05-14T15:35:19.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.296+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 msgr2=0x556502808f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.296+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x556502808f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8013b10 tx=0x7fb5c8013e20 comp rx=0 tx=0).stop 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 shutdown_connections 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x556502818af0 0x55650281af10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502804d80 0x556502808f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502680e40 0x5565028084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/55868573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565028089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5b97fa700 0 client.27527 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 >> 172.21.15.5:0/55868573 conn(0x5565026c8630 msgr2=0x5565027fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 shutdown_connections 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 1 -- 172.21.15.5:0/55868573 wait complete. 2024-05-14T15:35:19.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.297+0000 7fb5e6673b80 20 client.27527 trim_cache size 0 max 16384 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 Processor -- start 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 -- start start 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x5565027fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502804d80 0x556502808d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d01b0e80 con 0x5565027fee10 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d01ae710 con 0x556502680e40 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d01ac5a0 con 0x556502804d80 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5d51fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5d51fa700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46300/0 (socket says 172.21.15.5:46300) 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5d51fa700 1 -- 172.21.15.5:0/648097334 learned_addr learned my addr 172.21.15.5:0/648097334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.312+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x5565027fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5d51fa700 1 -- 172.21.15.5:0/648097334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502804d80 msgr2=0x556502808d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502804d80 0x556502808d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5d51fa700 1 -- 172.21.15.5:0/648097334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 msgr2=0x5565027fe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x5565027fe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5d51fa700 1 -- 172.21.15.5:0/648097334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565028092b0 con 0x5565027fee10 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000e690 tx=0x7fb5c0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ad3a36a6e01596 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5bb7fe700 1 -- 172.21.15.5:0/648097334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0035070 con 0x5565027fee10 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5bb7fe700 1 -- 172.21.15.5:0/648097334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c0005040 con 0x5565027fee10 2024-05-14T15:35:19.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.313+0000 7fb5bb7fe700 1 -- 172.21.15.5:0/648097334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c001d670 con 0x5565027fee10 2024-05-14T15:35:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 -- 172.21.15.5:0/648097334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 msgr2=0x5565027ff220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000e690 tx=0x7fb5c0009490 comp rx=0 tx=0).stop 2024-05-14T15:35:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 -- 172.21.15.5:0/648097334 shutdown_connections 2024-05-14T15:35:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556502804d80 0x556502808d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x5565027fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/648097334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565027fee10 0x5565027ff220 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 -- 172.21.15.5:0/648097334 >> 172.21.15.5:0/648097334 conn(0x5565026c8630 msgr2=0x5565027f3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 -- 172.21.15.5:0/648097334 shutdown_connections 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 -- 172.21.15.5:0/648097334 wait complete. 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.314+0000 7fb5e6673b80 1 -- start start 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 Processor -- start 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x556502810e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565027fee10 0x55650281f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556502804d80 0x55650281fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d01acbf0 con 0x556502804d80 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d01ae040 con 0x556502680e40 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d01ad2a0 con 0x5565027fee10 2024-05-14T15:35:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5e6673b80 10 client.0 _ll_register_callbacks cb 0x5565026b8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5d61fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556502804d80 0x55650281fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5d59fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x556502810e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5d61fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556502804d80 0x55650281fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46302/0 (socket says 172.21.15.5:46302) 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5d61fc700 1 -- 172.21.15.5:0/1893435390 learned_addr learned my addr 172.21.15.5:0/1893435390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.315+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565027fee10 0x55650281f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5d59fb700 1 -- 172.21.15.5:0/1893435390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565027fee10 msgr2=0x55650281f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565027fee10 0x55650281f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5d59fb700 1 -- 172.21.15.5:0/1893435390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556502804d80 msgr2=0x55650281fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556502804d80 0x55650281fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5d59fb700 1 -- 172.21.15.5:0/1893435390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556502811590 con 0x556502680e40 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5d59fb700 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x556502810e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0e0d60 tx=0x7fb5cc005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d808f28b06d77e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:35:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1893435390 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0b3090 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c0005040 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1893435390 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7fb5cc0b10c0 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.316+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1893435390 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb5cc0aa490 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 10 client.27533 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556502882c40 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1893435390 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c0005040 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1893435390 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb5cc011070 con 0x556502680e40 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 10 client.27533 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 10 client.27533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 20 client.27533 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5565028832d0 0x5565028856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 4 client.27533 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:35:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565027283c0 con 0x5565028832d0 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.317+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5565028832d0 0x5565028856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5b97fa700 10 client.27533 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5d51fa700 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5565028832d0 0x5565028856f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9641cd541b19bbf3 server_cookie=ac18639cba089e74 in_seq=0 out_seq=0 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1893435390 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565027283c0 con 0x5565028832d0 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5b97fa700 10 client.27533 handle_command_reply: tid=0 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 client.27533 shutdown 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 msgr2=0x556502810e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x556502810e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0e0d60 tx=0x7fb5cc005770 comp rx=0 tx=0).stop 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 shutdown_connections 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5565028832d0 0x5565028856f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556502804d80 0x55650281fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556502680e40 0x556502810e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5b97fa700 0 client.27533 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:35:19.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 --2- 172.21.15.5:0/1893435390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5565027fee10 0x55650281f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 >> 172.21.15.5:0/1893435390 conn(0x5565026c8630 msgr2=0x556502809ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 shutdown_connections 2024-05-14T15:35:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.318+0000 7fb5e6673b80 1 -- 172.21.15.5:0/1893435390 wait complete. 2024-05-14T15:35:19.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5e6673b80 20 client.27533 trim_cache size 0 max 16384 2024-05-14T15:35:19.325 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb5bc0b29f0 msgr2=0x7fb5bc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb5bc0b29f0 0x7fb5bc0b4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009680 tx=0x7fb5c0009260 comp rx=0 tx=0).stop 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 msgr2=0x7fb5d019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 0x7fb5d019ae70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc007a30 tx=0x7fb5cc007a60 comp rx=0 tx=0).stop 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 shutdown_connections 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb5bc0b29f0 0x7fb5bc0b4f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0109850 0x7fb5d019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5d01045d0 0x7fb5d019ae70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 --2- 172.21.15.5:0/2995991332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5d0108f00 0x7fb5d019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 >> 172.21.15.5:0/2995991332 conn(0x7fb5d00ffea0 msgr2=0x7fb5d010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 shutdown_connections 2024-05-14T15:35:19.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:19.319+0000 7fb5d7c5f700 1 -- 172.21.15.5:0/2995991332 wait complete. 2024-05-14T15:35:20.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 975435 9.50 MB/sec execute 401 sec latency 37.599 ms 2024-05-14T15:35:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:19 smithi144 ceph-mon[35650]: pgmap v329: 129 pgs: 129 active+clean; 84 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2024-05-14T15:35:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:19 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:19 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:19 smithi193 ceph-mon[36001]: pgmap v329: 129 pgs: 129 active+clean; 84 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2024-05-14T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:19 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:19 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:20.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:19 smithi005 ceph-mon[31559]: pgmap v329: 129 pgs: 129 active+clean; 84 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2024-05-14T15:35:20.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:19 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:20.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:19 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:20.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:35:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:21.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 977162 9.50 MB/sec execute 402 sec latency 21.986 ms 2024-05-14T15:35:21.329 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-05-14T15:35:21.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: Unable to set osd_memory_target on smithi144 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-14T15:35:21.331 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:21 smithi005 ceph-mon[31559]: pgmap v330: 129 pgs: 129 active+clean; 84 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 204 op/s 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.548 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: Unable to set osd_memory_target on smithi144 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-14T15:35:21.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:21 smithi144 ceph-mon[35650]: pgmap v330: 129 pgs: 129 active+clean; 84 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 204 op/s 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi193 to 176.2M 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: Unable to set osd_memory_target on smithi193 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-05-14T15:35:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-05-14T15:35:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: Unable to set osd_memory_target on smithi144 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-14T15:35:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:21 smithi193 ceph-mon[36001]: pgmap v330: 129 pgs: 129 active+clean; 84 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 204 op/s 2024-05-14T15:35:22.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 978505 9.49 MB/sec execute 403 sec latency 24.661 ms 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1585292663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 msgr2=0x7f20e4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1585292663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e4108e00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009990 tx=0x7f20e0009ca0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1585292663 shutdown_connections 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1585292663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 0x7f20e410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1585292663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e4108e00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1585292663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 0x7f20e4102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1585292663 >> 172.21.15.5:0/1585292663 conn(0x7f20e40fe6f0 msgr2=0x7f20e4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1585292663 shutdown_connections 2024-05-14T15:35:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.923+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1585292663 wait complete. 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:35:22.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:35:22.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 Processor -- start 2024-05-14T15:35:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 -- start start 2024-05-14T15:35:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 0x7f20e419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 0x7f20e419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4112b00 con 0x7f20e41089f0 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4112980 con 0x7f20e4103310 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.924+0000 7f20eb7fe700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4112c80 con 0x7f20e41029c0 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46320/0 (socket says 172.21.15.5:46320) 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e9d9b700 1 -- 172.21.15.5:0/1859517466 learned_addr learned my addr 172.21.15.5:0/1859517466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 0x7f20e419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e959a700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 0x7f20e419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 -- 172.21.15.5:0/1859517466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 msgr2=0x7f20e419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 0x7f20e419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 -- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 msgr2=0x7f20e419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 -- 172.21.15.5:0/1859517466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e41a70c0 con 0x7f20e4103310 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e959a700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 0x7f20e419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.925+0000 7f20e8d99700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 0x7f20e419c2b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20d400e9c0 tx=0x7f20d400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.926+0000 7f20da7fc700 1 -- 172.21.15.5:0/1859517466 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d400c9c0 con 0x7f20e4103310 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.926+0000 7f20da7fc700 1 -- 172.21.15.5:0/1859517466 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20d400cb60 con 0x7f20e4103310 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.926+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e41a73b0 con 0x7f20e4103310 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.926+0000 7f20da7fc700 1 -- 172.21.15.5:0/1859517466 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4018680 con 0x7f20e4103310 2024-05-14T15:35:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.926+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a7940 con 0x7f20e4103310 2024-05-14T15:35:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.929+0000 7f20da7fc700 1 -- 172.21.15.5:0/1859517466 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f20d4010430 con 0x7f20e4103310 2024-05-14T15:35:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.929+0000 7f20da7fc700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f20d00b2a70 0x7f20d00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.929+0000 7f20e959a700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f20d00b2a70 0x7f20d00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.930+0000 7f20da7fc700 1 -- 172.21.15.5:0/1859517466 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7f20d40e2bb0 con 0x7f20e4103310 2024-05-14T15:35:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.930+0000 7f20e959a700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f20d00b2a70 0x7f20d00b4f80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006010 tx=0x7f20e0005960 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 Processor -- start 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 -- start start 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29a6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a6d80 0x5614f29a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a0e10 0x5614f29a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614f28c2940 con 0x5614f2822e40 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614f2827da0 con 0x5614f29a6d80 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20fa212b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614f28be900 con 0x5614f29a0e10 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46332/0 (socket says 172.21.15.5:46332) 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 -- 172.21.15.5:0/433728088 learned_addr learned my addr 172.21.15.5:0/433728088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e8d99700 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a6d80 0x5614f29a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a0e10 0x5614f29a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 -- 172.21.15.5:0/433728088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a0e10 msgr2=0x5614f29a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a0e10 0x5614f29a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 -- 172.21.15.5:0/433728088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a6d80 msgr2=0x5614f29a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a6d80 0x5614f29a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.931+0000 7f20e959a700 1 -- 172.21.15.5:0/433728088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f280f8d0 con 0x5614f2822e40 2024-05-14T15:35:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.932+0000 7f20e959a700 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29a6840 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f20e0015000 tx=0x7f20e001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615763a95b43f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.932+0000 7f20cf7fe700 1 -- 172.21.15.5:0/433728088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0004a30 con 0x5614f2822e40 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.932+0000 7f20cf7fe700 1 -- 172.21.15.5:0/433728088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0004bd0 con 0x5614f2822e40 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.932+0000 7f20cf7fe700 1 -- 172.21.15.5:0/433728088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000f6c0 con 0x5614f2822e40 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 -- 172.21.15.5:0/433728088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 msgr2=0x5614f29a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29a6840 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f20e0015000 tx=0x7f20e001e000 comp rx=0 tx=0).stop 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 -- 172.21.15.5:0/433728088 shutdown_connections 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a0e10 0x5614f29a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29a6840 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 --2- 172.21.15.5:0/433728088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a6d80 0x5614f29a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 -- 172.21.15.5:0/433728088 >> 172.21.15.5:0/433728088 conn(0x5614f286a630 msgr2=0x5614f299ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 -- 172.21.15.5:0/433728088 shutdown_connections 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.933+0000 7f20fa212b80 1 -- 172.21.15.5:0/433728088 wait complete. 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 -- start start 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 Processor -- start 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 0x5614f29aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a6d80 0x5614f29aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614f29ad7c0 con 0x5614f2822e40 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614f287bed0 con 0x5614f29a0e10 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614f29ad940 con 0x5614f29a6d80 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.934+0000 7f20fa212b80 10 client.0 _ll_register_callbacks cb 0x5614f285a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:22.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e8d99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 0x5614f29aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e8d99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 0x5614f29aaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55594/0 (socket says 172.21.15.5:55594) 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e8d99700 1 -- 172.21.15.5:0/2361805125 learned_addr learned my addr 172.21.15.5:0/2361805125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e959a700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a6d80 0x5614f29aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 -- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 msgr2=0x5614f29aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 0x5614f29aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 -- 172.21.15.5:0/2361805125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 msgr2=0x5614f29aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 -- 172.21.15.5:0/2361805125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f2879760 con 0x5614f29a6d80 2024-05-14T15:35:22.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e8d99700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 0x5614f29aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e959a700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a6d80 0x5614f29aafd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009c10 tx=0x7f20dc00e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199355c4c97af5d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.935+0000 7f20ccff9700 1 -- 172.21.15.5:0/2361805125 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc01a070 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e0004bd0 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20ccff9700 1 -- 172.21.15.5:0/2361805125 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7f20dc003c00 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20ccff9700 1 -- 172.21.15.5:0/2361805125 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f20dc026050 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 10 client.27526 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5614f29c0d20 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20ccff9700 1 -- 172.21.15.5:0/2361805125 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0004bd0 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20ccff9700 1 -- 172.21.15.5:0/2361805125 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f20dc01e410 con 0x5614f29a6d80 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 10 client.27526 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 10 client.27526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 20 client.27526 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29c3680 0x5614f29c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 4 client.27526 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.936+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f28d0320 con 0x5614f29c3680 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.937+0000 7f20e959a700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29c3680 0x5614f29c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.937+0000 7f20ccff9700 10 client.27526 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.937+0000 7f20e959a700 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29c3680 0x5614f29c5af0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb4508ec55a3023 server_cookie=14adfb940207cba5 in_seq=0 out_seq=0 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20ccff9700 1 -- 172.21.15.5:0/2361805125 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614f28d0320 con 0x5614f29c3680 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20ccff9700 10 client.27526 handle_command_reply: tid=0 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 client.27526 shutdown 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a6d80 msgr2=0x5614f29aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a6d80 0x5614f29aafd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009c10 tx=0x7f20dc00e7d0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 shutdown_connections 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29c3680 0x5614f29c5af0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f29a6d80 0x5614f29aafd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f2822e40 0x5614f29aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20ccff9700 0 client.27526 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:22.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2361805125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f29a0e10 0x5614f29aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.938+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 >> 172.21.15.5:0/2361805125 conn(0x5614f286a630 msgr2=0x5614f299f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:22.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.939+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 shutdown_connections 2024-05-14T15:35:22.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.939+0000 7f20fa212b80 1 -- 172.21.15.5:0/2361805125 wait complete. 2024-05-14T15:35:22.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.939+0000 7f20fa212b80 20 client.27526 trim_cache size 0 max 16384 2024-05-14T15:35:22.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 Processor -- start 2024-05-14T15:35:22.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 -- start start 2024-05-14T15:35:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41b0f40 con 0x5614f29a0e10 2024-05-14T15:35:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41ae7d0 con 0x5614f2822e40 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20fa212b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41ac640 con 0x5614f286a630 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20e959a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20e8d99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20e959a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59376/0 (socket says 172.21.15.5:59376) 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.955+0000 7f20e959a700 1 -- 172.21.15.5:0/2514037115 learned_addr learned my addr 172.21.15.5:0/2514037115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e959a700 1 -- 172.21.15.5:0/2514037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 msgr2=0x5614f29bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e959a700 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e959a700 1 -- 172.21.15.5:0/2514037115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 msgr2=0x5614f29ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e959a700 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e959a700 1 -- 172.21.15.5:0/2514037115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f29ad290 con 0x5614f286a630 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e8d99700 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20e959a700 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29a08d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005850 tx=0x7f20e002fd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2137f09f3e0471 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20ceffd700 1 -- 172.21.15.5:0/2514037115 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000f560 con 0x5614f286a630 2024-05-14T15:35:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20ceffd700 1 -- 172.21.15.5:0/2514037115 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0040070 con 0x5614f286a630 2024-05-14T15:35:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.956+0000 7f20ceffd700 1 -- 172.21.15.5:0/2514037115 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e002f870 con 0x5614f286a630 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 -- 172.21.15.5:0/2514037115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 msgr2=0x5614f29a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29a08d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005850 tx=0x7f20e002fd50 comp rx=0 tx=0).stop 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 -- 172.21.15.5:0/2514037115 shutdown_connections 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29a08d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 --2- 172.21.15.5:0/2514037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 -- 172.21.15.5:0/2514037115 >> 172.21.15.5:0/2514037115 conn(0x5614f29ab120 msgr2=0x5614f29abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 -- 172.21.15.5:0/2514037115 shutdown_connections 2024-05-14T15:35:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 -- 172.21.15.5:0/2514037115 wait complete. 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 -- start start 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.957+0000 7f20fa212b80 1 Processor -- start 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29b8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41acc90 con 0x5614f29a0e10 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41ae100 con 0x5614f2822e40 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20e41ad360 con 0x5614f286a630 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20fa212b80 10 client.0 _ll_register_callbacks cb 0x5614f285a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e959a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e959a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55606/0 (socket says 172.21.15.5:55606) 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e959a700 1 -- 172.21.15.5:0/3819592070 learned_addr learned my addr 172.21.15.5:0/3819592070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e8d99700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e9d9b700 1 -- 172.21.15.5:0/3819592070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 msgr2=0x5614f29b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e9d9b700 1 -- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 msgr2=0x5614f29b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29b8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.958+0000 7f20e9d9b700 1 -- 172.21.15.5:0/3819592070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f2a2bfd0 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.959+0000 7f20e959a700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.959+0000 7f20e9d9b700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29b8eb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f20dc01e9d0 tx=0x7f20dc01e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b01be69130af8d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.959+0000 7f20ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.959+0000 7f20ccff9700 1 -- 172.21.15.5:0/3819592070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc01a040 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.959+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e0040070 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.959+0000 7f20ccff9700 1 -- 172.21.15.5:0/3819592070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9650+0+0 (secure 0 0 0) 0x7f20dc003a00 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20ccff9700 1 -- 172.21.15.5:0/3819592070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f20dc039050 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 10 client.17949 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5614f2a2d1d0 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20ccff9700 1 -- 172.21.15.5:0/3819592070 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0040070 con 0x5614f29a0e10 2024-05-14T15:35:22.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20ccff9700 1 -- 172.21.15.5:0/3819592070 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f20dc005570 con 0x5614f29a0e10 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 10 client.17949 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 20 client.17949 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29a6d80 0x5614f2a2f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 4 client.17949 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614f28ca3c0 con 0x5614f29a6d80 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.960+0000 7f20e959a700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29a6d80 0x5614f2a2f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.961+0000 7f20e959a700 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29a6d80 0x5614f2a2f880 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435161c51a8f5d98 server_cookie=83a1decbe1cedbe3 in_seq=0 out_seq=0 2024-05-14T15:35:22.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.961+0000 7f20ccff9700 10 client.17949 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:22.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20ccff9700 1 -- 172.21.15.5:0/3819592070 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614f28ca3c0 con 0x5614f29a6d80 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20ccff9700 10 client.17949 handle_command_reply: tid=0 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 client.17949 shutdown 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 msgr2=0x5614f29b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29b8eb0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f20dc01e9d0 tx=0x7f20dc01e610 comp rx=0 tx=0).stop 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 shutdown_connections 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5614f29a6d80 0x5614f2a2f880 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614f29a0e10 0x5614f29b8eb0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614f2822e40 0x5614f29b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 --2- 172.21.15.5:0/3819592070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614f286a630 0x5614f29b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 >> 172.21.15.5:0/3819592070 conn(0x5614f29ab120 msgr2=0x5614f29a4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20ccff9700 0 client.17949 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 shutdown_connections 2024-05-14T15:35:22.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 1 -- 172.21.15.5:0/3819592070 wait complete. 2024-05-14T15:35:22.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.962+0000 7f20fa212b80 20 client.17949 trim_cache size 0 max 16384 2024-05-14T15:35:22.968 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:35:22.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f20d00b2a70 msgr2=0x7f20d00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f20d00b2a70 0x7f20d00b4f80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006010 tx=0x7f20e0005960 comp rx=0 tx=0).stop 2024-05-14T15:35:22.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 msgr2=0x7f20e419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 0x7f20e419c2b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20d400e9c0 tx=0x7f20d400ecd0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 shutdown_connections 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f20d00b2a70 0x7f20d00b4f80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e41089f0 0x7f20e419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e41029c0 0x7f20e419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 --2- 172.21.15.5:0/1859517466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4103310 0x7f20e419c2b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 >> 172.21.15.5:0/1859517466 conn(0x7f20e40fe6f0 msgr2=0x7f20e4106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 shutdown_connections 2024-05-14T15:35:22.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:22.963+0000 7f20eb7fe700 1 -- 172.21.15.5:0/1859517466 wait complete. 2024-05-14T15:35:23.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 980570 9.50 MB/sec execute 404 sec latency 41.976 ms 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:35:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 -- 172.21.15.5:0/2389075254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc109ca0 msgr2=0x7fcdfc103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 --2- 172.21.15.5:0/2389075254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc109ca0 0x7fcdfc103840 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ca0 tx=0x7fcdf402a8d0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 -- 172.21.15.5:0/2389075254 shutdown_connections 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 --2- 172.21.15.5:0/2389075254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdfc10a140 0x7fcdfc103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 --2- 172.21.15.5:0/2389075254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 0x7fcdfc1107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 --2- 172.21.15.5:0/2389075254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc109ca0 0x7fcdfc103840 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 -- 172.21.15.5:0/2389075254 >> 172.21.15.5:0/2389075254 conn(0x7fcdfc0ff340 msgr2=0x7fcdfc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 -- 172.21.15.5:0/2389075254 shutdown_connections 2024-05-14T15:35:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.753+0000 7fce04135640 1 -- 172.21.15.5:0/2389075254 wait complete. 2024-05-14T15:35:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 Processor -- start 2024-05-14T15:35:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 -- start start 2024-05-14T15:35:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 0x7fcdfc1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdfc10a140 0x7fcdfc1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc19a8a0 0x7fcdfc19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc113f60 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc113de0 con 0x7fcdfc1043f0 2024-05-14T15:35:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.755+0000 7fce04135640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1140e0 con 0x7fcdfc10a140 2024-05-14T15:35:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 0x7fcdfc1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 0x7fcdfc1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55620/0 (socket says 172.21.15.5:55620) 2024-05-14T15:35:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce01ed1640 1 -- 172.21.15.5:0/3153314172 learned_addr learned my addr 172.21.15.5:0/3153314172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce026d2640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc19a8a0 0x7fcdfc19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce026d2640 1 -- 172.21.15.5:0/3153314172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdfc10a140 msgr2=0x7fcdfc1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce026d2640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdfc10a140 0x7fcdfc1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce026d2640 1 -- 172.21.15.5:0/3153314172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 msgr2=0x7fcdfc1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce026d2640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 0x7fcdfc1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.756+0000 7fce026d2640 1 -- 172.21.15.5:0/3153314172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc19b430 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.757+0000 7fce026d2640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc19a8a0 0x7fcdfc19acb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800b3e0 tx=0x7fcdf800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.758+0000 7fcdf2ffd640 1 -- 172.21.15.5:0/3153314172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8015020 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.758+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc1a7510 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.758+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc1a7910 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.758+0000 7fcdf2ffd640 1 -- 172.21.15.5:0/3153314172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf8011ce0 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.758+0000 7fcdf2ffd640 1 -- 172.21.15.5:0/3153314172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8010e00 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.762+0000 7fce0afe0740 1 Processor -- start 2024-05-14T15:35:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.763+0000 7fcdf2ffd640 1 -- 172.21.15.5:0/3153314172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcdf8018020 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 -- start start 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa84660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec3a9e95d0 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ec3a9a2890 con 0x55ec3a6e31b0 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce0afe0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec3a9e5590 con 0x55ec3a6c24d0 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa84120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46398/0 (socket says 172.21.15.5:46398) 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce01ed1640 1 -- 172.21.15.5:0/957508965 learned_addr learned my addr 172.21.15.5:0/957508965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.764+0000 7fce016d0640 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce016d0640 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa84660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce01ed1640 1 -- 172.21.15.5:0/957508965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6c24d0 msgr2=0x55ec3aa84660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce01ed1640 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa84660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce01ed1640 1 -- 172.21.15.5:0/957508965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6e31b0 msgr2=0x55ec3aa83be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce01ed1640 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa83be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce01ed1640 1 -- 172.21.15.5:0/957508965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3a6fb210 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.765+0000 7fce01ed1640 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa84120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ca0 tx=0x7fcdf4031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714abed77f452d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.766+0000 7fcdf2ffd640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcde80b2b60 0x7fcde80b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.766+0000 7fcde7fff640 1 -- 172.21.15.5:0/957508965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4031460 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.766+0000 7fcde7fff640 1 -- 172.21.15.5:0/957508965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf4036040 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.766+0000 7fcde7fff640 1 -- 172.21.15.5:0/957508965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4040c80 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.766+0000 7fcdf2ffd640 1 -- 172.21.15.5:0/3153314172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9562+0+0 (secure 0 0 0) 0x7fcdf80e1fb0 con 0x7fcdfc19a8a0 2024-05-14T15:35:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.768+0000 7fce0afe0740 1 -- 172.21.15.5:0/957508965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 msgr2=0x55ec3aa84120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.768+0000 7fce0afe0740 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa84120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ca0 tx=0x7fcdf4031b00 comp rx=0 tx=0).stop 2024-05-14T15:35:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.768+0000 7fce016d0640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcde80b2b60 0x7fcde80b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.770+0000 7fce016d0640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcde80b2b60 0x7fcde80b5070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55ec3aa80d00 tx=0x7fcdec005a60 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 -- 172.21.15.5:0/957508965 shutdown_connections 2024-05-14T15:35:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa84660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa84120 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 --2- 172.21.15.5:0/957508965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa83be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 -- 172.21.15.5:0/957508965 >> 172.21.15.5:0/957508965 conn(0x55ec3a6d7300 msgr2=0x55ec3aa80340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 -- 172.21.15.5:0/957508965 shutdown_connections 2024-05-14T15:35:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.773+0000 7fce0afe0740 1 -- 172.21.15.5:0/957508965 wait complete. 2024-05-14T15:35:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.774+0000 7fce0afe0740 1 -- start start 2024-05-14T15:35:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.774+0000 7fce0afe0740 1 Processor -- start 2024-05-14T15:35:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa8e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa9a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec3aa90cb0 con 0x55ec3a6c24d0 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ec3a9a6370 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec3aa90e30 con 0x55ec3a6e31b0 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.775+0000 7fce0afe0740 10 client.0 _ll_register_callbacks cb 0x55ec3a6faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46406/0 (socket says 172.21.15.5:46406) 2024-05-14T15:35:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 -- 172.21.15.5:0/1898014778 learned_addr learned my addr 172.21.15.5:0/1898014778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 -- 172.21.15.5:0/1898014778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 msgr2=0x55ec3aa9a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa9a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 -- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 msgr2=0x55ec3aa8e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa8e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.776+0000 7fce016d0640 1 -- 172.21.15.5:0/1898014778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3a7c7620 con 0x55ec3a6c24d0 2024-05-14T15:35:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.777+0000 7fce016d0640 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8e700 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcdec012b50 tx=0x7fcdec005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f9557eb2d41fd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.777+0000 7fcde57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.777+0000 7fcde57fa640 1 -- 172.21.15.5:0/1898014778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec018070 con 0x55ec3a6c24d0 2024-05-14T15:35:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.777+0000 7fcde57fa640 1 -- 172.21.15.5:0/1898014778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9562+0+0 (secure 0 0 0) 0x7fcdec01dac0 con 0x55ec3a6c24d0 2024-05-14T15:35:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.779+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf4036040 con 0x55ec3a6c24d0 2024-05-14T15:35:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.779+0000 7fcde57fa640 1 -- 172.21.15.5:0/1898014778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcdec01dcc0 con 0x55ec3a6c24d0 2024-05-14T15:35:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.779+0000 7fce0afe0740 10 client.17961 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.779+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec3aaa5910 con 0x55ec3a6c24d0 2024-05-14T15:35:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.780+0000 7fcde57fa640 1 -- 172.21.15.5:0/1898014778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf4036040 con 0x55ec3a6c24d0 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fcde57fa640 1 -- 172.21.15.5:0/1898014778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fcdec00f040 con 0x55ec3a6c24d0 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce0afe0740 10 client.17961 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce0afe0740 10 client.17961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce0afe0740 20 client.17961 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce0afe0740 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6a1d80 0x55ec3aaa8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce0afe0740 4 client.17961 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec3a9f71e0 con 0x55ec3a6a1d80 2024-05-14T15:35:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.781+0000 7fce01ed1640 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6a1d80 0x55ec3aaa8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.784+0000 7fce01ed1640 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6a1d80 0x55ec3aaa8f00 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4406dfc52e7fca52 server_cookie=9f583bbb3614a5b1 in_seq=0 out_seq=0 2024-05-14T15:35:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.784+0000 7fcde57fa640 10 client.17961 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.786+0000 7fcde57fa640 1 -- 172.21.15.5:0/1898014778 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec3a9f71e0 con 0x55ec3a6a1d80 2024-05-14T15:35:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.786+0000 7fcde57fa640 10 client.17961 handle_command_reply: tid=0 2024-05-14T15:35:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.787+0000 7fce0afe0740 1 client.17961 shutdown 2024-05-14T15:35:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.787+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 msgr2=0x55ec3aa8e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.787+0000 7fce0afe0740 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8e700 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcdec012b50 tx=0x7fcdec005ad0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.788+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 shutdown_connections 2024-05-14T15:35:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.788+0000 7fce0afe0740 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6a1d80 0x55ec3aaa8f00 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.788+0000 7fce0afe0740 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa9a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.788+0000 7fce0afe0740 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8e700 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.788+0000 7fce0afe0740 1 --2- 172.21.15.5:0/1898014778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa8e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.788+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 >> 172.21.15.5:0/1898014778 conn(0x55ec3a6d7300 msgr2=0x55ec3aa83b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.789+0000 7fcde57fa640 0 client.17961 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.790+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 shutdown_connections 2024-05-14T15:35:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.790+0000 7fce0afe0740 1 -- 172.21.15.5:0/1898014778 wait complete. 2024-05-14T15:35:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.790+0000 7fce0afe0740 20 client.17961 trim_cache size 0 max 16384 2024-05-14T15:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.819+0000 7fce0afe0740 1 Processor -- start 2024-05-14T15:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.819+0000 7fce0afe0740 1 -- start start 2024-05-14T15:35:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.819+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa88840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.819+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.819+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa99dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce0afe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0012a10 con 0x55ec3a6c24d0 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce0afe0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcde00102e0 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce0afe0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcde000e150 con 0x55ec3a6e31b0 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa88840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46420/0 (socket says 172.21.15.5:46420) 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 -- 172.21.15.5:0/752507956 learned_addr learned my addr 172.21.15.5:0/752507956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa88840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55640/0 (socket says 172.21.15.5:55640) 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce026d2640 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa99dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 -- 172.21.15.5:0/752507956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 msgr2=0x55ec3aa99dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa99dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 -- 172.21.15.5:0/752507956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 msgr2=0x55ec3aa88840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa88840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 -- 172.21.15.5:0/752507956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3aa90850 con 0x55ec3a6c24d0 2024-05-14T15:35:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce01ed1640 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa88840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:23.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.820+0000 7fce016d0640 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8cf90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584c7e54aac364be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.821+0000 7fcde77fe640 1 -- 172.21.15.5:0/752507956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec018040 con 0x55ec3a6c24d0 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.821+0000 7fcde77fe640 1 -- 172.21.15.5:0/752507956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec01de60 con 0x55ec3a6c24d0 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.821+0000 7fcde77fe640 1 -- 172.21.15.5:0/752507956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec01e710 con 0x55ec3a6c24d0 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.821+0000 7fce0afe0740 1 -- 172.21.15.5:0/752507956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 msgr2=0x55ec3aa8cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.821+0000 7fce0afe0740 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8cf90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec012700 comp rx=0 tx=0).stop 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 -- 172.21.15.5:0/752507956 shutdown_connections 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6e31b0 0x55ec3aa99dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa8cf90 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 --2- 172.21.15.5:0/752507956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa88840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 -- 172.21.15.5:0/752507956 >> 172.21.15.5:0/752507956 conn(0x55ec3a6d7300 msgr2=0x55ec3aa8c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 -- 172.21.15.5:0/752507956 shutdown_connections 2024-05-14T15:35:23.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 -- 172.21.15.5:0/752507956 wait complete. 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 -- start start 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.822+0000 7fce0afe0740 1 Processor -- start 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa91400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa91940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6e31b0 0x55ec3ab0f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcde000e7a0 con 0x55ec3a6e31b0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcde000fc10 con 0x55ec3a6c24d0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcde000ee70 con 0x55ec3a6ba2f0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce0afe0740 10 client.0 _ll_register_callbacks cb 0x7fcdec0127d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa91400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce016d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa91940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce01ed1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa91400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:59432/0 (socket says 172.21.15.5:59432) 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6e31b0 0x55ec3ab0f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce01ed1640 1 -- 172.21.15.5:0/4277041536 learned_addr learned my addr 172.21.15.5:0/4277041536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce016d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa91940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:55648/0 (socket says 172.21.15.5:55648) 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 -- 172.21.15.5:0/4277041536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 msgr2=0x55ec3aa91400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa91400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 -- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 msgr2=0x55ec3aa91940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa91940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 -- 172.21.15.5:0/4277041536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3ab0f7e0 con 0x55ec3a6e31b0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce016d0640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa91940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce01ed1640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa91400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fce026d2640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6e31b0 0x55ec3ab0f2a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80df3a0 tx=0x7fcdf80a8b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88ec8dc6a12b655 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fcde57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.823+0000 7fcde57fa640 1 -- 172.21.15.5:0/4277041536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80ad2e0 con 0x55ec3a6e31b0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fcde57fa640 1 -- 172.21.15.5:0/4277041536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9562+0+0 (secure 0 0 0) 0x7fcdf80090f0 con 0x55ec3a6e31b0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdec01de60 con 0x55ec3a6e31b0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fcde57fa640 1 -- 172.21.15.5:0/4277041536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcdf80b1d90 con 0x55ec3a6e31b0 2024-05-14T15:35:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 10 client.17967 fetch_fsmap learned FSMap version 32 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec3ab10970 con 0x55ec3a6e31b0 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fcde57fa640 1 -- 172.21.15.5:0/4277041536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec01de60 con 0x55ec3a6e31b0 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fcde57fa640 1 -- 172.21.15.5:0/4277041536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fcdf80a82b0 con 0x55ec3a6e31b0 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 10 client.17967 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 10 client.17967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 20 client.17967 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6771c0 0x55ec3ab13100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 4 client.17967 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.824+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec3a9f1050 con 0x55ec3a6771c0 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.825+0000 7fce01ed1640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6771c0 0x55ec3ab13100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.825+0000 7fcde57fa640 10 client.17967 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:23.825+0000 7fce01ed1640 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6771c0 0x55ec3ab13100 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2fcf693cf5019a2 server_cookie=7df7a3f1227bc1c4 in_seq=0 out_seq=0 2024-05-14T15:35:24.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 981234 9.48 MB/sec execute 405 sec latency 162.800 ms 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fcde57fa640 1 -- 172.21.15.5:0/4277041536 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec3a9f1050 con 0x55ec3a6771c0 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fcde57fa640 10 client.17967 handle_command_reply: tid=0 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 client.17967 shutdown 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6e31b0 msgr2=0x55ec3ab0f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6e31b0 0x55ec3ab0f2a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80df3a0 tx=0x7fcdf80a8b00 comp rx=0 tx=0).stop 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 shutdown_connections 2024-05-14T15:35:24.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55ec3a6771c0 0x55ec3ab13100 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec3a6e31b0 0x55ec3ab0f2a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fcde57fa640 0 client.17967 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ec3a6c24d0 0x55ec3aa91940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 --2- 172.21.15.5:0/4277041536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec3a6ba2f0 0x55ec3aa91400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 >> 172.21.15.5:0/4277041536 conn(0x55ec3a6d7300 msgr2=0x55ec3aa8f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.184+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 shutdown_connections 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce0afe0740 1 -- 172.21.15.5:0/4277041536 wait complete. 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce0afe0740 20 client.17967 trim_cache size 0 max 16384 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:35:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:35:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:35:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcde80b2b60 msgr2=0x7fcde80b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcde80b2b60 0x7fcde80b5070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55ec3aa80d00 tx=0x7fcdec005a60 comp rx=0 tx=0).stop 2024-05-14T15:35:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc19a8a0 msgr2=0x7fcdfc19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc19a8a0 0x7fcdfc19acb0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800b3e0 tx=0x7fcdf800b8a0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 shutdown_connections 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcde80b2b60 0x7fcde80b5070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdfc10a140 0x7fcdfc1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdfc19a8a0 0x7fcdfc19acb0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 --2- 172.21.15.5:0/3153314172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdfc1043f0 0x7fcdfc1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 >> 172.21.15.5:0/3153314172 conn(0x7fcdfc0ff340 msgr2=0x7fcdfc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 shutdown_connections 2024-05-14T15:35:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:24.185+0000 7fce04135640 1 -- 172.21.15.5:0/3153314172 wait complete. 2024-05-14T15:35:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:23 smithi144 ceph-mon[35650]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-14T15:35:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:23 smithi144 ceph-mon[35650]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-14T15:35:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:23 smithi144 ceph-mon[35650]: pgmap v331: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 269 op/s 2024-05-14T15:35:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:23 smithi144 ceph-mon[35650]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T15:35:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:23 smithi193 ceph-mon[36001]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-14T15:35:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:23 smithi193 ceph-mon[36001]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-14T15:35:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:23 smithi193 ceph-mon[36001]: pgmap v331: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 269 op/s 2024-05-14T15:35:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:23 smithi193 ceph-mon[36001]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T15:35:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:23 smithi005 ceph-mon[31559]: Adjusting osd_memory_target on smithi005 to 177.2M 2024-05-14T15:35:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:23 smithi005 ceph-mon[31559]: Unable to set osd_memory_target on smithi005 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-14T15:35:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:23 smithi005 ceph-mon[31559]: pgmap v331: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 269 op/s 2024-05-14T15:35:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:23 smithi005 ceph-mon[31559]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T15:35:24.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:35:24.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:35:25.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:24 smithi144 ceph-mon[35650]: daemon mds.e restarted 2024-05-14T15:35:25.048 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:35:24 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setuser ceph since I am not root 2024-05-14T15:35:25.048 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:35:24 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setgroup ceph since I am not root 2024-05-14T15:35:25.048 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:35:24 smithi144 ceph-mds[65770]: -- 172.21.15.144:0/309180038 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647d9047a40 con 0x5647d908f180 2024-05-14T15:35:25.048 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:35:24 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: starting mds.e at 2024-05-14T15:35:25.125 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:25.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 983318 9.49 MB/sec execute 406 sec latency 180.152 ms 2024-05-14T15:35:25.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:24 smithi005 ceph-mon[31559]: daemon mds.e restarted 2024-05-14T15:35:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:24 smithi193 ceph-mon[36001]: daemon mds.e restarted 2024-05-14T15:35:26.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 984555 9.48 MB/sec execute 407 sec latency 50.514 ms 2024-05-14T15:35:26.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:25 smithi005 ceph-mon[31559]: pgmap v333: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-05-14T15:35:26.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:25 smithi005 ceph-mon[31559]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T15:35:26.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:25 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6834/3068356254,v1:172.21.15.144:6835/3068356254] up:boot 2024-05-14T15:35:26.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:25 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:35:26.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:25 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:35:26.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:25 smithi005 ceph-mon[31559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:35:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:25 smithi144 ceph-mon[35650]: pgmap v333: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-05-14T15:35:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:25 smithi144 ceph-mon[35650]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T15:35:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:25 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6834/3068356254,v1:172.21.15.144:6835/3068356254] up:boot 2024-05-14T15:35:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:25 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:35:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:25 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:35:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:25 smithi144 ceph-mon[35650]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:25 smithi193 ceph-mon[36001]: pgmap v333: 129 pgs: 129 active+clean; 92 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-05-14T15:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:25 smithi193 ceph-mon[36001]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T15:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:25 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6834/3068356254,v1:172.21.15.144:6835/3068356254] up:boot 2024-05-14T15:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:25 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T15:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:25 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:25 smithi193 ceph-mon[36001]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 -- 172.21.15.5:0/840360371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 msgr2=0x7f36f0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 --2- 172.21.15.5:0/840360371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f0108200 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f36d80099c0 tx=0x7f36d8009cd0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 -- 172.21.15.5:0/840360371 shutdown_connections 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 --2- 172.21.15.5:0/840360371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f0108740 0x7f36f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 --2- 172.21.15.5:0/840360371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f01034c0 0x7f36f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 --2- 172.21.15.5:0/840360371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f0108200 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 -- 172.21.15.5:0/840360371 >> 172.21.15.5:0/840360371 conn(0x7f36f00fedc0 msgr2=0x7f36f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 -- 172.21.15.5:0/840360371 shutdown_connections 2024-05-14T15:35:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.915+0000 7f36f576b700 1 -- 172.21.15.5:0/840360371 wait complete. 2024-05-14T15:35:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.916+0000 7f36f576b700 1 Processor -- start 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.916+0000 7f36f576b700 1 -- start start 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.916+0000 7f36f576b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 0x7f36f019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.916+0000 7f36f576b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36f576b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0108740 0x7f36f0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36f576b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0113350 con 0x7f36f0107df0 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36f576b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01131d0 con 0x7f36f01034c0 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36f576b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01134d0 con 0x7f36f0108740 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 0x7f36f019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36e67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 0x7f36f019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57064/0 (socket says 172.21.15.5:57064) 2024-05-14T15:35:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36e67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43634/0 (socket says 172.21.15.5:43634) 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 -- 172.21.15.5:0/1240674574 learned_addr learned my addr 172.21.15.5:0/1240674574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0108740 0x7f36f0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 -- 172.21.15.5:0/1240674574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0108740 msgr2=0x7f36f0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0108740 0x7f36f0199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 -- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 msgr2=0x7f36f019ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f019ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 -- 172.21.15.5:0/1240674574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f019a280 con 0x7f36f01034c0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0108740 0x7f36f0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36eeffd700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 0x7f36f019f9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36e000ef10 tx=0x7f36e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.917+0000 7f36ecff9700 1 -- 172.21.15.5:0/1240674574 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e000c9c0 con 0x7f36f01034c0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.918+0000 7f36ecff9700 1 -- 172.21.15.5:0/1240674574 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e0019070 con 0x7f36f01034c0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.918+0000 7f36ecff9700 1 -- 172.21.15.5:0/1240674574 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0014480 con 0x7f36f01034c0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.918+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f019a510 con 0x7f36f01034c0 2024-05-14T15:35:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.918+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f01a6770 con 0x7f36f01034c0 2024-05-14T15:35:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.920+0000 7f36ecff9700 1 -- 172.21.15.5:0/1240674574 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f36e0012050 con 0x7f36f01034c0 2024-05-14T15:35:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.922+0000 7f36ecff9700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36dc0b2a70 0x7f36dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.922+0000 7f36e67fc700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36dc0b2a70 0x7f36dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.922+0000 7f36ecff9700 1 -- 172.21.15.5:0/1240674574 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f36e00e28f0 con 0x7f36f01034c0 2024-05-14T15:35:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.922+0000 7f370417fb80 1 Processor -- start 2024-05-14T15:35:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.922+0000 7f370417fb80 1 -- start start 2024-05-14T15:35:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.922+0000 7f36e67fc700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36dc0b2a70 0x7f36dc0b4f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009990 tx=0x7f36d801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f370417fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f370417fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc3db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f370417fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3e0a0 0x556accc3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f370417fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556accb5d9c0 con 0x556accc43f60 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f370417fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556accac2e20 con 0x556accc3e0a0 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f370417fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556accb59980 con 0x556accabdec0 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc3db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc3db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43642/0 (socket says 172.21.15.5:43642) 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 -- 172.21.15.5:0/2709501302 learned_addr learned my addr 172.21.15.5:0/2709501302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3e0a0 0x556accc3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36eeffd700 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 -- 172.21.15.5:0/2709501302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 msgr2=0x556accc43a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc43a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 -- 172.21.15.5:0/2709501302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3e0a0 msgr2=0x556accc3e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3e0a0 0x556accc3e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 -- 172.21.15.5:0/2709501302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556accaaa950 con 0x556accc43f60 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3e0a0 0x556accc3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e67fc700 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc3db60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f36d800a010 tx=0x7f36d801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b0fa4d7d3e5ee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.923+0000 7f36e57fa700 1 -- 172.21.15.5:0/2709501302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8034070 con 0x556accc43f60 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.924+0000 7f36e57fa700 1 -- 172.21.15.5:0/2709501302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d8003c70 con 0x556accc43f60 2024-05-14T15:35:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.924+0000 7f36e57fa700 1 -- 172.21.15.5:0/2709501302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d802f470 con 0x556accc43f60 2024-05-14T15:35:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.924+0000 7f370417fb80 1 -- 172.21.15.5:0/2709501302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 msgr2=0x556accc3db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.924+0000 7f370417fb80 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc3db60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f36d800a010 tx=0x7f36d801f000 comp rx=0 tx=0).stop 2024-05-14T15:35:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 -- 172.21.15.5:0/2709501302 shutdown_connections 2024-05-14T15:35:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3e0a0 0x556accc3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc43a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 --2- 172.21.15.5:0/2709501302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc3db60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 -- 172.21.15.5:0/2709501302 >> 172.21.15.5:0/2709501302 conn(0x556accb056b0 msgr2=0x556accc36a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 -- 172.21.15.5:0/2709501302 shutdown_connections 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 -- 172.21.15.5:0/2709501302 wait complete. 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 -- start start 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 Processor -- start 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc47810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc47d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc48290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.925+0000 7f370417fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556accc4aa00 con 0x556accc43f60 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f370417fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556accb16f50 con 0x556accabdec0 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f370417fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556accc4ab80 con 0x556accc3e0a0 2024-05-14T15:35:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f370417fb80 10 client.0 _ll_register_callbacks cb 0x556accaf59d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc47810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc48290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc47810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57080/0 (socket says 172.21.15.5:57080) 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f36eeffd700 1 -- 172.21.15.5:0/3936152641 learned_addr learned my addr 172.21.15.5:0/3936152641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.926+0000 7f36e67fc700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc47d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36ef7fe700 1 -- 172.21.15.5:0/3936152641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 msgr2=0x556accc47d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc47d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36ef7fe700 1 -- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 msgr2=0x556accc47810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc47810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36ef7fe700 1 -- 172.21.15.5:0/3936152641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556accb147e0 con 0x556accc43f60 2024-05-14T15:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36eeffd700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc47810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc48290 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f36e8005940 tx=0x7f36e8009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121929071b44c53d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36d67fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36d67fc700 1 -- 172.21.15.5:0/3936152641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004a30 con 0x556accc43f60 2024-05-14T15:35:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36d67fc700 1 -- 172.21.15.5:0/3936152641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f36e8004bd0 con 0x556accc43f60 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d8003c70 con 0x556accc43f60 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.927+0000 7f36d67fc700 1 -- 172.21.15.5:0/3936152641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f36e800f7b0 con 0x556accc43f60 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 10 client.17985 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556accc3b9b0 con 0x556accc43f60 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f36d67fc700 1 -- 172.21.15.5:0/3936152641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e8029070 con 0x556accc43f60 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f36d67fc700 1 -- 172.21.15.5:0/3936152641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f36e8005460 con 0x556accc43f60 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 10 client.17985 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 20 client.17985 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc3ce70 0x556accc3d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 4 client.17985 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556accb6b3a0 con 0x556accc3ce70 2024-05-14T15:35:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.928+0000 7f36eeffd700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc3ce70 0x556accc3d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.929+0000 7f36d67fc700 10 client.17985 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.929+0000 7f36eeffd700 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc3ce70 0x556accc3d280 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ce380a524c0b3b server_cookie=7bac66eef883d530 in_seq=0 out_seq=0 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.931+0000 7f36d67fc700 1 -- 172.21.15.5:0/3936152641 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556accb6b3a0 con 0x556accc3ce70 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.931+0000 7f36d67fc700 10 client.17985 handle_command_reply: tid=0 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.931+0000 7f370417fb80 1 client.17985 shutdown 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.931+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 msgr2=0x556accc48290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.931+0000 7f370417fb80 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc48290 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f36e8005940 tx=0x7f36e8009e00 comp rx=0 tx=0).stop 2024-05-14T15:35:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 shutdown_connections 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc3ce70 0x556accc3d280 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc43f60 0x556accc48290 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc47810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f36d67fc700 0 client.17985 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 --2- 172.21.15.5:0/3936152641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc47d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 >> 172.21.15.5:0/3936152641 conn(0x556accb056b0 msgr2=0x556accc3a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 shutdown_connections 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 1 -- 172.21.15.5:0/3936152641 wait complete. 2024-05-14T15:35:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.932+0000 7f370417fb80 20 client.17985 trim_cache size 0 max 16384 2024-05-14T15:35:26.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 Processor -- start 2024-05-14T15:35:26.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 -- start start 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc53e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3ce70 0x556accc480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3e0a0 0x556accc46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01b0c90 con 0x556accc3e0a0 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ae520 con 0x556accc3ce70 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f370417fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ac370 con 0x556accabdec0 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc53e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3e0a0 0x556accc46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc53e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:40342/0 (socket says 172.21.15.5:40342) 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f36eeffd700 1 -- 172.21.15.5:0/2576931876 learned_addr learned my addr 172.21.15.5:0/2576931876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.975+0000 7f36e67fc700 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3ce70 0x556accc480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36eeffd700 1 -- 172.21.15.5:0/2576931876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3ce70 msgr2=0x556accc480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36eeffd700 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3ce70 0x556accc480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36eeffd700 1 -- 172.21.15.5:0/2576931876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3e0a0 msgr2=0x556accc46810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36eeffd700 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3e0a0 0x556accc46810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36eeffd700 1 -- 172.21.15.5:0/2576931876 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556accc32a60 con 0x556accabdec0 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3e0a0 0x556accc46810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36eeffd700 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc53e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f36e00a6a20 tx=0x7f36e0009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc1210dd1bde0d3a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36e4ff9700 1 -- 172.21.15.5:0/2576931876 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00a9eb0 con 0x556accabdec0 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36e4ff9700 1 -- 172.21.15.5:0/2576931876 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e00a97b0 con 0x556accabdec0 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.976+0000 7f36e4ff9700 1 -- 172.21.15.5:0/2576931876 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00b2180 con 0x556accabdec0 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 -- 172.21.15.5:0/2576931876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 msgr2=0x556accc53e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc53e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f36e00a6a20 tx=0x7f36e0009eb0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 -- 172.21.15.5:0/2576931876 shutdown_connections 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3e0a0 0x556accc46810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accabdec0 0x556accc53e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 --2- 172.21.15.5:0/2576931876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accc3ce70 0x556accc480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 -- 172.21.15.5:0/2576931876 >> 172.21.15.5:0/2576931876 conn(0x556accb056b0 msgr2=0x556accc46100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 -- 172.21.15.5:0/2576931876 shutdown_connections 2024-05-14T15:35:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.977+0000 7f370417fb80 1 -- 172.21.15.5:0/2576931876 wait complete. 2024-05-14T15:35:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 -- start start 2024-05-14T15:35:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 Processor -- start 2024-05-14T15:35:26.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3ce70 0x556accc59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc5a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ac9e0 con 0x556accc3ce70 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ade50 con 0x556accabdec0 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ad0b0 con 0x556accc3e0a0 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f36e67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3ce70 0x556accc59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f370417fb80 10 client.0 _ll_register_callbacks cb 0x556accaf59d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc5a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f36eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc5d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57094/0 (socket says 172.21.15.5:57094) 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc5a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:40356/0 (socket says 172.21.15.5:40356) 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.978+0000 7f36eeffd700 1 -- 172.21.15.5:0/274819643 learned_addr learned my addr 172.21.15.5:0/274819643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36ef7fe700 1 -- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 msgr2=0x556accc5d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc5d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36ef7fe700 1 -- 172.21.15.5:0/274819643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3ce70 msgr2=0x556accc59d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3ce70 0x556accc59d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36ef7fe700 1 -- 172.21.15.5:0/274819643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556accc5a7d0 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36ef7fe700 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc5a290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36e801c040 tx=0x7f36e8005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e87e7755ebc769f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36d67fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36d67fc700 1 -- 172.21.15.5:0/274819643 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8032060 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36d67fc700 1 -- 172.21.15.5:0/274819643 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f36e8019dd0 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e00a97b0 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36d67fc700 1 -- 172.21.15.5:0/274819643 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f36e8039050 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f370417fb80 10 client.27565 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556acccc0fa0 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.979+0000 7f36d67fc700 1 -- 172.21.15.5:0/274819643 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e00a97b0 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f36d67fc700 1 -- 172.21.15.5:0/274819643 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f36e8003680 con 0x556accc3e0a0 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f370417fb80 10 client.27565 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f370417fb80 10 client.27565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:26.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f370417fb80 20 client.27565 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f370417fb80 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc43f60 0x556acccc3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f370417fb80 4 client.27565 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556accb65440 con 0x556accc43f60 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f36eeffd700 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc43f60 0x556acccc3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f36d67fc700 10 client.27565 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.980+0000 7f36eeffd700 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc43f60 0x556acccc3600 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b2eb8d6a449802 server_cookie=ee62c960759e5e0e in_seq=0 out_seq=0 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.982+0000 7f36d67fc700 1 -- 172.21.15.5:0/274819643 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556accb65440 con 0x556accc43f60 2024-05-14T15:35:26.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.982+0000 7f36d67fc700 10 client.27565 handle_command_reply: tid=0 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.982+0000 7f370417fb80 1 client.27565 shutdown 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 msgr2=0x556accc5a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc5a290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36e801c040 tx=0x7f36e8005980 comp rx=0 tx=0).stop 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 shutdown_connections 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x556accc43f60 0x556acccc3600 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556accc3e0a0 0x556accc5a290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556accabdec0 0x556accc5d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 --2- 172.21.15.5:0/274819643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556accc3ce70 0x556accc59d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 >> 172.21.15.5:0/274819643 conn(0x556accb056b0 msgr2=0x556accc30d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f36d67fc700 0 client.27565 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 shutdown_connections 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 1 -- 172.21.15.5:0/274819643 wait complete. 2024-05-14T15:35:26.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.983+0000 7f370417fb80 20 client.27565 trim_cache size 0 max 16384 2024-05-14T15:35:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:35:26.989 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:35:26.989 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8ee16471-e7b4-46fb-88f1-592aaa09d73b", 2024-05-14T15:35:26.989 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:35:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36dc0b2a70 msgr2=0x7f36dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36dc0b2a70 0x7f36dc0b4f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009990 tx=0x7f36d801a040 comp rx=0 tx=0).stop 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 msgr2=0x7f36f019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 0x7f36f019f9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36e000ef10 tx=0x7f36e000c3a0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 shutdown_connections 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f36dc0b2a70 0x7f36dc0b4f80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36f0108740 0x7f36f0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36f01034c0 0x7f36f019f9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 --2- 172.21.15.5:0/1240674574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f0107df0 0x7f36f019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 >> 172.21.15.5:0/1240674574 conn(0x7f36f00fedc0 msgr2=0x7f36f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 shutdown_connections 2024-05-14T15:35:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:26.984+0000 7f36f576b700 1 -- 172.21.15.5:0/1240674574 wait complete. 2024-05-14T15:35:27.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 986443 9.48 MB/sec execute 408 sec latency 21.115 ms 2024-05-14T15:35:27.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:35:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:35:28.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:27 smithi005 ceph-mon[31559]: pgmap v335: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-14T15:35:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:27 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:35:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:27 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:27 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:35:28.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:27 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:28.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 987941 9.48 MB/sec execute 409 sec latency 32.119 ms 2024-05-14T15:35:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:27 smithi144 ceph-mon[35650]: pgmap v335: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-14T15:35:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:27 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:35:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:27 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:27 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:35:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:27 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:27 smithi193 ceph-mon[36001]: pgmap v335: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-05-14T15:35:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:27 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:35:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:27 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:27 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:35:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:27 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:35:29.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 989427 9.47 MB/sec execute 410 sec latency 22.667 ms 2024-05-14T15:35:29.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 -- 172.21.15.5:0/2264000426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec108f00 msgr2=0x7f78ec109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2264000426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec108f00 0x7f78ec109310 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f78d40099c0 tx=0x7f78d4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 -- 172.21.15.5:0/2264000426 shutdown_connections 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2264000426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109850 0x7f78ec1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2264000426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2264000426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec108f00 0x7f78ec109310 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 -- 172.21.15.5:0/2264000426 >> 172.21.15.5:0/2264000426 conn(0x7f78ec0ffe80 msgr2=0x7f78ec1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 -- 172.21.15.5:0/2264000426 shutdown_connections 2024-05-14T15:35:29.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.735+0000 7f78f09e3700 1 -- 172.21.15.5:0/2264000426 wait complete. 2024-05-14T15:35:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 Processor -- start 2024-05-14T15:35:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 -- start start 2024-05-14T15:35:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec108f00 0x7f78ec19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec109850 0x7f78ec19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec114460 con 0x7f78ec109850 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1142e0 con 0x7f78ec108f00 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.736+0000 7f78f09e3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1145e0 con 0x7f78ec1045d0 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:40370/0 (socket says 172.21.15.5:40370) 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 -- 172.21.15.5:0/2154661940 learned_addr learned my addr 172.21.15.5:0/2154661940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ead9d700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec109850 0x7f78ec19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec108f00 0x7f78ec19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 -- 172.21.15.5:0/2154661940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec108f00 msgr2=0x7f78ec19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec108f00 0x7f78ec19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 -- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec109850 msgr2=0x7f78ec19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec109850 0x7f78ec19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 -- 172.21.15.5:0/2154661940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec19fcc0 con 0x7f78ec1045d0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec108f00 0x7f78ec19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78ea59c700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec19adf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00ef10 tx=0x7f78dc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78e37fe700 1 -- 172.21.15.5:0/2154661940 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00c9c0 con 0x7f78ec1045d0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.737+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec19ff50 con 0x7f78ec1045d0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.738+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec076b00 con 0x7f78ec1045d0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.738+0000 7f78e37fe700 1 -- 172.21.15.5:0/2154661940 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78dc019070 con 0x7f78ec1045d0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.738+0000 7f78e37fe700 1 -- 172.21.15.5:0/2154661940 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc014480 con 0x7f78ec1045d0 2024-05-14T15:35:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.740+0000 7f78e37fe700 1 -- 172.21.15.5:0/2154661940 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f78dc012050 con 0x7f78ec1045d0 2024-05-14T15:35:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.741+0000 7f78e37fe700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b2a70 0x7f78d80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.741+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b2a70 0x7f78d80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.741+0000 7f78e37fe700 1 -- 172.21.15.5:0/2154661940 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f78dc0e28b0 con 0x7f78ec1045d0 2024-05-14T15:35:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.741+0000 7f78ff3f7b80 1 Processor -- start 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 -- start start 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b2a70 0x7f78d80b4f80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78d4000c00 tx=0x7f78d401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36989720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc3698df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3698e470 0x55bc3698e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc368ab860 con 0x55bc3698e470 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc3684a7c0 con 0x55bc3680be40 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bc368a7820 con 0x55bc36989c60 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ead9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3698e470 0x55bc3698e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36989720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ead9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3698e470 0x55bc3698e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43712/0 (socket says 172.21.15.5:43712) 2024-05-14T15:35:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36989720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:57112/0 (socket says 172.21.15.5:57112) 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78ead9d700 1 -- 172.21.15.5:0/3283249890 learned_addr learned my addr 172.21.15.5:0/3283249890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.742+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc3698df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78ea59c700 1 -- 172.21.15.5:0/3283249890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 msgr2=0x55bc3698df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78ea59c700 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc3698df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78ea59c700 1 -- 172.21.15.5:0/3283249890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3698e470 msgr2=0x55bc3698e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78ea59c700 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3698e470 0x55bc3698e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78ea59c700 1 -- 172.21.15.5:0/3283249890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc367f88d0 con 0x55bc3680be40 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78ea59c700 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36989720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0dfbe0 tx=0x7f78dc0a91d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2cbdaf88036ab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78e0ff9700 1 -- 172.21.15.5:0/3283249890 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc091320 con 0x55bc3680be40 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.743+0000 7f78e0ff9700 1 -- 172.21.15.5:0/3283249890 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78dc0a9770 con 0x55bc3680be40 2024-05-14T15:35:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78e0ff9700 1 -- 172.21.15.5:0/3283249890 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0a9c30 con 0x55bc3680be40 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/3283249890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 msgr2=0x55bc36989720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36989720 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f78dc0dfbe0 tx=0x7f78dc0a91d0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/3283249890 shutdown_connections 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3698e470 0x55bc3698e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36989720 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/3283249890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc3698df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/3283249890 >> 172.21.15.5:0/3283249890 conn(0x55bc36853600 msgr2=0x55bc369844b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/3283249890 shutdown_connections 2024-05-14T15:35:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.744+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/3283249890 wait complete. 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.745+0000 7f78ff3f7b80 1 -- start start 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.745+0000 7f78ff3f7b80 1 Processor -- start 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.745+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36993190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc369936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3698e470 0x55bc36993c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc36996420 con 0x55bc36989c60 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc369936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36993190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc369936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43724/0 (socket says 172.21.15.5:43724) 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 -- 172.21.15.5:0/4156271697 learned_addr learned my addr 172.21.15.5:0/4156271697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc36811c30 con 0x55bc3680be40 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bc369965a0 con 0x55bc3698e470 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ff3f7b80 10 client.0 _ll_register_callbacks cb 0x55bc36843920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ead9d700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3698e470 0x55bc36993c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 -- 172.21.15.5:0/4156271697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3698e470 msgr2=0x55bc36993c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3698e470 0x55bc36993c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 -- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 msgr2=0x55bc36993190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36993190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 -- 172.21.15.5:0/4156271697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc36847690 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78ea59c700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36993190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc369936d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f78d40099c0 tx=0x7f78d4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361193e91fc5b71c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.746+0000 7f78d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78d1ffb700 1 -- 172.21.15.5:0/4156271697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4035060 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78d1ffb700 1 -- 172.21.15.5:0/4156271697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f78d400f660 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78dc0a9770 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78d1ffb700 1 -- 172.21.15.5:0/4156271697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f78d4003e30 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 10 client.18003 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc369a1030 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78d1ffb700 1 -- 172.21.15.5:0/4156271697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78dc0a9770 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78d1ffb700 1 -- 172.21.15.5:0/4156271697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f78d4026810 con 0x55bc36989c60 2024-05-14T15:35:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 10 client.18003 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 20 client.18003 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc369a3670 0x55bc369a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 4 client.18003 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.747+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc368b9240 con 0x55bc369a3670 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.748+0000 7f78ea59c700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc369a3670 0x55bc369a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.748+0000 7f78ea59c700 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc369a3670 0x55bc369a5a90 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671cefcb9d142556 server_cookie=dd1769a1e3d5e21a in_seq=0 out_seq=0 2024-05-14T15:35:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.748+0000 7f78d1ffb700 10 client.18003 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.749+0000 7f78d1ffb700 1 -- 172.21.15.5:0/4156271697 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc368b9240 con 0x55bc369a3670 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.749+0000 7f78d1ffb700 10 client.18003 handle_command_reply: tid=0 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.749+0000 7f78ff3f7b80 1 client.18003 shutdown 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 msgr2=0x55bc369936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc369936d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f78d40099c0 tx=0x7f78d4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 shutdown_connections 2024-05-14T15:35:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc369a3670 0x55bc369a5a90 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3698e470 0x55bc36993c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3680be40 0x55bc36993190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78d1ffb700 0 client.18003 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/4156271697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc369936d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 >> 172.21.15.5:0/4156271697 conn(0x55bc36853600 msgr2=0x55bc3698e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 shutdown_connections 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/4156271697 wait complete. 2024-05-14T15:35:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.750+0000 7f78ff3f7b80 20 client.18003 trim_cache size 0 max 16384 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.770+0000 7f78ff3f7b80 1 Processor -- start 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.770+0000 7f78ff3f7b80 1 -- start start 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 0x55bc3698f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc36988b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc3699f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc011660 con 0x55bc3680be40 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc00eef0 con 0x55bc3698e470 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc00cd40 con 0x55bc36989c60 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 0x55bc3698f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78e9d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc36988b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 0x55bc3698f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43732/0 (socket says 172.21.15.5:43732) 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 -- 172.21.15.5:0/1448517784 learned_addr learned my addr 172.21.15.5:0/1448517784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ead9d700 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc3699f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 -- 172.21.15.5:0/1448517784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 msgr2=0x55bc36988b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc36988b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 -- 172.21.15.5:0/1448517784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 msgr2=0x55bc3699f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc3699f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 -- 172.21.15.5:0/1448517784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc36985c30 con 0x55bc3680be40 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc36988b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ead9d700 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc3699f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.771+0000 7f78ea59c700 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 0x55bc3698f890 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009c10 tx=0x7f78dc009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0389ef969ba6292 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.772+0000 7f78d3fff700 1 -- 172.21.15.5:0/1448517784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0a9280 con 0x55bc3680be40 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.772+0000 7f78d3fff700 1 -- 172.21.15.5:0/1448517784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78dc0c2070 con 0x55bc3680be40 2024-05-14T15:35:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.772+0000 7f78d3fff700 1 -- 172.21.15.5:0/1448517784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0b2140 con 0x55bc3680be40 2024-05-14T15:35:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.772+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/1448517784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 msgr2=0x55bc3698f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.772+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 0x55bc3698f890 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009c10 tx=0x7f78dc009eb0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/1448517784 shutdown_connections 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc3699f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3680be40 0x55bc3698f890 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/1448517784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc36989c60 0x55bc36988b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/1448517784 >> 172.21.15.5:0/1448517784 conn(0x55bc36853600 msgr2=0x55bc36988450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/1448517784 shutdown_connections 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/1448517784 wait complete. 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 -- start start 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 Processor -- start 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.773+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3680be40 0x55bc36997b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc36998040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc36a0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc00d3b0 con 0x55bc36989c60 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc00e820 con 0x55bc3698e470 2024-05-14T15:35:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc00da80 con 0x55bc3680be40 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 10 client.0 _ll_register_callbacks cb 0x55bc36843920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc36998040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc36998040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43748/0 (socket says 172.21.15.5:43748) 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 -- 172.21.15.5:0/657304028 learned_addr learned my addr 172.21.15.5:0/657304028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ea59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3680be40 0x55bc36997b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ead9d700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc36a0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 -- 172.21.15.5:0/657304028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3680be40 msgr2=0x55bc36997b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3680be40 0x55bc36997b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 -- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 msgr2=0x55bc36a0ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc36a0ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 -- 172.21.15.5:0/657304028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc36a0d380 con 0x55bc36989c60 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ea59c700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3680be40 0x55bc36997b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78e9d9b700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc36998040 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x55bc369850a0 tx=0x7f78d400e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489a34f265acb6ab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78d1ffb700 1 -- 172.21.15.5:0/657304028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4043070 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78dc0c2070 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.774+0000 7f78d1ffb700 1 -- 172.21.15.5:0/657304028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f78d4026640 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78d1ffb700 1 -- 172.21.15.5:0/657304028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f78d4035040 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 10 client.18009 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc36a0e4c0 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78d1ffb700 1 -- 172.21.15.5:0/657304028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78dc0c2070 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78d1ffb700 1 -- 172.21.15.5:0/657304028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f78d403e040 con 0x55bc36989c60 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 10 client.18009 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 20 client.18009 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc36a0eb70 0x55bc36a10f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 4 client.18009 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc368b32e0 con 0x55bc36a0eb70 2024-05-14T15:35:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.775+0000 7f78ea59c700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc36a0eb70 0x55bc36a10f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78d1ffb700 10 client.18009 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ea59c700 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc36a0eb70 0x55bc36a10f90 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e26e7ab5fcd83b8 server_cookie=714493d15578ab7d in_seq=0 out_seq=0 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78d1ffb700 1 -- 172.21.15.5:0/657304028 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bc368b32e0 con 0x55bc36a0eb70 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78d1ffb700 10 client.18009 handle_command_reply: tid=0 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 client.18009 shutdown 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 msgr2=0x55bc36998040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc36998040 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x55bc369850a0 tx=0x7f78d400e690 comp rx=0 tx=0).stop 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 shutdown_connections 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bc36a0eb70 0x55bc36a10f90 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc3698e470 0x55bc36a0ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc3680be40 0x55bc36997b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 --2- 172.21.15.5:0/657304028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc36989c60 0x55bc36998040 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 >> 172.21.15.5:0/657304028 conn(0x55bc36853600 msgr2=0x55bc36990890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78d1ffb700 0 client.18009 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 shutdown_connections 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.776+0000 7f78ff3f7b80 1 -- 172.21.15.5:0/657304028 wait complete. 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78ff3f7b80 20 client.18009 trim_cache size 0 max 16384 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:35:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "8ee16471-e7b4-46fb-88f1-592aaa09d73b": { 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8ee16471-e7b4-46fb-88f1-592aaa09d73b", 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:35:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b2a70 msgr2=0x7f78d80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b2a70 0x7f78d80b4f80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78d4000c00 tx=0x7f78d401a040 comp rx=0 tx=0).stop 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 msgr2=0x7f78ec19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec19adf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00ef10 tx=0x7f78dc00c3a0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 shutdown_connections 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b2a70 0x7f78d80b4f80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec109850 0x7f78ec19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec1045d0 0x7f78ec19adf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 --2- 172.21.15.5:0/2154661940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec108f00 0x7f78ec19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 >> 172.21.15.5:0/2154661940 conn(0x7f78ec0ffe80 msgr2=0x7f78ec10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 shutdown_connections 2024-05-14T15:35:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:35:29.777+0000 7f78f09e3700 1 -- 172.21.15.5:0/2154661940 wait complete. 2024-05-14T15:35:30.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:29 smithi005 ceph-mon[31559]: pgmap v336: 129 pgs: 129 active+clean; 82 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-05-14T15:35:30.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 991440 9.48 MB/sec execute 411 sec latency 20.244 ms 2024-05-14T15:35:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:29 smithi144 ceph-mon[35650]: pgmap v336: 129 pgs: 129 active+clean; 82 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-05-14T15:35:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:29 smithi193 ceph-mon[36001]: pgmap v336: 129 pgs: 129 active+clean; 82 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-05-14T15:35:30.644 INFO:tasks.cephfs.filesystem:scrub status for tag:8ee16471-e7b4-46fb-88f1-592aaa09d73b - {'path': '/', 'tag': '8ee16471-e7b4-46fb-88f1-592aaa09d73b', 'options': 'recursive,force'} 2024-05-14T15:35:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:30 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:35:31.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 992825 9.47 MB/sec execute 412 sec latency 21.247 ms 2024-05-14T15:35:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:30 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:35:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:30 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:35:32.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 994895 9.48 MB/sec execute 413 sec latency 21.716 ms 2024-05-14T15:35:32.199 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:31 smithi144 ceph-mon[35650]: pgmap v337: 129 pgs: 129 active+clean; 82 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 KiB/s wr, 210 op/s 2024-05-14T15:35:32.205 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:31 smithi193 ceph-mon[36001]: pgmap v337: 129 pgs: 129 active+clean; 82 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 KiB/s wr, 210 op/s 2024-05-14T15:35:32.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:31 smithi005 ceph-mon[31559]: pgmap v337: 129 pgs: 129 active+clean; 82 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 KiB/s wr, 210 op/s 2024-05-14T15:35:33.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 996177 9.47 MB/sec execute 414 sec latency 17.139 ms 2024-05-14T15:35:34.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 998302 9.48 MB/sec execute 415 sec latency 20.201 ms 2024-05-14T15:35:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:33 smithi144 ceph-mon[35650]: pgmap v338: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-05-14T15:35:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:33 smithi193 ceph-mon[36001]: pgmap v338: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-05-14T15:35:34.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:33 smithi005 ceph-mon[31559]: pgmap v338: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-05-14T15:35:35.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 999226 9.46 MB/sec execute 416 sec latency 20.386 ms 2024-05-14T15:35:36.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1001530 9.47 MB/sec execute 417 sec latency 45.685 ms 2024-05-14T15:35:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:35 smithi144 ceph-mon[35650]: pgmap v339: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-05-14T15:35:36.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:35 smithi193 ceph-mon[36001]: pgmap v339: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-05-14T15:35:36.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:35 smithi005 ceph-mon[31559]: pgmap v339: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 241 op/s 2024-05-14T15:35:37.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1002586 9.46 MB/sec execute 418 sec latency 21.373 ms 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 -- 172.21.15.5:0/1169713360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 msgr2=0x7f52cc10dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 --2- 172.21.15.5:0/1169713360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc10dc20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009900 tx=0x7f52bc02a410 comp rx=0 tx=0).stop 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 -- 172.21.15.5:0/1169713360 shutdown_connections 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 --2- 172.21.15.5:0/1169713360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc10e1f0 0x7f52cc1106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 --2- 172.21.15.5:0/1169713360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc10dc20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 --2- 172.21.15.5:0/1169713360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52cc104da0 0x7f52cc1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 -- 172.21.15.5:0/1169713360 >> 172.21.15.5:0/1169713360 conn(0x7f52cc0782e0 msgr2=0x7f52cc078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 -- 172.21.15.5:0/1169713360 shutdown_connections 2024-05-14T15:35:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.292+0000 7f52d3884640 1 -- 172.21.15.5:0/1169713360 wait complete. 2024-05-14T15:35:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.293+0000 7f52d3884640 1 Processor -- start 2024-05-14T15:35:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.293+0000 7f52d3884640 1 -- start start 2024-05-14T15:35:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52cc104da0 0x7f52cc1a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc1a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc10e1f0 0x7f52cc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc113ea0 con 0x7f52cc10b800 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc113d20 con 0x7f52cc104da0 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc114020 con 0x7f52cc10e1f0 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc1a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc1a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50064/0 (socket says 172.21.15.5:50064) 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 -- 172.21.15.5:0/1032098425 learned_addr learned my addr 172.21.15.5:0/1032098425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d1e21640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc10e1f0 0x7f52cc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d1620640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52cc104da0 0x7f52cc1a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 -- 172.21.15.5:0/1032098425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc10e1f0 msgr2=0x7f52cc19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc10e1f0 0x7f52cc19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 -- 172.21.15.5:0/1032098425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52cc104da0 msgr2=0x7f52cc1a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52cc104da0 0x7f52cc1a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 -- 172.21.15.5:0/1032098425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc19cf00 con 0x7f52cc10b800 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc1a2b90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005ab0 tx=0x7f52bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52c27fc640 1 -- 172.21.15.5:0/1032098425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc03f070 con 0x7f52cc10b800 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc19d190 con 0x7f52cc10b800 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52c27fc640 1 -- 172.21.15.5:0/1032098425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc031b60 con 0x7f52cc10b800 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.294+0000 7f52c27fc640 1 -- 172.21.15.5:0/1032098425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035030 con 0x7f52cc10b800 2024-05-14T15:35:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.295+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1a94d0 con 0x7f52cc10b800 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.297+0000 7f52da80b740 1 Processor -- start 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.297+0000 7f52da80b740 1 -- start start 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.297+0000 7f52c27fc640 1 -- 172.21.15.5:0/1032098425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f52bc042070 con 0x7f52cc10b800 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.297+0000 7f52da80b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x55758670abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52da80b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x55758670b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52da80b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863494d0 0x55758670b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52da80b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575866705d0 con 0x55758636a1b0 2024-05-14T15:35:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52da80b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557586629890 con 0x5575863494d0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52da80b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55758666c590 con 0x5575863412f0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x55758670abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x55758670abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50068/0 (socket says 172.21.15.5:50068) 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 -- 172.21.15.5:0/3022595459 learned_addr learned my addr 172.21.15.5:0/3022595459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1620640 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x55758670b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863494d0 0x55758670b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 -- 172.21.15.5:0/3022595459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 msgr2=0x55758670b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x55758670b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 -- 172.21.15.5:0/3022595459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863494d0 msgr2=0x55758670b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863494d0 0x55758670b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 -- 172.21.15.5:0/3022595459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557586382210 con 0x55758636a1b0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52d1e21640 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x55758670abe0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009f50 tx=0x7f52c40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e704a35f26f49590 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52c27fc640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52b80b2950 0x7f52b80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52b77fe640 1 -- 172.21.15.5:0/3022595459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4004d20 con 0x55758636a1b0 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52b77fe640 1 -- 172.21.15.5:0/3022595459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c4004050 con 0x55758636a1b0 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.298+0000 7f52b77fe640 1 -- 172.21.15.5:0/3022595459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4005280 con 0x55758636a1b0 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.299+0000 7f52d1620640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52b80b2950 0x7f52b80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.299+0000 7f52c27fc640 1 -- 172.21.15.5:0/1032098425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f52bc1030c0 con 0x7f52cc10b800 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.299+0000 7f52d1620640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52b80b2950 0x7f52b80b4e60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55758670bf90 tx=0x7f52c8005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.299+0000 7f52da80b740 1 -- 172.21.15.5:0/3022595459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 msgr2=0x55758670abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.299+0000 7f52da80b740 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x55758670abe0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009f50 tx=0x7f52c40118d0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 -- 172.21.15.5:0/3022595459 shutdown_connections 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x55758670b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863494d0 0x55758670b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 --2- 172.21.15.5:0/3022595459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x55758670abe0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 -- 172.21.15.5:0/3022595459 >> 172.21.15.5:0/3022595459 conn(0x55758635e300 msgr2=0x557586707340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 -- 172.21.15.5:0/3022595459 shutdown_connections 2024-05-14T15:35:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 -- 172.21.15.5:0/3022595459 wait complete. 2024-05-14T15:35:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 -- start start 2024-05-14T15:35:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.300+0000 7f52da80b740 1 Processor -- start 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 0x557586715160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557586717d10 con 0x5575863412f0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55758662d370 con 0x55758636a1b0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557586717e90 con 0x5575863494d0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52da80b740 10 client.0 _ll_register_callbacks cb 0x557586381a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 0x557586715160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 0x557586715160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50080/0 (socket says 172.21.15.5:50080) 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 -- 172.21.15.5:0/1327352714 learned_addr learned my addr 172.21.15.5:0/1327352714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1e21640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 -- 172.21.15.5:0/1327352714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 msgr2=0x5575867156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 -- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 msgr2=0x5575867211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d1620640 1 -- 172.21.15.5:0/1327352714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758644e620 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.301+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52d1620640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 0x557586715160 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f52c80045f0 tx=0x7f52c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8777749d49d419 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52b4ff9640 1 -- 172.21.15.5:0/1327352714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8005020 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52b4ff9640 1 -- 172.21.15.5:0/1327352714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f52c80051c0 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52c4004050 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52b4ff9640 1 -- 172.21.15.5:0/1327352714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f52c80053c0 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 10 client.18021 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557586724060 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52b4ff9640 1 -- 172.21.15.5:0/1327352714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c4004050 con 0x5575863412f0 2024-05-14T15:35:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52b4ff9640 1 -- 172.21.15.5:0/1327352714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f52c8027070 con 0x5575863412f0 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 10 client.18021 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 20 client.18021 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557586328d80 0x557586727570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 4 client.18021 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.302+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55758667e1e0 con 0x557586328d80 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.303+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557586328d80 0x557586727570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.303+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557586328d80 0x557586727570 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5ab1422eedac1b server_cookie=5895b69037b418bc in_seq=0 out_seq=0 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.303+0000 7f52b4ff9640 10 client.18021 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52b4ff9640 1 -- 172.21.15.5:0/1327352714 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55758667e1e0 con 0x557586328d80 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52b4ff9640 10 client.18021 handle_command_reply: tid=0 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 client.18021 shutdown 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 msgr2=0x557586715160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 0x557586715160 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f52c80045f0 tx=0x7f52c8004730 comp rx=0 tx=0).stop 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 shutdown_connections 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557586328d80 0x557586727570 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52b4ff9640 0 client.18021 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 --2- 172.21.15.5:0/1327352714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863412f0 0x557586715160 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.304+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 >> 172.21.15.5:0/1327352714 conn(0x55758635e300 msgr2=0x55758670aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.305+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 shutdown_connections 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.305+0000 7f52da80b740 1 -- 172.21.15.5:0/1327352714 wait complete. 2024-05-14T15:35:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.305+0000 7f52da80b740 20 client.18021 trim_cache size 0 max 16384 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.321+0000 7f52da80b740 1 Processor -- start 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.321+0000 7f52da80b740 1 -- start start 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52da80b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x5575867170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52da80b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 0x557586706080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52da80b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52da80b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1b5de0 con 0x5575863494d0 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52da80b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1b36b0 con 0x55758636a1b0 2024-05-14T15:35:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52da80b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1b1500 con 0x5575863412f0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 0x557586706080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 0x557586706080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50096/0 (socket says 172.21.15.5:50096) 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 -- 172.21.15.5:0/3328486447 learned_addr learned my addr 172.21.15.5:0/3328486447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d1620640 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x5575867170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d1e21640 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 -- 172.21.15.5:0/3328486447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 msgr2=0x5575867170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x5575867170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 -- 172.21.15.5:0/3328486447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 msgr2=0x5575867065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 -- 172.21.15.5:0/3328486447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758670bfc0 con 0x5575863494d0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d1620640 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x5575867170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 0x557586706080 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0c6110 tx=0x7f52bc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c12b6cdd0f5f8a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.322+0000 7f52b6ffd640 1 -- 172.21.15.5:0/3328486447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc049040 con 0x5575863494d0 2024-05-14T15:35:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.323+0000 7f52b6ffd640 1 -- 172.21.15.5:0/3328486447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc0c9ba0 con 0x5575863494d0 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.323+0000 7f52b6ffd640 1 -- 172.21.15.5:0/3328486447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc031d10 con 0x5575863494d0 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.323+0000 7f52da80b740 1 -- 172.21.15.5:0/3328486447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 msgr2=0x557586706080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.323+0000 7f52da80b740 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 0x557586706080 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0c6110 tx=0x7f52bc031400 comp rx=0 tx=0).stop 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 -- 172.21.15.5:0/3328486447 shutdown_connections 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55758636a1b0 0x5575867065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575863494d0 0x557586706080 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 --2- 172.21.15.5:0/3328486447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863412f0 0x5575867170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 -- 172.21.15.5:0/3328486447 >> 172.21.15.5:0/3328486447 conn(0x55758635e300 msgr2=0x557586705970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 -- 172.21.15.5:0/3328486447 shutdown_connections 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 -- 172.21.15.5:0/3328486447 wait complete. 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 -- start start 2024-05-14T15:35:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.324+0000 7f52da80b740 1 Processor -- start 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863412f0 0x55758671d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x557586718f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1b1b70 con 0x55758636a1b0 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1b2fe0 con 0x5575863412f0 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1b2240 con 0x5575863494d0 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52da80b740 10 client.0 _ll_register_callbacks cb 0x7f52c8009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d1e21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x557586718f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:49254/0 (socket says 172.21.15.5:49254) 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d1620640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863412f0 0x55758671d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 -- 172.21.15.5:0/2496221705 learned_addr learned my addr 172.21.15.5:0/2496221705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 -- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863412f0 msgr2=0x55758671d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863412f0 0x55758671d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 -- 172.21.15.5:0/2496221705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 msgr2=0x557586718f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x557586718f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 -- 172.21.15.5:0/2496221705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575867195b0 con 0x5575863494d0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52d0e1f640 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867189d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0c9d90 tx=0x7f52bc0c9b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd94176634eebc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.325+0000 7f52b4ff9640 1 -- 172.21.15.5:0/2496221705 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc049040 con 0x5575863494d0 2024-05-14T15:35:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52bc0c9ba0 con 0x5575863494d0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52b4ff9640 1 -- 172.21.15.5:0/2496221705 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f52bc0c6320 con 0x5575863494d0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52b4ff9640 1 -- 172.21.15.5:0/2496221705 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f52bc0ca090 con 0x5575863494d0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 10 client.27601 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55758678f780 con 0x5575863494d0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52b4ff9640 1 -- 172.21.15.5:0/2496221705 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc0c9ba0 con 0x5575863494d0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52b4ff9640 1 -- 172.21.15.5:0/2496221705 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f52bc0ca2f0 con 0x5575863494d0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 10 client.27601 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 10 client.27601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 20 client.27601 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5575862fe1c0 0x557586791e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 4 client.27601 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557586678050 con 0x5575862fe1c0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.326+0000 7f52d1620640 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5575862fe1c0 0x557586791e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.327+0000 7f52d1620640 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5575862fe1c0 0x557586791e60 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29a64f2424c5c0e server_cookie=a33b16c26bbc8eb3 in_seq=0 out_seq=0 2024-05-14T15:35:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.327+0000 7f52b4ff9640 10 client.27601 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52b4ff9640 1 -- 172.21.15.5:0/2496221705 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557586678050 con 0x5575862fe1c0 2024-05-14T15:35:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52b4ff9640 10 client.27601 handle_command_reply: tid=0 2024-05-14T15:35:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 client.27601 shutdown 2024-05-14T15:35:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 msgr2=0x5575867189d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867189d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0c9d90 tx=0x7f52bc0c9b10 comp rx=0 tx=0).stop 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 shutdown_connections 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5575862fe1c0 0x557586791e60 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575863494d0 0x5575867189d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55758636a1b0 0x557586718f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52b4ff9640 0 client.27601 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 --2- 172.21.15.5:0/2496221705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575863412f0 0x55758671d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 >> 172.21.15.5:0/2496221705 conn(0x55758635e300 msgr2=0x557586716520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 shutdown_connections 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.706+0000 7f52da80b740 1 -- 172.21.15.5:0/2496221705 wait complete. 2024-05-14T15:35:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52da80b740 20 client.27601 trim_cache size 0 max 16384 2024-05-14T15:35:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:35:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:35:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:35:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52b80b2950 msgr2=0x7f52b80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52b80b2950 0x7f52b80b4e60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55758670bf90 tx=0x7f52c8005f90 comp rx=0 tx=0).stop 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 msgr2=0x7f52cc1a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc1a2b90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005ab0 tx=0x7f52bc003c10 comp rx=0 tx=0).stop 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 shutdown_connections 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f52b80b2950 0x7f52b80b4e60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc10e1f0 0x7f52cc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc10b800 0x7f52cc1a2b90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 --2- 172.21.15.5:0/1032098425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52cc104da0 0x7f52cc1a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 >> 172.21.15.5:0/1032098425 conn(0x7f52cc0782e0 msgr2=0x7f52cc10a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 shutdown_connections 2024-05-14T15:35:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:37.707+0000 7f52d3884640 1 -- 172.21.15.5:0/1032098425 wait complete. 2024-05-14T15:35:38.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1003441 9.45 MB/sec execute 419 sec latency 266.229 ms 2024-05-14T15:35:38.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:37 smithi144 ceph-mon[35650]: pgmap v340: 129 pgs: 129 active+clean; 93 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-05-14T15:35:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:37 smithi193 ceph-mon[36001]: pgmap v340: 129 pgs: 129 active+clean; 93 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-05-14T15:35:38.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:37 smithi005 ceph-mon[31559]: pgmap v340: 129 pgs: 129 active+clean; 93 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-05-14T15:35:39.156 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1004632 9.44 MB/sec execute 420 sec latency 59.010 ms 2024-05-14T15:35:40.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1006033 9.44 MB/sec execute 421 sec latency 19.220 ms 2024-05-14T15:35:40.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:39 smithi144 ceph-mon[35650]: pgmap v341: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-05-14T15:35:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:39 smithi193 ceph-mon[36001]: pgmap v341: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-05-14T15:35:40.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:39 smithi005 ceph-mon[31559]: pgmap v341: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-05-14T15:35:41.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1007996 9.44 MB/sec execute 422 sec latency 16.166 ms 2024-05-14T15:35:42.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1009509 9.44 MB/sec execute 423 sec latency 35.331 ms 2024-05-14T15:35:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:41 smithi144 ceph-mon[35650]: pgmap v342: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-14T15:35:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:41 smithi193 ceph-mon[36001]: pgmap v342: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-14T15:35:42.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:41 smithi005 ceph-mon[31559]: pgmap v342: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-05-14T15:35:43.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1011348 9.44 MB/sec execute 424 sec latency 22.569 ms 2024-05-14T15:35:44.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1012881 9.44 MB/sec execute 425 sec latency 29.993 ms 2024-05-14T15:35:44.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:43 smithi144 ceph-mon[35650]: pgmap v343: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-05-14T15:35:44.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:43 smithi193 ceph-mon[36001]: pgmap v343: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-05-14T15:35:44.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:43 smithi005 ceph-mon[31559]: pgmap v343: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 282 op/s 2024-05-14T15:35:45.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1014153 9.43 MB/sec execute 426 sec latency 39.592 ms 2024-05-14T15:35:46.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1016196 9.44 MB/sec execute 427 sec latency 23.300 ms 2024-05-14T15:35:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:45 smithi144 ceph-mon[35650]: pgmap v344: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-05-14T15:35:46.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:45 smithi193 ceph-mon[36001]: pgmap v344: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-05-14T15:35:46.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:45 smithi005 ceph-mon[31559]: pgmap v344: 129 pgs: 129 active+clean; 89 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-05-14T15:35:47.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1017579 9.43 MB/sec execute 428 sec latency 18.091 ms 2024-05-14T15:35:48.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1019701 9.44 MB/sec execute 429 sec latency 16.776 ms 2024-05-14T15:35:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:47 smithi144 ceph-mon[35650]: pgmap v345: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-05-14T15:35:48.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:47 smithi193 ceph-mon[36001]: pgmap v345: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-05-14T15:35:48.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:47 smithi005 ceph-mon[31559]: pgmap v345: 129 pgs: 129 active+clean; 95 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-05-14T15:35:49.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1020829 9.43 MB/sec execute 430 sec latency 29.350 ms 2024-05-14T15:35:50.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1022792 9.43 MB/sec execute 431 sec latency 71.643 ms 2024-05-14T15:35:50.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:49 smithi144 ceph-mon[35650]: pgmap v346: 129 pgs: 129 active+clean; 96 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2024-05-14T15:35:50.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:49 smithi193 ceph-mon[36001]: pgmap v346: 129 pgs: 129 active+clean; 96 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2024-05-14T15:35:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:49 smithi005 ceph-mon[31559]: pgmap v346: 129 pgs: 129 active+clean; 96 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.9 MiB/s wr, 270 op/s 2024-05-14T15:35:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.829+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/902249847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 msgr2=0x7f8f48102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.829+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/902249847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f48102f90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8f3801a910 tx=0x7f8f3803b450 comp rx=0 tx=0).stop 2024-05-14T15:35:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.832+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/902249847 shutdown_connections 2024-05-14T15:35:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.832+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/902249847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f48108000 0x7f8f48108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.832+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/902249847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f481075d0 0x7f8f481079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.832+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/902249847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f48102f90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.832+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/902249847 >> 172.21.15.5:0/902249847 conn(0x7f8f480fe390 msgr2=0x7f8f481007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.836+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/902249847 shutdown_connections 2024-05-14T15:35:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.838+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/902249847 wait complete. 2024-05-14T15:35:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 Processor -- start 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 -- start start 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f481105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f481075d0 0x7f8f48110af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f48108000 0x7f8f48073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810b450 con 0x7f8f48108000 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810b2d0 con 0x7f8f481075d0 2024-05-14T15:35:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.839+0000 7f8f4ef1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4810b5d0 con 0x7f8f48102b80 2024-05-14T15:35:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.841+0000 7f8f4d4bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f48108000 0x7f8f48073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.842+0000 7f8f4d4bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f48108000 0x7f8f48073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54512/0 (socket says 172.21.15.5:54512) 2024-05-14T15:35:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.842+0000 7f8f4d4bc640 1 -- 172.21.15.5:0/1737792089 learned_addr learned my addr 172.21.15.5:0/1737792089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.842+0000 7f8f47fff640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f481075d0 0x7f8f48110af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.842+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f481105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.843+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1737792089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f481075d0 msgr2=0x7f8f48110af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.843+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f481075d0 0x7f8f48110af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.843+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f48108000 msgr2=0x7f8f48073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.843+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f48108000 0x7f8f48073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.843+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1737792089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f48074350 con 0x7f8f48102b80 2024-05-14T15:35:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.844+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f481105b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f38044040 tx=0x7f8f380407c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.846+0000 7f8f45ffb640 1 -- 172.21.15.5:0/1737792089 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38051070 con 0x7f8f48102b80 2024-05-14T15:35:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.846+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f48074580 con 0x7f8f48102b80 2024-05-14T15:35:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.847+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481ad940 con 0x7f8f48102b80 2024-05-14T15:35:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.847+0000 7f8f45ffb640 1 -- 172.21.15.5:0/1737792089 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f38040c80 con 0x7f8f48102b80 2024-05-14T15:35:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.847+0000 7f8f45ffb640 1 -- 172.21.15.5:0/1737792089 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3804d430 con 0x7f8f48102b80 2024-05-14T15:35:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.853+0000 7f8f55dcf740 1 Processor -- start 2024-05-14T15:35:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.853+0000 7f8f55dcf740 1 -- start start 2024-05-14T15:35:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.854+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a112dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.854+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d642f0 0x5574a112e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.854+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a112e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.854+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574a10935d0 con 0x5574a0d8d1b0 2024-05-14T15:35:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.854+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574a104c890 con 0x5574a0d6c4d0 2024-05-14T15:35:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.854+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574a108f590 con 0x5574a0d642f0 2024-05-14T15:35:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.856+0000 7f8f47fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a112e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.856+0000 7f8f47fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a112e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49390/0 (socket says 172.21.15.5:49390) 2024-05-14T15:35:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.856+0000 7f8f47fff640 1 -- 172.21.15.5:0/715250082 learned_addr learned my addr 172.21.15.5:0/715250082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.857+0000 7f8f45ffb640 1 -- 172.21.15.5:0/1737792089 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8f38054070 con 0x7f8f48102b80 2024-05-14T15:35:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.857+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a112dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.858+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d642f0 0x5574a112e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.858+0000 7f8f47fff640 1 -- 172.21.15.5:0/715250082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d642f0 msgr2=0x5574a112e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.858+0000 7f8f47fff640 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d642f0 0x5574a112e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.858+0000 7f8f47fff640 1 -- 172.21.15.5:0/715250082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 msgr2=0x5574a112dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.858+0000 7f8f47fff640 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a112dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.858+0000 7f8f47fff640 1 -- 172.21.15.5:0/715250082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a0da5210 con 0x5574a0d6c4d0 2024-05-14T15:35:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.859+0000 7f8f47fff640 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a112e660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000a480 tx=0x7f8f400096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea80d7ddb7ed54d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.859+0000 7f8f2effd640 1 -- 172.21.15.5:0/715250082 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40013790 con 0x5574a0d6c4d0 2024-05-14T15:35:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.859+0000 7f8f2effd640 1 -- 172.21.15.5:0/715250082 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40019070 con 0x5574a0d6c4d0 2024-05-14T15:35:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.859+0000 7f8f2effd640 1 -- 172.21.15.5:0/715250082 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40014d20 con 0x5574a0d6c4d0 2024-05-14T15:35:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.859+0000 7f8f45ffb640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f300b2b70 0x7f8f300b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.860+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f300b2b70 0x7f8f300b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.860+0000 7f8f45ffb640 1 -- 172.21.15.5:0/1737792089 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f8f381140f0 con 0x7f8f48102b80 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.860+0000 7f8f55dcf740 1 -- 172.21.15.5:0/715250082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 msgr2=0x5574a112e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.860+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a112e660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000a480 tx=0x7f8f400096d0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 -- 172.21.15.5:0/715250082 shutdown_connections 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d642f0 0x5574a112e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a112e660 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/715250082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a112dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 -- 172.21.15.5:0/715250082 >> 172.21.15.5:0/715250082 conn(0x5574a0d81300 msgr2=0x5574a112a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f300b2b70 0x7f8f300b5080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005fd0 tx=0x7f8f3c00b520 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 -- 172.21.15.5:0/715250082 shutdown_connections 2024-05-14T15:35:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 -- 172.21.15.5:0/715250082 wait complete. 2024-05-14T15:35:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.861+0000 7f8f55dcf740 1 -- start start 2024-05-14T15:35:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 Processor -- start 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a1138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a11386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d8d1b0 0x5574a11441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574a113ad10 con 0x5574a0d642f0 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574a1050370 con 0x5574a0d6c4d0 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574a113ae90 con 0x5574a0d8d1b0 2024-05-14T15:35:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f55dcf740 10 client.0 _ll_register_callbacks cb 0x5574a0da4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.862+0000 7f8f4ccbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a1138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4ccbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a1138190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54538/0 (socket says 172.21.15.5:54538) 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/2418070901 learned_addr learned my addr 172.21.15.5:0/2418070901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d8d1b0 0x5574a11441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f47fff640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a11386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 -- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 msgr2=0x5574a11386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a11386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 -- 172.21.15.5:0/2418070901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 msgr2=0x5574a1138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a1138190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 -- 172.21.15.5:0/2418070901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a0e71620 con 0x5574a0d8d1b0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d8d1b0 0x5574a11441f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c010720 tx=0x7f8f3c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65399ed9bef2e772 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.863+0000 7f8f27fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f27fff640 1 -- 172.21.15.5:0/2418070901 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c005220 con 0x5574a0d8d1b0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f40019070 con 0x5574a0d8d1b0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f27fff640 1 -- 172.21.15.5:0/2418070901 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f8f3c0053c0 con 0x5574a0d8d1b0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f27fff640 1 -- 172.21.15.5:0/2418070901 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8f3c02f050 con 0x5574a0d8d1b0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f55dcf740 10 client.27610 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5574a11470e0 con 0x5574a0d8d1b0 2024-05-14T15:35:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.864+0000 7f8f27fff640 1 -- 172.21.15.5:0/2418070901 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40019070 con 0x5574a0d8d1b0 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f27fff640 1 -- 172.21.15.5:0/2418070901 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8f3c004b50 con 0x5574a0d8d1b0 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f55dcf740 10 client.27610 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f55dcf740 10 client.27610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f55dcf740 20 client.27610 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d4bd80 0x5574a114a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f55dcf740 4 client.27610 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a10a11e0 con 0x5574a0d4bd80 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d4bd80 0x5574a114a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f27fff640 10 client.27610 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.865+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d4bd80 0x5574a114a7f0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d5d441b170ee5b server_cookie=1343e1d14e2dd079 in_seq=0 out_seq=0 2024-05-14T15:35:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.866+0000 7f8f27fff640 1 -- 172.21.15.5:0/2418070901 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574a10a11e0 con 0x5574a0d4bd80 2024-05-14T15:35:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.866+0000 7f8f27fff640 10 client.27610 handle_command_reply: tid=0 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.866+0000 7f8f55dcf740 1 client.27610 shutdown 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.866+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d8d1b0 msgr2=0x5574a11441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.866+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d8d1b0 0x5574a11441f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c010720 tx=0x7f8f3c0046c0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 shutdown_connections 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d4bd80 0x5574a114a7f0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d8d1b0 0x5574a11441f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d6c4d0 0x5574a11386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/2418070901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a1138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 >> 172.21.15.5:0/2418070901 conn(0x5574a0d81300 msgr2=0x5574a112f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f27fff640 0 client.27610 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 shutdown_connections 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 1 -- 172.21.15.5:0/2418070901 wait complete. 2024-05-14T15:35:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.867+0000 7f8f55dcf740 20 client.27610 trim_cache size 0 max 16384 2024-05-14T15:35:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.893+0000 7f8f55dcf740 1 Processor -- start 2024-05-14T15:35:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.893+0000 7f8f55dcf740 1 -- start start 2024-05-14T15:35:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.894+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a113a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.894+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a1137860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.894+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d8d1b0 0x5574a1140140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.894+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34012830 con 0x5574a0d642f0 2024-05-14T15:35:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.894+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34010100 con 0x5574a0d8d1b0 2024-05-14T15:35:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.894+0000 7f8f55dcf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400df50 con 0x5574a0d6c4d0 2024-05-14T15:35:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a113a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a113a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54546/0 (socket says 172.21.15.5:54546) 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1421773308 learned_addr learned my addr 172.21.15.5:0/1421773308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f47fff640 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a1137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d8d1b0 0x5574a1140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1421773308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 msgr2=0x5574a1137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a1137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1421773308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d8d1b0 msgr2=0x5574a1140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d8d1b0 0x5574a1140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/1421773308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a1133780 con 0x5574a0d642f0 2024-05-14T15:35:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.895+0000 7f8f47fff640 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a1137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:35:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.896+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a113a0d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006010 tx=0x7f8f3803b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fca66b903146973 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.896+0000 7f8f2e7fc640 1 -- 172.21.15.5:0/1421773308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380f5070 con 0x5574a0d642f0 2024-05-14T15:35:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.896+0000 7f8f2e7fc640 1 -- 172.21.15.5:0/1421773308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f380daac0 con 0x5574a0d642f0 2024-05-14T15:35:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.896+0000 7f8f2e7fc640 1 -- 172.21.15.5:0/1421773308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3805b030 con 0x5574a0d642f0 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 -- 172.21.15.5:0/1421773308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 msgr2=0x5574a113a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a113a0d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006010 tx=0x7f8f3803b950 comp rx=0 tx=0).stop 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 -- 172.21.15.5:0/1421773308 shutdown_connections 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a1137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d8d1b0 0x5574a1140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/1421773308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d642f0 0x5574a113a0d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.897+0000 7f8f55dcf740 1 -- 172.21.15.5:0/1421773308 >> 172.21.15.5:0/1421773308 conn(0x5574a0d81300 msgr2=0x5574a112d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.898+0000 7f8f55dcf740 1 -- 172.21.15.5:0/1421773308 shutdown_connections 2024-05-14T15:35:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.898+0000 7f8f55dcf740 1 -- 172.21.15.5:0/1421773308 wait complete. 2024-05-14T15:35:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.898+0000 7f8f55dcf740 1 -- start start 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.898+0000 7f8f55dcf740 1 Processor -- start 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 0x5574a113b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a113b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a113bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f4ccbb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 0x5574a113b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f4ccbb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 0x5574a113b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:49428/0 (socket says 172.21.15.5:49428) 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/4082717968 learned_addr learned my addr 172.21.15.5:0/4082717968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400e5c0 con 0x5574a0d8d1b0 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400fa30 con 0x5574a0d642f0 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400ec90 con 0x5574a0d6c4d0 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.899+0000 7f8f55dcf740 10 client.0 _ll_register_callbacks cb 0x7f8f3c0103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:35:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4d4bc640 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a113bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/4082717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 msgr2=0x5574a113b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a113b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/4082717968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 msgr2=0x5574a113bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a113bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4ccbb640 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a11b0c70 con 0x5574a0d642f0 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.900+0000 7f8f4ccbb640 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 0x5574a113b110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f38044010 tx=0x7f8f380db020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ff793bf6e61aee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:35:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.901+0000 7f8f27fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.901+0000 7f8f27fff640 1 -- 172.21.15.5:0/4082717968 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3805b040 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.901+0000 7f8f27fff640 1 -- 172.21.15.5:0/4082717968 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f8f380d7240 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.901+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f380daac0 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.901+0000 7f8f27fff640 1 -- 172.21.15.5:0/4082717968 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8f380f5040 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 10 client.27638 fetch_fsmap learned FSMap version 34 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5574a11b1e30 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f27fff640 1 -- 172.21.15.5:0/4082717968 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f380daac0 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f27fff640 1 -- 172.21.15.5:0/4082717968 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8f380e3980 con 0x5574a0d642f0 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 10 client.27638 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 10 client.27638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 20 client.27638 populate_metadata read hostname 'smithi005' 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d211c0 0x5574a11b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:35:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 4 client.27638 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:35:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.902+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574a109b050 con 0x5574a0d211c0 2024-05-14T15:35:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.903+0000 7f8f47fff640 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d211c0 0x5574a11b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:35:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.908+0000 7f8f47fff640 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d211c0 0x5574a11b44d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166a031c3a3cd462 server_cookie=ebf011690c518780 in_seq=0 out_seq=0 2024-05-14T15:35:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:50.909+0000 7f8f27fff640 10 client.27638 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:51.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1023503 9.42 MB/sec execute 432 sec latency 181.885 ms 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f27fff640 1 -- 172.21.15.5:0/4082717968 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574a109b050 con 0x5574a0d211c0 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f27fff640 10 client.27638 handle_command_reply: tid=0 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 client.27638 shutdown 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 msgr2=0x5574a113b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 0x5574a113b110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f38044010 tx=0x7f8f380db020 comp rx=0 tx=0).stop 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 shutdown_connections 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5574a0d211c0 0x5574a11b44d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574a0d6c4d0 0x5574a113b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574a0d8d1b0 0x5574a113bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 --2- 172.21.15.5:0/4082717968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574a0d642f0 0x5574a113b110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 >> 172.21.15.5:0/4082717968 conn(0x5574a0d81300 msgr2=0x5574a112d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f27fff640 0 client.27638 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 shutdown_connections 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 1 -- 172.21.15.5:0/4082717968 wait complete. 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.278+0000 7f8f55dcf740 20 client.27638 trim_cache size 0 max 16384 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:35:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f300b2b70 msgr2=0x7f8f300b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f300b2b70 0x7f8f300b5080 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005fd0 tx=0x7f8f3c00b520 comp rx=0 tx=0).stop 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 msgr2=0x7f8f481105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f481105b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f38044040 tx=0x7f8f380407c0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 shutdown_connections 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f8f300b2b70 0x7f8f300b5080 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f48108000 0x7f8f48073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f481075d0 0x7f8f48110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 --2- 172.21.15.5:0/1737792089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f48102b80 0x7f8f481105b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.279+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 >> 172.21.15.5:0/1737792089 conn(0x7f8f480fe390 msgr2=0x7f8f48100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.280+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 shutdown_connections 2024-05-14T15:35:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:35:51.280+0000 7f8f4ef1f640 1 -- 172.21.15.5:0/1737792089 wait complete. 2024-05-14T15:35:52.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1024727 9.41 MB/sec execute 433 sec latency 259.532 ms 2024-05-14T15:35:52.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:51 smithi144 ceph-mon[35650]: pgmap v347: 129 pgs: 129 active+clean; 96 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2024-05-14T15:35:52.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:51 smithi193 ceph-mon[36001]: pgmap v347: 129 pgs: 129 active+clean; 96 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2024-05-14T15:35:52.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:51 smithi005 ceph-mon[31559]: pgmap v347: 129 pgs: 129 active+clean; 96 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2024-05-14T15:35:53.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1026801 9.42 MB/sec execute 434 sec latency 17.162 ms 2024-05-14T15:35:54.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1028073 9.41 MB/sec execute 435 sec latency 24.043 ms 2024-05-14T15:35:54.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:53 smithi144 ceph-mon[35650]: pgmap v348: 129 pgs: 129 active+clean; 85 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.7 MiB/s wr, 419 op/s 2024-05-14T15:35:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:53 smithi193 ceph-mon[36001]: pgmap v348: 129 pgs: 129 active+clean; 85 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.7 MiB/s wr, 419 op/s 2024-05-14T15:35:54.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:53 smithi005 ceph-mon[31559]: pgmap v348: 129 pgs: 129 active+clean; 85 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.7 MiB/s wr, 419 op/s 2024-05-14T15:35:55.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1030041 9.41 MB/sec execute 436 sec latency 46.051 ms 2024-05-14T15:35:56.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1030850 9.40 MB/sec execute 437 sec latency 22.007 ms 2024-05-14T15:35:56.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:55 smithi144 ceph-mon[35650]: pgmap v349: 129 pgs: 129 active+clean; 85 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 262 op/s 2024-05-14T15:35:56.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:55 smithi193 ceph-mon[36001]: pgmap v349: 129 pgs: 129 active+clean; 85 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 262 op/s 2024-05-14T15:35:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:55 smithi005 ceph-mon[31559]: pgmap v349: 129 pgs: 129 active+clean; 85 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 262 op/s 2024-05-14T15:35:57.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1032656 9.40 MB/sec execute 438 sec latency 46.079 ms 2024-05-14T15:35:58.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1034147 9.40 MB/sec execute 439 sec latency 17.869 ms 2024-05-14T15:35:58.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:57 smithi144 ceph-mon[35650]: pgmap v350: 129 pgs: 129 active+clean; 84 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-05-14T15:35:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:57 smithi193 ceph-mon[36001]: pgmap v350: 129 pgs: 129 active+clean; 84 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-05-14T15:35:58.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:57 smithi005 ceph-mon[31559]: pgmap v350: 129 pgs: 129 active+clean; 84 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 281 op/s 2024-05-14T15:35:59.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1035652 9.39 MB/sec execute 440 sec latency 40.641 ms 2024-05-14T15:36:00.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1037443 9.40 MB/sec execute 441 sec latency 155.791 ms 2024-05-14T15:36:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:35:59 smithi144 ceph-mon[35650]: pgmap v351: 129 pgs: 129 active+clean; 90 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-05-14T15:36:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:35:59 smithi193 ceph-mon[36001]: pgmap v351: 129 pgs: 129 active+clean; 90 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-05-14T15:36:00.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:35:59 smithi005 ceph-mon[31559]: pgmap v351: 129 pgs: 129 active+clean; 90 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-05-14T15:36:00.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:36:00.993 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:01.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1038122 9.38 MB/sec execute 442 sec latency 98.689 ms 2024-05-14T15:36:02.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:01 smithi005 ceph-mon[31559]: pgmap v352: 129 pgs: 129 active+clean; 90 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-05-14T15:36:02.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1038352 9.36 MB/sec execute 443 sec latency 140.546 ms 2024-05-14T15:36:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:01 smithi144 ceph-mon[35650]: pgmap v352: 129 pgs: 129 active+clean; 90 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-05-14T15:36:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:01 smithi193 ceph-mon[36001]: pgmap v352: 129 pgs: 129 active+clean; 90 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-05-14T15:36:03.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1039365 9.35 MB/sec execute 444 sec latency 104.068 ms 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 -- 172.21.15.5:0/2852221977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 msgr2=0x7f4f041038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 --2- 172.21.15.5:0/2852221977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f041038d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40099c0 tx=0x7f4ef4009cd0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 -- 172.21.15.5:0/2852221977 shutdown_connections 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 --2- 172.21.15.5:0/2852221977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f04108740 0x7f4f041100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 --2- 172.21.15.5:0/2852221977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f041038d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 --2- 172.21.15.5:0/2852221977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f04108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 -- 172.21.15.5:0/2852221977 >> 172.21.15.5:0/2852221977 conn(0x7f4f040fedc0 msgr2=0x7f4f041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.402+0000 7f4f09583700 1 -- 172.21.15.5:0/2852221977 shutdown_connections 2024-05-14T15:36:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.403+0000 7f4f09583700 1 -- 172.21.15.5:0/2852221977 wait complete. 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 Processor -- start 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 -- start start 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f0419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f0419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f04108740 0x7f4f0419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041133c0 con 0x7f4f041034c0 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04113240 con 0x7f4f04107df0 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4f09583700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04113540 con 0x7f4f04108740 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4efbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f0419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4efbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f0419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:52770/0 (socket says 172.21.15.5:52770) 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.404+0000 7f4efbfff700 1 -- 172.21.15.5:0/744718198 learned_addr learned my addr 172.21.15.5:0/744718198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f0419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4efbfff700 1 -- 172.21.15.5:0/744718198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f04108740 msgr2=0x7f4f0419f670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T15:36:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4efbfff700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f04108740 0x7f4f0419f670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4efbfff700 1 -- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 msgr2=0x7f4f0419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4efbfff700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f0419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4efbfff700 1 -- 172.21.15.5:0/744718198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0419fd10 con 0x7f4f04107df0 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f0419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4efbfff700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f0419b3e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e970 tx=0x7f4eec00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f00ff9700 1 -- 172.21.15.5:0/744718198 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec00cbd0 con 0x7f4f04107df0 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f00ff9700 1 -- 172.21.15.5:0/744718198 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eec00cd70 con 0x7f4f04107df0 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041a0000 con 0x7f4f04107df0 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f00ff9700 1 -- 172.21.15.5:0/744718198 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec005610 con 0x7f4f04107df0 2024-05-14T15:36:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.405+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041a7920 con 0x7f4f04107df0 2024-05-14T15:36:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.408+0000 7f4f00ff9700 1 -- 172.21.15.5:0/744718198 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4eec021440 con 0x7f4f04107df0 2024-05-14T15:36:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.409+0000 7f4f00ff9700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef00b29f0 0x7f4ef00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.409+0000 7f4f17f97b80 1 Processor -- start 2024-05-14T15:36:03.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.409+0000 7f4f17f97b80 1 -- start start 2024-05-14T15:36:03.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.409+0000 7f4f00ff9700 1 -- 172.21.15.5:0/744718198 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f4eec0e2d00 con 0x7f4f04107df0 2024-05-14T15:36:03.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.409+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef00b29f0 0x7f4ef00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76529840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be765238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be76524220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55be76445940 con 0x55be76523e10 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55be763aada0 con 0x55be763a5e40 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be76441900 con 0x55be76529d80 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be76524220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be76524220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42760/0 (socket says 172.21.15.5:42760) 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f037fe700 1 -- 172.21.15.5:0/327334618 learned_addr learned my addr 172.21.15.5:0/327334618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4efbfff700 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be765238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef00b29f0 0x7f4ef00b4f00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4006010 tx=0x7f4ef401a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76529840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4efbfff700 1 -- 172.21.15.5:0/327334618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 msgr2=0x55be76529840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4efbfff700 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76529840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4efbfff700 1 -- 172.21.15.5:0/327334618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 msgr2=0x55be76524220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4efbfff700 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be76524220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.410+0000 7f4efbfff700 1 -- 172.21.15.5:0/327334618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be763928d0 con 0x55be76529d80 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.411+0000 7f4efbfff700 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be765238d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0e0090 tx=0x7f4eec017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0634d4bad45b2a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.411+0000 7f4ef97fa700 1 -- 172.21.15.5:0/327334618 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0b23d0 con 0x55be76529d80 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.411+0000 7f4ef97fa700 1 -- 172.21.15.5:0/327334618 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eec0a9730 con 0x55be76529d80 2024-05-14T15:36:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.411+0000 7f4ef97fa700 1 -- 172.21.15.5:0/327334618 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0a9e80 con 0x55be76529d80 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 -- 172.21.15.5:0/327334618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 msgr2=0x55be765238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be765238d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0e0090 tx=0x7f4eec017000 comp rx=0 tx=0).stop 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 -- 172.21.15.5:0/327334618 shutdown_connections 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be76524220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76529840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/327334618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be765238d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 -- 172.21.15.5:0/327334618 >> 172.21.15.5:0/327334618 conn(0x55be763ed630 msgr2=0x55be7651ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 -- 172.21.15.5:0/327334618 shutdown_connections 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.412+0000 7f4f17f97b80 1 -- 172.21.15.5:0/327334618 wait complete. 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 -- start start 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 Processor -- start 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 0x55be7652d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76523e10 0x55be7652da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be7652df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55be76530820 con 0x55be763a5e40 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55be763feed0 con 0x55be76529d80 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be765309a0 con 0x55be76523e10 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f17f97b80 10 client.0 _ll_register_callbacks cb 0x55be763dd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 0x55be7652d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 0x55be7652d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42774/0 (socket says 172.21.15.5:42774) 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f02ffd700 1 -- 172.21.15.5:0/2025989562 learned_addr learned my addr 172.21.15.5:0/2025989562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be7652df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.413+0000 7f4efbfff700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76523e10 0x55be7652da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f037fe700 1 -- 172.21.15.5:0/2025989562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76523e10 msgr2=0x55be7652da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76523e10 0x55be7652da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f037fe700 1 -- 172.21.15.5:0/2025989562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 msgr2=0x55be7652d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 0x55be7652d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f037fe700 1 -- 172.21.15.5:0/2025989562 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be763fc760 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 0x55be7652d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be7652df40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4efc011b80 tx=0x7f4efc011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f224b944c96fa776 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4eea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2025989562 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc01a030 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eec0a9730 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2025989562 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f4efc0092e0 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2025989562 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4efc024050 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f17f97b80 10 client.27650 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.414+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55be7653b480 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2025989562 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eec0a9730 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2025989562 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f4efc007730 con 0x55be76529d80 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f17f97b80 10 client.27650 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f17f97b80 10 client.27650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f17f97b80 20 client.27650 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be7653dac0 0x55be7653ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f17f97b80 4 client.27650 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be76453320 con 0x55be7653dac0 2024-05-14T15:36:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.415+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be7653dac0 0x55be7653ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.416+0000 7f4eea7fc700 10 client.27650 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.416+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be7653dac0 0x55be7653ff40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2374ee5d1604b13 server_cookie=a279218d73318d9a in_seq=0 out_seq=0 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.416+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2025989562 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be76453320 con 0x55be7653dac0 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4eea7fc700 10 client.27650 handle_command_reply: tid=0 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 client.27650 shutdown 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 msgr2=0x55be7652df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be7652df40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4efc011b80 tx=0x7f4efc011e90 comp rx=0 tx=0).stop 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 shutdown_connections 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be7653dac0 0x55be7653ff40 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be7652df40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be763a5e40 0x55be7652d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4eea7fc700 0 client.27650 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2025989562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76523e10 0x55be7652da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 >> 172.21.15.5:0/2025989562 conn(0x55be763ed630 msgr2=0x55be765228c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 shutdown_connections 2024-05-14T15:36:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2025989562 wait complete. 2024-05-14T15:36:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.417+0000 7f4f17f97b80 20 client.27650 trim_cache size 0 max 16384 2024-05-14T15:36:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 Processor -- start 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 -- start start 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 0x55be765238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be7652cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be765398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b5220 con 0x55be76523e10 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b2ab0 con 0x55be76529d80 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b0900 con 0x55be763a5e40 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.438+0000 7f4efbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be7652cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be765398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 0x55be765238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be765398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:52786/0 (socket says 172.21.15.5:52786) 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 0x55be765238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57022/0 (socket says 172.21.15.5:57022) 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f037fe700 1 -- 172.21.15.5:0/3059529437 learned_addr learned my addr 172.21.15.5:0/3059529437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 -- 172.21.15.5:0/3059529437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 msgr2=0x55be765398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be765398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 -- 172.21.15.5:0/3059529437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 msgr2=0x55be7652cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be7652cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 -- 172.21.15.5:0/3059529437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be765293c0 con 0x55be763a5e40 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 0x55be765238d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55be76540670 tx=0x7f4ef401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=debea805731ad70f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.439+0000 7f4ef8ff9700 1 -- 172.21.15.5:0/3059529437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4003e60 con 0x55be763a5e40 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.440+0000 7f4ef8ff9700 1 -- 172.21.15.5:0/3059529437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef40318e0 con 0x55be763a5e40 2024-05-14T15:36:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.440+0000 7f4ef8ff9700 1 -- 172.21.15.5:0/3059529437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400f460 con 0x55be763a5e40 2024-05-14T15:36:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 -- 172.21.15.5:0/3059529437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 msgr2=0x55be765238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 0x55be765238d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55be76540670 tx=0x7f4ef401e000 comp rx=0 tx=0).stop 2024-05-14T15:36:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 -- 172.21.15.5:0/3059529437 shutdown_connections 2024-05-14T15:36:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be76529d80 0x55be765398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be763a5e40 0x55be765238d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/3059529437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be7652cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 -- 172.21.15.5:0/3059529437 >> 172.21.15.5:0/3059529437 conn(0x55be763ed630 msgr2=0x55be7652eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 -- 172.21.15.5:0/3059529437 shutdown_connections 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 -- 172.21.15.5:0/3059529437 wait complete. 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 -- start start 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.441+0000 7f4f17f97b80 1 Processor -- start 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76545090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be765455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be76545b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b0f70 con 0x55be76523e10 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b23e0 con 0x55be763a5e40 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b1640 con 0x55be76529d80 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f17f97b80 10 client.0 _ll_register_callbacks cb 0x55be763dd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be76545b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4efbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be765455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be76545b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57038/0 (socket says 172.21.15.5:57038) 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 -- 172.21.15.5:0/2581981384 learned_addr learned my addr 172.21.15.5:0/2581981384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76545090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 -- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 msgr2=0x55be76545090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76545090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 -- 172.21.15.5:0/2581981384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 msgr2=0x55be765455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be765455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f037fe700 1 -- 172.21.15.5:0/2581981384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be765a7380 con 0x55be76529d80 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76545090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.442+0000 7f4efbfff700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be765455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f037fe700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be76545b10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4efc007bd0 tx=0x7f4efc009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f960e216a9da96c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4eea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2581981384 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc01a040 con 0x55be76529d80 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef40318e0 con 0x55be76529d80 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2581981384 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f4efc003a00 con 0x55be76529d80 2024-05-14T15:36:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2581981384 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4efc035050 con 0x55be76529d80 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 10 client.27625 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55be765a8610 con 0x55be76529d80 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2581981384 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef40318e0 con 0x55be76529d80 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2581981384 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f4efc030030 con 0x55be76529d80 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 10 client.27625 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 10 client.27625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 20 client.27625 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be765a8d40 0x55be765ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 4 client.27625 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.443+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be765291f0 con 0x55be765a8d40 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.444+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be765a8d40 0x55be765ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.444+0000 7f4eea7fc700 10 client.27625 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.444+0000 7f4f02ffd700 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be765a8d40 0x55be765ab160 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0539e48a22d8c1a server_cookie=b4a5dda5986088d4 in_seq=0 out_seq=0 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4eea7fc700 1 -- 172.21.15.5:0/2581981384 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be765291f0 con 0x55be765a8d40 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4eea7fc700 10 client.27625 handle_command_reply: tid=0 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 client.27625 shutdown 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 msgr2=0x55be76545b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be76545b10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4efc007bd0 tx=0x7f4efc009510 comp rx=0 tx=0).stop 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 shutdown_connections 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55be765a8d40 0x55be765ab160 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be76529d80 0x55be76545b10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55be763a5e40 0x55be76545090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 --2- 172.21.15.5:0/2581981384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be76523e10 0x55be765455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 >> 172.21.15.5:0/2581981384 conn(0x55be763ed630 msgr2=0x55be7652eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4eea7fc700 0 client.27625 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 shutdown_connections 2024-05-14T15:36:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.445+0000 7f4f17f97b80 1 -- 172.21.15.5:0/2581981384 wait complete. 2024-05-14T15:36:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.446+0000 7f4f17f97b80 20 client.27625 trim_cache size 0 max 16384 2024-05-14T15:36:03.451 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:36:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:36:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:36:03.451 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.446+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef00b29f0 msgr2=0x7f4ef00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.446+0000 7f4f09583700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef00b29f0 0x7f4ef00b4f00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4006010 tx=0x7f4ef401a040 comp rx=0 tx=0).stop 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.446+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 msgr2=0x7f4f0419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.446+0000 7f4f09583700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f0419b3e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00e970 tx=0x7f4eec00ed30 comp rx=0 tx=0).stop 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 shutdown_connections 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ef00b29f0 0x7f4ef00b4f00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f04108740 0x7f4f0419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f041034c0 0x7f4f0419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 --2- 172.21.15.5:0/744718198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f04107df0 0x7f4f0419b3e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 >> 172.21.15.5:0/744718198 conn(0x7f4f040fedc0 msgr2=0x7f4f0406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 shutdown_connections 2024-05-14T15:36:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:03.447+0000 7f4f09583700 1 -- 172.21.15.5:0/744718198 wait complete. 2024-05-14T15:36:04.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:03 smithi005 ceph-mon[31559]: pgmap v353: 129 pgs: 129 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-05-14T15:36:04.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1041025 9.36 MB/sec execute 445 sec latency 16.637 ms 2024-05-14T15:36:04.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:36:04.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:36:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:03 smithi144 ceph-mon[35650]: pgmap v353: 129 pgs: 129 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-05-14T15:36:04.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:03 smithi193 ceph-mon[36001]: pgmap v353: 129 pgs: 129 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-05-14T15:36:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 -- 172.21.15.5:0/670425787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f538010e230 msgr2=0x7f5380110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 --2- 172.21.15.5:0/670425787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f538010e230 0x7f5380110700 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5378009ca0 tx=0x7f537802a950 comp rx=0 tx=0).stop 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 -- 172.21.15.5:0/670425787 shutdown_connections 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 --2- 172.21.15.5:0/670425787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f538010e230 0x7f5380110700 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 --2- 172.21.15.5:0/670425787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010b880 0x7f538010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 --2- 172.21.15.5:0/670425787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5380104e20 0x7f5380105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 -- 172.21.15.5:0/670425787 >> 172.21.15.5:0/670425787 conn(0x7f53800782e0 msgr2=0x7f5380078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 -- 172.21.15.5:0/670425787 shutdown_connections 2024-05-14T15:36:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.392+0000 7f5386f38640 1 -- 172.21.15.5:0/670425787 wait complete. 2024-05-14T15:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 Processor -- start 2024-05-14T15:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 -- start start 2024-05-14T15:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 0x7f53801a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538010b880 0x7f53801a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010e230 0x7f538019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5380113ec0 con 0x7f5380104e20 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5380113d40 con 0x7f538010e230 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.393+0000 7f5386f38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5380114040 con 0x7f538010b880 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 0x7f53801a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 0x7f53801a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42824/0 (socket says 172.21.15.5:42824) 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 -- 172.21.15.5:0/3511059020 learned_addr learned my addr 172.21.15.5:0/3511059020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f537ffff640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538010b880 0x7f53801a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f53854d5640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010e230 0x7f538019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 -- 172.21.15.5:0/3511059020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538010b880 msgr2=0x7f53801a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538010b880 0x7f53801a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 -- 172.21.15.5:0/3511059020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010e230 msgr2=0x7f538019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010e230 0x7f538019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 -- 172.21.15.5:0/3511059020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538019adf0 con 0x7f5380104e20 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f53854d5640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010e230 0x7f538019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.394+0000 7f5384cd4640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 0x7f53801a0540 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f537400e9f0 tx=0x7f537400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.395+0000 7f537dffb640 1 -- 172.21.15.5:0/3511059020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537400cd80 con 0x7f5380104e20 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.395+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538019b080 con 0x7f5380104e20 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.395+0000 7f537dffb640 1 -- 172.21.15.5:0/3511059020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53740054d0 con 0x7f5380104e20 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.395+0000 7f537dffb640 1 -- 172.21.15.5:0/3511059020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374010690 con 0x7f5380104e20 2024-05-14T15:36:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.395+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53801a72e0 con 0x7f5380104e20 2024-05-14T15:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f537dffb640 1 -- 172.21.15.5:0/3511059020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5374004e90 con 0x7f5380104e20 2024-05-14T15:36:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 Processor -- start 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 -- start start 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12dd92f0 0x559a131a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131a3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a131085d0 con 0x559a12dd92f0 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a130c1890 con 0x559a12e021b0 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f538dded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a13104590 con 0x559a12de14d0 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12dd92f0 0x559a131a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f53854d5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f53854d5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:52816/0 (socket says 172.21.15.5:52816) 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12dd92f0 0x559a131a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42838/0 (socket says 172.21.15.5:42838) 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.398+0000 7f53854d5640 1 -- 172.21.15.5:0/2166492777 learned_addr learned my addr 172.21.15.5:0/2166492777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f537ffff640 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f537dffb640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f53700b2950 0x7f53700b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 -- 172.21.15.5:0/2166492777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 msgr2=0x559a131a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 -- 172.21.15.5:0/2166492777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12dd92f0 msgr2=0x559a131a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12dd92f0 0x559a131a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 -- 172.21.15.5:0/2166492777 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a12e1a210 con 0x559a12e021b0 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f537dffb640 1 -- 172.21.15.5:0/3511059020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f53740e2e00 con 0x7f5380104e20 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f53700b2950 0x7f53700b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f53854d5640 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131a2be0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5378009ca0 tx=0x7f53780093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3984aa7583d16ec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f536affd640 1 -- 172.21.15.5:0/2166492777 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378034040 con 0x559a12e021b0 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.399+0000 7f536affd640 1 -- 172.21.15.5:0/2166492777 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537802fd40 con 0x559a12e021b0 2024-05-14T15:36:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f536affd640 1 -- 172.21.15.5:0/2166492777 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378030740 con 0x559a12e021b0 2024-05-14T15:36:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 -- 172.21.15.5:0/2166492777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 msgr2=0x559a131a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131a2be0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5378009ca0 tx=0x7f53780093f0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f53854d5640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f53700b2950 0x7f53700b4e60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f53780094e0 tx=0x7f537804d000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 -- 172.21.15.5:0/2166492777 shutdown_connections 2024-05-14T15:36:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12dd92f0 0x559a131a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 --2- 172.21.15.5:0/2166492777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131a2be0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.400+0000 7f538dded740 1 -- 172.21.15.5:0/2166492777 >> 172.21.15.5:0/2166492777 conn(0x559a12df6300 msgr2=0x559a1319f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.401+0000 7f538dded740 1 -- 172.21.15.5:0/2166492777 shutdown_connections 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.401+0000 7f538dded740 1 -- 172.21.15.5:0/2166492777 wait complete. 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.401+0000 7f538dded740 1 -- start start 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.401+0000 7f538dded740 1 Processor -- start 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 0x559a131ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131b9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a131afcb0 con 0x559a12e021b0 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a130c5370 con 0x559a12de14d0 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a131afe30 con 0x559a12dd92f0 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 0x559a131ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f538dded740 10 client.0 _ll_register_callbacks cb 0x559a12e19a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 0x559a131ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:52822/0 (socket says 172.21.15.5:52822) 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 -- 172.21.15.5:0/2715848149 learned_addr learned my addr 172.21.15.5:0/2715848149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f53854d5640 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 -- 172.21.15.5:0/2715848149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 msgr2=0x559a131ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 -- 172.21.15.5:0/2715848149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 msgr2=0x559a131b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 -- 172.21.15.5:0/2715848149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a12ee6620 con 0x559a12de14d0 2024-05-14T15:36:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.402+0000 7f537ffff640 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 0x559a131ad6b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f536c009b00 tx=0x7f536c016f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b1600051666e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f535ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f535ffff640 1 -- 172.21.15.5:0/2715848149 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c00fd70 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f535ffff640 1 -- 172.21.15.5:0/2715848149 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f536c004050 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537802fd40 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f535ffff640 1 -- 172.21.15.5:0/2715848149 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f536c029050 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f538dded740 10 client.27668 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a131bc000 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f535ffff640 1 -- 172.21.15.5:0/2715848149 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f537802fd40 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f535ffff640 1 -- 172.21.15.5:0/2715848149 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f536c0161f0 con 0x559a12de14d0 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f538dded740 10 client.27668 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f538dded740 10 client.27668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.404+0000 7f538dded740 20 client.27668 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.405+0000 7f538dded740 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12dc0d80 0x559a131bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.405+0000 7f538dded740 4 client.27668 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.405+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a131161e0 con 0x559a12dc0d80 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.405+0000 7f5384cd4640 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12dc0d80 0x559a131bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.405+0000 7f535ffff640 10 client.27668 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.405+0000 7f5384cd4640 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12dc0d80 0x559a131bf4d0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3838aa19405fce server_cookie=9e30e5fcbfd55ff1 in_seq=0 out_seq=0 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.407+0000 7f535ffff640 1 -- 172.21.15.5:0/2715848149 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a131161e0 con 0x559a12dc0d80 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.407+0000 7f535ffff640 10 client.27668 handle_command_reply: tid=0 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.407+0000 7f538dded740 1 client.27668 shutdown 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 msgr2=0x559a131ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 0x559a131ad6b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f536c009b00 tx=0x7f536c016f00 comp rx=0 tx=0).stop 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 shutdown_connections 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12dc0d80 0x559a131bf4d0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12de14d0 0x559a131ad6b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 --2- 172.21.15.5:0/2715848149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 >> 172.21.15.5:0/2715848149 conn(0x559a12df6300 msgr2=0x559a131a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f535ffff640 0 client.27668 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 shutdown_connections 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 1 -- 172.21.15.5:0/2715848149 wait complete. 2024-05-14T15:36:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.408+0000 7f538dded740 20 client.27668 trim_cache size 0 max 16384 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 Processor -- start 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 -- start start 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12dd92f0 0x559a131aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53801b1a70 con 0x559a12e021b0 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53801af340 con 0x559a12dd92f0 2024-05-14T15:36:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.428+0000 7f538dded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53801ad190 con 0x559a12de14d0 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f53854d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12dd92f0 0x559a131aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57088/0 (socket says 172.21.15.5:57088) 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f53854d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42850/0 (socket says 172.21.15.5:42850) 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 -- 172.21.15.5:0/695462099 learned_addr learned my addr 172.21.15.5:0/695462099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 -- 172.21.15.5:0/695462099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12dd92f0 msgr2=0x559a131aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12dd92f0 0x559a131aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 -- 172.21.15.5:0/695462099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 msgr2=0x559a131c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 -- 172.21.15.5:0/695462099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1319a9a0 con 0x559a12de14d0 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f537ffff640 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131c4c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c0037f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37ae355e3ddd037 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.429+0000 7f536a7fc640 1 -- 172.21.15.5:0/695462099 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c026040 con 0x559a12de14d0 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.430+0000 7f536a7fc640 1 -- 172.21.15.5:0/695462099 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c00fd70 con 0x559a12de14d0 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.430+0000 7f536a7fc640 1 -- 172.21.15.5:0/695462099 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c003e50 con 0x559a12de14d0 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.430+0000 7f538dded740 1 -- 172.21.15.5:0/695462099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 msgr2=0x559a131c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.430+0000 7f538dded740 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131c4c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c0037f0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 -- 172.21.15.5:0/695462099 shutdown_connections 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12de14d0 0x559a131c4c40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12e021b0 0x559a131c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 --2- 172.21.15.5:0/695462099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12dd92f0 0x559a131aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 -- 172.21.15.5:0/695462099 >> 172.21.15.5:0/695462099 conn(0x559a12df6300 msgr2=0x559a131ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 -- 172.21.15.5:0/695462099 shutdown_connections 2024-05-14T15:36:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 -- 172.21.15.5:0/695462099 wait complete. 2024-05-14T15:36:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 -- start start 2024-05-14T15:36:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.431+0000 7f538dded740 1 Processor -- start 2024-05-14T15:36:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12de14d0 0x559a131b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53801ad800 con 0x559a12de14d0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53801aec70 con 0x559a12e021b0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53801aded0 con 0x559a12dd92f0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f538dded740 10 client.0 _ll_register_callbacks cb 0x7f536c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f537ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12de14d0 0x559a131b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:57098/0 (socket says 172.21.15.5:57098) 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 -- 172.21.15.5:0/3460245752 learned_addr learned my addr 172.21.15.5:0/3460245752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f53854d5640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 -- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 msgr2=0x559a131b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 -- 172.21.15.5:0/3460245752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12de14d0 msgr2=0x559a131b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12de14d0 0x559a131b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 -- 172.21.15.5:0/3460245752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a131b1410 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f537ffff640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12de14d0 0x559a131b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f5384cd4640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131b5540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f537400c6a0 tx=0x7f53740048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb984691a623e9d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.432+0000 7f535ffff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f535ffff640 1 -- 172.21.15.5:0/3460245752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53740c0070 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f535ffff640 1 -- 172.21.15.5:0/3460245752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9474+0+0 (secure 0 0 0) 0x7f53740a5fb0 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c00fd70 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f535ffff640 1 -- 172.21.15.5:0/3460245752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5374005180 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 10 client.27643 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a13227650 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f535ffff640 1 -- 172.21.15.5:0/3460245752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c00fd70 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f535ffff640 1 -- 172.21.15.5:0/3460245752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f53740c3020 con 0x559a12dd92f0 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 10 client.27643 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 10 client.27643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 20 client.27643 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12d961c0 0x559a13229db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 4 client.27643 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.433+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a13110050 con 0x559a12d961c0 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.434+0000 7f537ffff640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12d961c0 0x559a13229db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.434+0000 7f535ffff640 10 client.27643 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.434+0000 7f537ffff640 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12d961c0 0x559a13229db0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4289094f5b6168ae server_cookie=83de786ff5f2732f in_seq=0 out_seq=0 2024-05-14T15:36:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f535ffff640 1 -- 172.21.15.5:0/3460245752 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a13110050 con 0x559a12d961c0 2024-05-14T15:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f535ffff640 10 client.27643 handle_command_reply: tid=0 2024-05-14T15:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 client.27643 shutdown 2024-05-14T15:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 msgr2=0x559a131b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131b5540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f537400c6a0 tx=0x7f53740048f0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 shutdown_connections 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559a12d961c0 0x559a13229db0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a12e021b0 0x559a131b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12de14d0 0x559a131b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 --2- 172.21.15.5:0/3460245752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a12dd92f0 0x559a131b5540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 >> 172.21.15.5:0/3460245752 conn(0x559a12df6300 msgr2=0x559a131ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.836+0000 7f535ffff640 0 client.27643 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.837+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 shutdown_connections 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.837+0000 7f538dded740 1 -- 172.21.15.5:0/3460245752 wait complete. 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.837+0000 7f538dded740 20 client.27643 trim_cache size 0 max 16384 2024-05-14T15:36:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:36:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f53700b2950 msgr2=0x7f53700b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f53700b2950 0x7f53700b4e60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f53780094e0 tx=0x7f537804d000 comp rx=0 tx=0).stop 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 msgr2=0x7f53801a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 0x7f53801a0540 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f537400e9f0 tx=0x7f537400edf0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 shutdown_connections 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f53700b2950 0x7f53700b4e60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538010b880 0x7f53801a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f538010e230 0x7f538019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 --2- 172.21.15.5:0/3511059020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5380104e20 0x7f53801a0540 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.839+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 >> 172.21.15.5:0/3511059020 conn(0x7f53800782e0 msgr2=0x7f53801012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.840+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 shutdown_connections 2024-05-14T15:36:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:04.840+0000 7f5386f38640 1 -- 172.21.15.5:0/3511059020 wait complete. 2024-05-14T15:36:05.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1041554 9.34 MB/sec execute 446 sec latency 299.237 ms 2024-05-14T15:36:06.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:05 smithi005 ceph-mon[31559]: pgmap v354: 129 pgs: 129 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-05-14T15:36:06.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1042854 9.33 MB/sec execute 447 sec latency 71.963 ms 2024-05-14T15:36:06.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:05 smithi144 ceph-mon[35650]: pgmap v354: 129 pgs: 129 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-05-14T15:36:06.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:05 smithi193 ceph-mon[36001]: pgmap v354: 129 pgs: 129 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-05-14T15:36:07.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1043329 9.31 MB/sec execute 448 sec latency 103.214 ms 2024-05-14T15:36:07.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:06 smithi005 ceph-mon[31559]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T15:36:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:06 smithi144 ceph-mon[35650]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T15:36:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:06 smithi193 ceph-mon[36001]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T15:36:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.445+0000 7f0c38b46700 1 -- 172.21.15.5:0/3022758062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 msgr2=0x7f0c34117c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.445+0000 7f0c38b46700 1 --2- 172.21.15.5:0/3022758062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 0x7f0c34117c00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009f90 tx=0x7f0c2c00b330 comp rx=0 tx=0).stop 2024-05-14T15:36:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.448+0000 7f0c38b46700 1 -- 172.21.15.5:0/3022758062 shutdown_connections 2024-05-14T15:36:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.448+0000 7f0c38b46700 1 --2- 172.21.15.5:0/3022758062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 0x7f0c34117c00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.448+0000 7f0c38b46700 1 --2- 172.21.15.5:0/3022758062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3410b020 0x7f0c3410b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.448+0000 7f0c38b46700 1 --2- 172.21.15.5:0/3022758062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410f950 0x7f0c3410fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.448+0000 7f0c38b46700 1 -- 172.21.15.5:0/3022758062 >> 172.21.15.5:0/3022758062 conn(0x7f0c3406f990 msgr2=0x7f0c3406fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.452+0000 7f0c38b46700 1 -- 172.21.15.5:0/3022758062 shutdown_connections 2024-05-14T15:36:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.454+0000 7f0c38b46700 1 -- 172.21.15.5:0/3022758062 wait complete. 2024-05-14T15:36:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.455+0000 7f0c38b46700 1 Processor -- start 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 -- start start 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 0x7f0c341b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3410f950 0x7f0c341b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 0x7f0c341b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3412bb00 con 0x7f0c3410b020 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3412b980 con 0x7f0c3410f950 2024-05-14T15:36:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.460+0000 7f0c38b46700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3412bc80 con 0x7f0c341102a0 2024-05-14T15:36:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.463+0000 7f0c3259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 0x7f0c341b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.463+0000 7f0c3259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 0x7f0c341b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58408/0 (socket says 172.21.15.5:58408) 2024-05-14T15:36:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.463+0000 7f0c3259c700 1 -- 172.21.15.5:0/140861973 learned_addr learned my addr 172.21.15.5:0/140861973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:07.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.465+0000 7f0c32d9d700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 0x7f0c341b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.467+0000 7f0c3259c700 1 -- 172.21.15.5:0/140861973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 msgr2=0x7f0c341b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.467+0000 7f0c3259c700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 0x7f0c341b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.467+0000 7f0c3259c700 1 -- 172.21.15.5:0/140861973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3410f950 msgr2=0x7f0c341b29d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T15:36:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.467+0000 7f0c3259c700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3410f950 0x7f0c341b29d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.467+0000 7f0c3259c700 1 -- 172.21.15.5:0/140861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c341b7300 con 0x7f0c3410b020 2024-05-14T15:36:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.468+0000 7f0c3259c700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 0x7f0c341b2430 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400b550 tx=0x7f0c2400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:07.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.469+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24015020 con 0x7f0c3410b020 2024-05-14T15:36:07.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.469+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c24011ce0 con 0x7f0c3410b020 2024-05-14T15:36:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.470+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c341b75f0 con 0x7f0c3410b020 2024-05-14T15:36:07.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.470+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341bef10 con 0x7f0c3410b020 2024-05-14T15:36:07.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.473+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c34113330 con 0x7f0c3410b020 2024-05-14T15:36:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.475+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24010dd0 con 0x7f0c3410b020 2024-05-14T15:36:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.475+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0c24018020 con 0x7f0c3410b020 2024-05-14T15:36:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.476+0000 7f0c237fe700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c1c0b2ae0 0x7f0c1c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.476+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f0c24011990 con 0x7f0c3410b020 2024-05-14T15:36:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.477+0000 7f0c31d9b700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c1c0b2ae0 0x7f0c1c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.480+0000 7f0c31d9b700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c1c0b2ae0 0x7f0c1c0b4ff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406b550 tx=0x7f0c28009310 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:07.489+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c2401f020 con 0x7f0c3410b020 2024-05-14T15:36:08.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.012+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c340512c0 con 0x7f0c3410b020 2024-05-14T15:36:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.018+0000 7f0c237fe700 1 -- 172.21.15.5:0/140861973 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11864 (secure 0 0 0) 0x7f0c240a97c0 con 0x7f0c3410b020 2024-05-14T15:36:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:36:08.020 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:35:24.885756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27547":{"gid":27547,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3068356254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3068356254},{"type":"v1","addr":"172.21.15.144:6835","nonce":3068356254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:36:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.022+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c1c0b2ae0 msgr2=0x7f0c1c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.022+0000 7f0c38b46700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c1c0b2ae0 0x7f0c1c0b4ff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406b550 tx=0x7f0c28009310 comp rx=0 tx=0).stop 2024-05-14T15:36:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 msgr2=0x7f0c341b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 0x7f0c341b2430 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400b550 tx=0x7f0c2400b910 comp rx=0 tx=0).stop 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 shutdown_connections 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f0c1c0b2ae0 0x7f0c1c0b4ff0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c341102a0 0x7f0c341b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3410b020 0x7f0c341b2430 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 --2- 172.21.15.5:0/140861973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3410f950 0x7f0c341b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 >> 172.21.15.5:0/140861973 conn(0x7f0c3406f990 msgr2=0x7f0c3410d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 shutdown_connections 2024-05-14T15:36:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:08.023+0000 7f0c38b46700 1 -- 172.21.15.5:0/140861973 wait complete. 2024-05-14T15:36:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-14T15:36:08.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1043853 9.31 MB/sec execute 449 sec latency 66.782 ms 2024-05-14T15:36:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:07 smithi144 ceph-mon[35650]: pgmap v356: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 274 op/s 2024-05-14T15:36:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:07 smithi193 ceph-mon[36001]: pgmap v356: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 274 op/s 2024-05-14T15:36:08.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:07 smithi005 ceph-mon[31559]: pgmap v356: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 94 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 274 op/s 2024-05-14T15:36:08.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:36:09.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1044242 9.29 MB/sec execute 450 sec latency 138.945 ms 2024-05-14T15:36:09.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:08 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/140861973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:36:09.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:08 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/140861973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:36:09.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:08 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/140861973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:36:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:10.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1044549 9.28 MB/sec execute 451 sec latency 91.207 ms 2024-05-14T15:36:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:09 smithi144 ceph-mon[35650]: pgmap v357: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 82 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:36:10.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:09 smithi193 ceph-mon[36001]: pgmap v357: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 82 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:36:10.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:09 smithi005 ceph-mon[31559]: pgmap v357: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 82 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:36:11.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1044826 9.26 MB/sec execute 452 sec latency 157.585 ms 2024-05-14T15:36:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.517+0000 7fcb505a8700 1 -- 172.21.15.5:0/3385649987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4810a930 msgr2=0x7fcb4810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.517+0000 7fcb505a8700 1 --2- 172.21.15.5:0/3385649987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4810a930 0x7fcb4810ad40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb380098f0 tx=0x7fcb38009c00 comp rx=0 tx=0).stop 2024-05-14T15:36:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.518+0000 7fcb505a8700 1 -- 172.21.15.5:0/3385649987 shutdown_connections 2024-05-14T15:36:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.518+0000 7fcb505a8700 1 --2- 172.21.15.5:0/3385649987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4811d930 0x7fcb4811fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.518+0000 7fcb505a8700 1 --2- 172.21.15.5:0/3385649987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4810a930 0x7fcb4810ad40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.518+0000 7fcb505a8700 1 --2- 172.21.15.5:0/3385649987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810b280 0x7fcb4811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.518+0000 7fcb505a8700 1 -- 172.21.15.5:0/3385649987 >> 172.21.15.5:0/3385649987 conn(0x7fcb4806f4d0 msgr2=0x7fcb4806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.520+0000 7fcb505a8700 1 -- 172.21.15.5:0/3385649987 shutdown_connections 2024-05-14T15:36:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.520+0000 7fcb505a8700 1 -- 172.21.15.5:0/3385649987 wait complete. 2024-05-14T15:36:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.521+0000 7fcb505a8700 1 Processor -- start 2024-05-14T15:36:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.521+0000 7fcb505a8700 1 -- start start 2024-05-14T15:36:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.522+0000 7fcb505a8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810a930 0x7fcb481a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.522+0000 7fcb505a8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4810b280 0x7fcb481aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.522+0000 7fcb505a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4811d930 0x7fcb481ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb4e344700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810a930 0x7fcb481a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb505a8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481233a0 con 0x7fcb4810a930 2024-05-14T15:36:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb4e344700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810a930 0x7fcb481a9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58438/0 (socket says 172.21.15.5:58438) 2024-05-14T15:36:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb4e344700 1 -- 172.21.15.5:0/1622497799 learned_addr learned my addr 172.21.15.5:0/1622497799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb4eb45700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4811d930 0x7fcb481ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48123220 con 0x7fcb4811d930 2024-05-14T15:36:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.523+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48123520 con 0x7fcb4810b280 2024-05-14T15:36:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.525+0000 7fcb4eb45700 1 -- 172.21.15.5:0/1622497799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4810b280 msgr2=0x7fcb481aa2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T15:36:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.525+0000 7fcb4eb45700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4810b280 0x7fcb481aa2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.525+0000 7fcb4eb45700 1 -- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810a930 msgr2=0x7fcb481a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.525+0000 7fcb4eb45700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810a930 0x7fcb481a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.525+0000 7fcb4eb45700 1 -- 172.21.15.5:0/1622497799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb481aebe0 con 0x7fcb4811d930 2024-05-14T15:36:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.525+0000 7fcb4eb45700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4811d930 0x7fcb481ae540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000bdf0 tx=0x7fcb40009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.526+0000 7fcb477fe700 1 -- 172.21.15.5:0/1622497799 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40019070 con 0x7fcb4811d930 2024-05-14T15:36:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.526+0000 7fcb477fe700 1 -- 172.21.15.5:0/1622497799 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb4000cc10 con 0x7fcb4811d930 2024-05-14T15:36:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.526+0000 7fcb477fe700 1 -- 172.21.15.5:0/1622497799 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400125a0 con 0x7fcb4811d930 2024-05-14T15:36:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.526+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb481aeed0 con 0x7fcb4811d930 2024-05-14T15:36:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.526+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481b67f0 con 0x7fcb4811d930 2024-05-14T15:36:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.532+0000 7fcb5efbcb80 1 Processor -- start 2024-05-14T15:36:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 -- start start 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb477fe700 1 -- 172.21.15.5:0/1622497799 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcb4001c070 con 0x7fcb4811d930 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880287720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880287c60 0x55688028bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x55688028c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568801a9860 con 0x556880287c60 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5568801487c0 con 0x556880109e40 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5568801a5820 con 0x55688028c470 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.535+0000 7fcb4db43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880287720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.536+0000 7fcb4db43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880287720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:50818/0 (socket says 172.21.15.5:50818) 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.536+0000 7fcb4db43700 1 -- 172.21.15.5:0/2184007141 learned_addr learned my addr 172.21.15.5:0/2184007141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.536+0000 7fcb4db43700 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x55688028c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.536+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880287c60 0x55688028bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.539+0000 7fcb477fe700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb300b2c00 0x7fcb300b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.540+0000 7fcb477fe700 1 -- 172.21.15.5:0/1622497799 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7fcb400e1fd0 con 0x7fcb4811d930 2024-05-14T15:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.540+0000 7fcb4e344700 1 -- 172.21.15.5:0/2184007141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 msgr2=0x55688028c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.540+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x55688028c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.540+0000 7fcb4e344700 1 -- 172.21.15.5:0/2184007141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 msgr2=0x556880287720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.540+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880287720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.540+0000 7fcb4e344700 1 -- 172.21.15.5:0/2184007141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568800f68d0 con 0x556880287c60 2024-05-14T15:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.541+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880287c60 0x55688028bf30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcb380098c0 tx=0x7fcb3800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e184bc581d91a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.542+0000 7fcb44ff9700 1 -- 172.21.15.5:0/2184007141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38026070 con 0x556880287c60 2024-05-14T15:36:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.542+0000 7fcb44ff9700 1 -- 172.21.15.5:0/2184007141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb38005040 con 0x556880287c60 2024-05-14T15:36:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.542+0000 7fcb4eb45700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb300b2c00 0x7fcb300b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.542+0000 7fcb44ff9700 1 -- 172.21.15.5:0/2184007141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb380038e0 con 0x556880287c60 2024-05-14T15:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2184007141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880287c60 msgr2=0x55688028bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880287c60 0x55688028bf30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcb380098c0 tx=0x7fcb3800bf60 comp rx=0 tx=0).stop 2024-05-14T15:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2184007141 shutdown_connections 2024-05-14T15:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x55688028c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880287720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2184007141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880287c60 0x55688028bf30 secure :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fcb380098c0 tx=0x7fcb3800bf60 comp rx=0 tx=0).stop 2024-05-14T15:36:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.543+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2184007141 >> 172.21.15.5:0/2184007141 conn(0x556880151600 msgr2=0x5568802824b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.544+0000 7fcb4eb45700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb300b2c00 0x7fcb300b5110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000bfd0 tx=0x7fcb4000e040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.544+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2184007141 shutdown_connections 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.544+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2184007141 wait complete. 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 -- start start 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 Processor -- start 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880291170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x55688029d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568802943c0 con 0x556880291bf0 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55688010fc30 con 0x556880109e40 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556880294540 con 0x55688028c470 2024-05-14T15:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.545+0000 7fcb5efbcb80 10 client.0 _ll_register_callbacks cb 0x556880141920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:11.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802916b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:56104/0 (socket says 172.21.15.5:56104) 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 -- 172.21.15.5:0/2019381354 learned_addr learned my addr 172.21.15.5:0/2019381354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 -- 172.21.15.5:0/2019381354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 msgr2=0x556880291170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880291170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 -- 172.21.15.5:0/2019381354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 msgr2=0x55688029d8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x55688029d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.546+0000 7fcb4e344700 1 -- 172.21.15.5:0/2019381354 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556880145690 con 0x55688028c470 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.547+0000 7fcb4e344700 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802916b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009c40 tx=0x7fcb38005690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8d2a97899c926d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.547+0000 7fcb2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.547+0000 7fcb2dffb700 1 -- 172.21.15.5:0/2019381354 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38030070 con 0x55688028c470 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.547+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb38005040 con 0x55688028c470 2024-05-14T15:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.547+0000 7fcb2dffb700 1 -- 172.21.15.5:0/2019381354 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7fcb38003680 con 0x55688028c470 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb2dffb700 1 -- 172.21.15.5:0/2019381354 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb38039050 con 0x55688028c470 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 10 client.27655 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5568802943c0 con 0x55688028c470 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb2dffb700 1 -- 172.21.15.5:0/2019381354 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb38005040 con 0x55688028c470 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb2dffb700 1 -- 172.21.15.5:0/2019381354 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcb38033020 con 0x55688028c470 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 10 client.27655 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 10 client.27655 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 20 client.27655 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5568802a17d0 0x5568802a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 4 client.27655 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.548+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568801b7240 con 0x5568802a17d0 2024-05-14T15:36:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.549+0000 7fcb4db43700 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5568802a17d0 0x5568802a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.549+0000 7fcb4db43700 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5568802a17d0 0x5568802a3bf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8802a1b0298223ee server_cookie=3ad1df6a660122d in_seq=0 out_seq=0 2024-05-14T15:36:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.549+0000 7fcb2dffb700 10 client.27655 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.550+0000 7fcb2dffb700 1 -- 172.21.15.5:0/2019381354 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568801b7240 con 0x5568802a17d0 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.550+0000 7fcb2dffb700 10 client.27655 handle_command_reply: tid=0 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.550+0000 7fcb5efbcb80 1 client.27655 shutdown 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.551+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 msgr2=0x5568802916b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.551+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802916b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009c40 tx=0x7fcb38005690 comp rx=0 tx=0).stop 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 shutdown_connections 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x55688029d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x5568802a17d0 0x5568802a3bf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x556880291170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/2019381354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802916b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 >> 172.21.15.5:0/2019381354 conn(0x556880151600 msgr2=0x556880286450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb2dffb700 0 client.27655 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 shutdown_connections 2024-05-14T15:36:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.552+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/2019381354 wait complete. 2024-05-14T15:36:11.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.553+0000 7fcb5efbcb80 20 client.27655 trim_cache size 0 max 16384 2024-05-14T15:36:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.581+0000 7fcb5efbcb80 1 Processor -- start 2024-05-14T15:36:11.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.584+0000 7fcb5efbcb80 1 -- start start 2024-05-14T15:36:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.585+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x5568802900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.585+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.585+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x5568802a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.585+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481c1f90 con 0x556880291bf0 2024-05-14T15:36:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.585+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481bf820 con 0x556880109e40 2024-05-14T15:36:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.585+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481bd690 con 0x55688028c470 2024-05-14T15:36:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.587+0000 7fcb4e344700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.588+0000 7fcb4e344700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:56120/0 (socket says 172.21.15.5:56120) 2024-05-14T15:36:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.588+0000 7fcb4e344700 1 -- 172.21.15.5:0/193661199 learned_addr learned my addr 172.21.15.5:0/193661199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.588+0000 7fcb4db43700 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x5568802900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.588+0000 7fcb4db43700 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x5568802a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.589+0000 7fcb4e344700 1 -- 172.21.15.5:0/193661199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 msgr2=0x5568802900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.589+0000 7fcb4e344700 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x5568802900d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.589+0000 7fcb4e344700 1 -- 172.21.15.5:0/193661199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 msgr2=0x5568802a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.589+0000 7fcb4e344700 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x5568802a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.589+0000 7fcb4e344700 1 -- 172.21.15.5:0/193661199 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688028cc40 con 0x55688028c470 2024-05-14T15:36:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.589+0000 7fcb4e344700 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802822f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009ff0 tx=0x7fcb38005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a806f53425708f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.591+0000 7fcb2ffff700 1 -- 172.21.15.5:0/193661199 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38030070 con 0x55688028c470 2024-05-14T15:36:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.591+0000 7fcb2ffff700 1 -- 172.21.15.5:0/193661199 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3800f7f0 con 0x55688028c470 2024-05-14T15:36:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.591+0000 7fcb2ffff700 1 -- 172.21.15.5:0/193661199 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38026040 con 0x55688028c470 2024-05-14T15:36:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.593+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/193661199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 msgr2=0x5568802822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.593+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802822f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009ff0 tx=0x7fcb38005fa0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.596+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/193661199 shutdown_connections 2024-05-14T15:36:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.596+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880291bf0 0x5568802a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.596+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880109e40 0x5568802900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.596+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/193661199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x5568802822f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.596+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/193661199 >> 172.21.15.5:0/193661199 conn(0x556880151600 msgr2=0x556880281be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.598+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/193661199 shutdown_connections 2024-05-14T15:36:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.599+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/193661199 wait complete. 2024-05-14T15:36:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.599+0000 7fcb5efbcb80 1 -- start start 2024-05-14T15:36:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.599+0000 7fcb5efbcb80 1 Processor -- start 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.601+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880109e40 0x556880299a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.601+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x556880296db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.602+0000 7fcb5efbcb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880291bf0 0x5568802972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.602+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481bdce0 con 0x556880109e40 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.602+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481bf150 con 0x556880291bf0 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.602+0000 7fcb5efbcb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481be3b0 con 0x55688028c470 2024-05-14T15:36:11.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.602+0000 7fcb5efbcb80 10 client.0 _ll_register_callbacks cb 0x556880141920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.603+0000 7fcb4db43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880291bf0 0x5568802972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.603+0000 7fcb4db43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880109e40 0x556880299a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.603+0000 7fcb4db43700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880109e40 0x556880299a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58478/0 (socket says 172.21.15.5:58478) 2024-05-14T15:36:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.603+0000 7fcb4db43700 1 -- 172.21.15.5:0/1812910546 learned_addr learned my addr 172.21.15.5:0/1812910546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.603+0000 7fcb4e344700 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x556880296db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.608+0000 7fcb4db43700 1 -- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 msgr2=0x556880296db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.608+0000 7fcb4db43700 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x556880296db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.608+0000 7fcb4db43700 1 -- 172.21.15.5:0/1812910546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880109e40 msgr2=0x556880299a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.608+0000 7fcb4db43700 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880109e40 0x556880299a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.608+0000 7fcb4db43700 1 -- 172.21.15.5:0/1812910546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556880297830 con 0x556880291bf0 2024-05-14T15:36:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.608+0000 7fcb4db43700 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880291bf0 0x5568802972f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c005920 tx=0x7fcb3c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275d20a1889ebedc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.609+0000 7fcb2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.611+0000 7fcb2dffb700 1 -- 172.21.15.5:0/1812910546 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c020020 con 0x556880291bf0 2024-05-14T15:36:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.612+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3800f7f0 con 0x556880291bf0 2024-05-14T15:36:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.612+0000 7fcb2dffb700 1 -- 172.21.15.5:0/1812910546 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7fcb3c01c040 con 0x556880291bf0 2024-05-14T15:36:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.612+0000 7fcb2dffb700 1 -- 172.21.15.5:0/1812910546 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb3c030050 con 0x556880291bf0 2024-05-14T15:36:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.613+0000 7fcb5efbcb80 10 client.27689 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.613+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55688030b980 con 0x556880291bf0 2024-05-14T15:36:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.614+0000 7fcb2dffb700 1 -- 172.21.15.5:0/1812910546 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3800f7f0 con 0x556880291bf0 2024-05-14T15:36:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.614+0000 7fcb2dffb700 1 -- 172.21.15.5:0/1812910546 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcb3c003680 con 0x556880291bf0 2024-05-14T15:36:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.617+0000 7fcb5efbcb80 10 client.27689 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.617+0000 7fcb5efbcb80 10 client.27689 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:36:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.617+0000 7fcb5efbcb80 20 client.27689 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.617+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55688030c010 0x55688030e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.617+0000 7fcb5efbcb80 4 client.27689 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:36:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.617+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568801b12e0 con 0x55688030c010 2024-05-14T15:36:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.618+0000 7fcb4e344700 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55688030c010 0x55688030e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.627+0000 7fcb4e344700 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55688030c010 0x55688030e450 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d68a69a0d8197fb server_cookie=fc3877484d60694f in_seq=0 out_seq=0 2024-05-14T15:36:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.627+0000 7fcb2dffb700 10 client.27689 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.627+0000 7fcb2dffb700 1 -- 172.21.15.5:0/1812910546 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568801b12e0 con 0x55688030c010 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.627+0000 7fcb2dffb700 10 client.27689 handle_command_reply: tid=0 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.628+0000 7fcb5efbcb80 1 client.27689 shutdown 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880291bf0 msgr2=0x5568802972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880291bf0 0x5568802972f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c005920 tx=0x7fcb3c013040 comp rx=0 tx=0).stop 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 shutdown_connections 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55688030c010 0x55688030e450 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556880291bf0 0x5568802972f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556880109e40 0x556880299a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 --2- 172.21.15.5:0/1812910546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55688028c470 0x556880296db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 >> 172.21.15.5:0/1812910546 conn(0x556880151600 msgr2=0x556880286450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.629+0000 7fcb2dffb700 0 client.27689 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.632+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 shutdown_connections 2024-05-14T15:36:11.634 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:36:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.632+0000 7fcb5efbcb80 1 -- 172.21.15.5:0/1812910546 wait complete. 2024-05-14T15:36:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.632+0000 7fcb5efbcb80 20 client.27689 trim_cache size 0 max 16384 2024-05-14T15:36:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.634+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb300b2c00 msgr2=0x7fcb300b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.634+0000 7fcb505a8700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb300b2c00 0x7fcb300b5110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000bfd0 tx=0x7fcb4000e040 comp rx=0 tx=0).stop 2024-05-14T15:36:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.634+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4811d930 msgr2=0x7fcb481ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.634+0000 7fcb505a8700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4811d930 0x7fcb481ae540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000bdf0 tx=0x7fcb40009ed0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.636+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 shutdown_connections 2024-05-14T15:36:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.636+0000 7fcb505a8700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fcb300b2c00 0x7fcb300b5110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.636+0000 7fcb505a8700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb4811d930 0x7fcb481ae540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.636+0000 7fcb505a8700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb4810a930 0x7fcb481a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.636+0000 7fcb505a8700 1 --2- 172.21.15.5:0/1622497799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4810b280 0x7fcb481aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.636+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 >> 172.21.15.5:0/1622497799 conn(0x7fcb4806f4d0 msgr2=0x7fcb4811b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.637+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 shutdown_connections 2024-05-14T15:36:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:11.637+0000 7fcb505a8700 1 -- 172.21.15.5:0/1622497799 wait complete. 2024-05-14T15:36:12.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1045329 9.24 MB/sec execute 453 sec latency 48.356 ms 2024-05-14T15:36:12.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:11 smithi144 ceph-mon[35650]: pgmap v358: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 82 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:36:12.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:11 smithi193 ceph-mon[36001]: pgmap v358: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 82 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:36:12.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:11 smithi005 ceph-mon[31559]: pgmap v358: 129 pgs: 19 active+clean+snaptrim, 52 active+clean+snaptrim_wait, 58 active+clean; 82 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 334 op/s 2024-05-14T15:36:13.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1046307 9.23 MB/sec execute 454 sec latency 115.026 ms 2024-05-14T15:36:13.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:36:14.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:13 smithi005 ceph-mon[31559]: pgmap v359: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 65 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-05-14T15:36:14.109 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:14.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1048101 9.24 MB/sec execute 455 sec latency 110.700 ms 2024-05-14T15:36:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:13 smithi144 ceph-mon[35650]: pgmap v359: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 65 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-05-14T15:36:14.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:13 smithi193 ceph-mon[36001]: pgmap v359: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 65 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-05-14T15:36:15.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1049244 9.22 MB/sec execute 456 sec latency 29.485 ms 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.041+0000 7f5ffa297700 1 -- 172.21.15.5:0/3128691394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410a930 msgr2=0x7f5ff410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.041+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3128691394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410a930 0x7f5ff410ad40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b0a0 tx=0x7f5fec00b3b0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 -- 172.21.15.5:0/3128691394 shutdown_connections 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3128691394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff410b280 0x7f5ff41198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3128691394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff4111bd0 0x7f5ff4111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3128691394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410a930 0x7f5ff410ad40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 -- 172.21.15.5:0/3128691394 >> 172.21.15.5:0/3128691394 conn(0x7f5ff406f4d0 msgr2=0x7f5ff406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 -- 172.21.15.5:0/3128691394 shutdown_connections 2024-05-14T15:36:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.042+0000 7f5ffa297700 1 -- 172.21.15.5:0/3128691394 wait complete. 2024-05-14T15:36:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.044+0000 7f5ffa297700 1 Processor -- start 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.044+0000 7f5ffa297700 1 -- start start 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.045+0000 7f5ffa297700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff410a930 0x7f5ff410e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.045+0000 7f5ffa297700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410b280 0x7f5ff410c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.045+0000 7f5ffa297700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff4111bd0 0x7f5ff410cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.045+0000 7f5ffa297700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff411cd40 con 0x7f5ff4111bd0 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.045+0000 7f5ffa297700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff411cbc0 con 0x7f5ff410b280 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.045+0000 7f5ffa297700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff411cec0 con 0x7f5ff410a930 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.046+0000 7f5ff8834700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff4111bd0 0x7f5ff410cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.046+0000 7f5ff8834700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff4111bd0 0x7f5ff410cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60226/0 (socket says 172.21.15.5:60226) 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.046+0000 7f5ff8834700 1 -- 172.21.15.5:0/3270532539 learned_addr learned my addr 172.21.15.5:0/3270532539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.046+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410b280 0x7f5ff410c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.046+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff410a930 0x7f5ff410e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.047+0000 7f5ff3fff700 1 -- 172.21.15.5:0/3270532539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410b280 msgr2=0x7f5ff410c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.047+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410b280 0x7f5ff410c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.047+0000 7f5ff3fff700 1 -- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff4111bd0 msgr2=0x7f5ff410cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.047+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff4111bd0 0x7f5ff410cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.047+0000 7f5ff3fff700 1 -- 172.21.15.5:0/3270532539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff410d4b0 con 0x7f5ff410a930 2024-05-14T15:36:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.047+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff410a930 0x7f5ff410e220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e9c0 tx=0x7f5fe400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.048+0000 7f5ff17fa700 1 -- 172.21.15.5:0/3270532539 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4004220 con 0x7f5ff410a930 2024-05-14T15:36:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.048+0000 7f5ff17fa700 1 -- 172.21.15.5:0/3270532539 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40043c0 con 0x7f5ff410a930 2024-05-14T15:36:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.048+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41a7a20 con 0x7f5ff410a930 2024-05-14T15:36:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.048+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a7f60 con 0x7f5ff410a930 2024-05-14T15:36:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.050+0000 7f5ff17fa700 1 -- 172.21.15.5:0/3270532539 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4003cb0 con 0x7f5ff410a930 2024-05-14T15:36:16.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.053+0000 7f5ff17fa700 1 -- 172.21.15.5:0/3270532539 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5fe4018430 con 0x7f5ff410a930 2024-05-14T15:36:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.055+0000 7f5ff17fa700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5fdc0b2bc0 0x7f5fdc0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.055+0000 7f5ff17fa700 1 -- 172.21.15.5:0/3270532539 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f5fe40e3060 con 0x7f5ff410a930 2024-05-14T15:36:16.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.055+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5fdc0b2bc0 0x7f5fdc0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.055+0000 7f6008cabb80 1 Processor -- start 2024-05-14T15:36:16.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.055+0000 7f6008cabb80 1 -- start start 2024-05-14T15:36:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5fdc0b2bc0 0x7f5fdc0b50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ff406fad0 tx=0x7f5fec00b700 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a8156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8156d80 0x5557a81508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8151220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f6008cabb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557a8072940 con 0x5557a7fd2e40 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f6008cabb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5557a7fd7da0 con 0x5557a8156d80 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f6008cabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557a806e900 con 0x5557a8150e10 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.056+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a8156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a8156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60230/0 (socket says 172.21.15.5:60230) 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 -- 172.21.15.5:0/914379720 learned_addr learned my addr 172.21.15.5:0/914379720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8156d80 0x5557a81508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 -- 172.21.15.5:0/914379720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 msgr2=0x5557a8151220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8151220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 -- 172.21.15.5:0/914379720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8156d80 msgr2=0x5557a81508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8156d80 0x5557a81508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 -- 172.21.15.5:0/914379720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a7fbf8d0 con 0x5557a7fd2e40 2024-05-14T15:36:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.057+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a8156840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005cb0 tx=0x7f5fe40aa7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6ec285fb922de5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.058+0000 7f5fda7fc700 1 -- 172.21.15.5:0/914379720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40aa8b0 con 0x5557a7fd2e40 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.058+0000 7f5fda7fc700 1 -- 172.21.15.5:0/914379720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40aec60 con 0x5557a7fd2e40 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.058+0000 7f5fda7fc700 1 -- 172.21.15.5:0/914379720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40b3280 con 0x5557a7fd2e40 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 -- 172.21.15.5:0/914379720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 msgr2=0x5557a8156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a8156840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005cb0 tx=0x7f5fe40aa7b0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 -- 172.21.15.5:0/914379720 shutdown_connections 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a8156840 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 --2- 172.21.15.5:0/914379720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8156d80 0x5557a81508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.059+0000 7f6008cabb80 1 -- 172.21.15.5:0/914379720 >> 172.21.15.5:0/914379720 conn(0x5557a801a630 msgr2=0x5557a814ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.060+0000 7f6008cabb80 1 -- 172.21.15.5:0/914379720 shutdown_connections 2024-05-14T15:36:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.060+0000 7f6008cabb80 1 -- 172.21.15.5:0/914379720 wait complete. 2024-05-14T15:36:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.060+0000 7f6008cabb80 1 -- start start 2024-05-14T15:36:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.061+0000 7f6008cabb80 1 Processor -- start 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8150e10 0x5557a815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8156d80 0x5557a815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557a815d820 con 0x5557a7fd2e40 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5557a802bed0 con 0x5557a8150e10 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557a815d9a0 con 0x5557a8156d80 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a815a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60236/0 (socket says 172.21.15.5:60236) 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 -- 172.21.15.5:0/774661260 learned_addr learned my addr 172.21.15.5:0/774661260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f6008cabb80 10 client.0 _ll_register_callbacks cb 0x5557a800a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff8834700 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8156d80 0x5557a815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 -- 172.21.15.5:0/774661260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8156d80 msgr2=0x5557a815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8156d80 0x5557a815af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 -- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8150e10 msgr2=0x5557a815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.062+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8150e10 0x5557a815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.063+0000 7f5ff3fff700 1 -- 172.21.15.5:0/774661260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a8029760 con 0x5557a7fd2e40 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.063+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a815a490 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c5b0 tx=0x7f5fe40aadf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fff7a0b1b4ea34e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.063+0000 7f5fd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.063+0000 7f5fd37fe700 1 -- 172.21.15.5:0/774661260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40c3070 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.063+0000 7f5fd37fe700 1 -- 172.21.15.5:0/774661260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f5fe40b3490 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.063+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe40aec60 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.064+0000 7f5fd37fe700 1 -- 172.21.15.5:0/774661260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fe40b8050 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.064+0000 7f6008cabb80 10 client.18108 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.064+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5557a81681c0 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.064+0000 7f5fd37fe700 1 -- 172.21.15.5:0/774661260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40aec60 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f5fd37fe700 1 -- 172.21.15.5:0/774661260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f5fe40c9060 con 0x5557a7fd2e40 2024-05-14T15:36:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f6008cabb80 10 client.18108 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f6008cabb80 10 client.18108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f6008cabb80 20 client.18108 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f6008cabb80 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a816a690 0x5557a816cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f6008cabb80 4 client.18108 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a8080320 con 0x5557a816a690 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.065+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a816a690 0x5557a816cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.066+0000 7f5fd37fe700 10 client.18108 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:16.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.066+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a816a690 0x5557a816cab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13324cab49c6416b server_cookie=ed610701c4a5908f in_seq=0 out_seq=0 2024-05-14T15:36:16.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.067+0000 7f5fd37fe700 1 -- 172.21.15.5:0/774661260 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557a8080320 con 0x5557a816a690 2024-05-14T15:36:16.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.067+0000 7f5fd37fe700 10 client.18108 handle_command_reply: tid=0 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.067+0000 7f6008cabb80 1 client.18108 shutdown 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.067+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 msgr2=0x5557a815a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.067+0000 7f6008cabb80 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a815a490 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c5b0 tx=0x7f5fe40aadf0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 shutdown_connections 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a816a690 0x5557a816cab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8156d80 0x5557a815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a7fd2e40 0x5557a815a490 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 --2- 172.21.15.5:0/774661260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a8150e10 0x5557a815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f5fd37fe700 0 client.18108 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 >> 172.21.15.5:0/774661260 conn(0x5557a801a630 msgr2=0x5557a814f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 shutdown_connections 2024-05-14T15:36:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 1 -- 172.21.15.5:0/774661260 wait complete. 2024-05-14T15:36:16.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.068+0000 7f6008cabb80 20 client.18108 trim_cache size 0 max 16384 2024-05-14T15:36:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 Processor -- start 2024-05-14T15:36:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 -- start start 2024-05-14T15:36:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a81508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8156840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a8166c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0011850 con 0x5557a8156d80 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe000f0e0 con 0x5557a7fd2e40 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.088+0000 7f6008cabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe000cf50 con 0x5557a8150e10 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff8834700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a8166c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a81508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff8834700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a8166c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60242/0 (socket says 172.21.15.5:60242) 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a81508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35464/0 (socket says 172.21.15.5:35464) 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff8834700 1 -- 172.21.15.5:0/903400149 learned_addr learned my addr 172.21.15.5:0/903400149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 -- 172.21.15.5:0/903400149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 msgr2=0x5557a8156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 -- 172.21.15.5:0/903400149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 msgr2=0x5557a8166c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a8166c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 -- 172.21.15.5:0/903400149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a815d980 con 0x5557a7fd2e40 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff8834700 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a8166c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a81508d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40aaf10 tx=0x7f5fe40aa150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde120ae56d03206 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.089+0000 7f5fd9ffb700 1 -- 172.21.15.5:0/903400149 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40c3070 con 0x5557a7fd2e40 2024-05-14T15:36:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f5fd9ffb700 1 -- 172.21.15.5:0/903400149 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40aec60 con 0x5557a7fd2e40 2024-05-14T15:36:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f5fd9ffb700 1 -- 172.21.15.5:0/903400149 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40cbb60 con 0x5557a7fd2e40 2024-05-14T15:36:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f6008cabb80 1 -- 172.21.15.5:0/903400149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 msgr2=0x5557a81508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f6008cabb80 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a81508d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40aaf10 tx=0x7f5fe40aa150 comp rx=0 tx=0).stop 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f6008cabb80 1 -- 172.21.15.5:0/903400149 shutdown_connections 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f6008cabb80 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a8166c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f6008cabb80 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a81508d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.090+0000 7f6008cabb80 1 --2- 172.21.15.5:0/903400149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a8156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.091+0000 7f6008cabb80 1 -- 172.21.15.5:0/903400149 >> 172.21.15.5:0/903400149 conn(0x5557a801a630 msgr2=0x5557a8145580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.091+0000 7f6008cabb80 1 -- 172.21.15.5:0/903400149 shutdown_connections 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.091+0000 7f6008cabb80 1 -- 172.21.15.5:0/903400149 wait complete. 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.091+0000 7f6008cabb80 1 -- start start 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.091+0000 7f6008cabb80 1 Processor -- start 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a816cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a816cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a816d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe000d5a0 con 0x5557a8156d80 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe000ea10 con 0x5557a7fd2e40 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe000dc70 con 0x5557a8150e10 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a816cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a816cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 -- 172.21.15.5:0/1837683638 learned_addr learned my addr 172.21.15.5:0/1837683638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff8834700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a816d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a816cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f6008cabb80 10 client.0 _ll_register_callbacks cb 0x5557a800a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 -- 172.21.15.5:0/1837683638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 msgr2=0x5557a816cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a816cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 -- 172.21.15.5:0/1837683638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 msgr2=0x5557a816d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a816d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 -- 172.21.15.5:0/1837683638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a81d2d10 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5ff3fff700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a816cab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40aa840 tx=0x7f5fe4009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2484882e5fff900d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5fd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.092+0000 7f5fd37fe700 1 -- 172.21.15.5:0/1837683638 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40c3070 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f5fd37fe700 1 -- 172.21.15.5:0/1837683638 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f5fe40a6270 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe40aec60 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f5fd37fe700 1 -- 172.21.15.5:0/1837683638 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fe40b3710 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 10 client.27704 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5557a81d2d10 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f5fd37fe700 1 -- 172.21.15.5:0/1837683638 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe4089020 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f5fd37fe700 1 -- 172.21.15.5:0/1837683638 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f5fe40c9070 con 0x5557a7fd2e40 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 10 client.27704 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 10 client.27704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:36:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 20 client.27704 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:16.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:15 smithi005 ceph-mon[31559]: pgmap v360: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 65 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-05-14T15:36:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a81d4290 0x5557a81d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 4 client.27704 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.093+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557a807a3c0 con 0x5557a81d4290 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.094+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a81d4290 0x5557a81d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.094+0000 7f5ff37fe700 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a81d4290 0x5557a81d66b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4d2664b6f2cfafe server_cookie=81560f766ccc2a51 in_seq=0 out_seq=0 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.094+0000 7f5fd37fe700 10 client.27704 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.094+0000 7f5fd37fe700 1 -- 172.21.15.5:0/1837683638 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557a807a3c0 con 0x5557a81d4290 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.094+0000 7f5fd37fe700 10 client.27704 handle_command_reply: tid=0 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.094+0000 7f6008cabb80 1 client.27704 shutdown 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 msgr2=0x5557a816cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a816cab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40aa840 tx=0x7f5fe4009ef0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 shutdown_connections 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5557a81d4290 0x5557a81d66b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557a8156d80 0x5557a816d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5557a7fd2e40 0x5557a816cab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 --2- 172.21.15.5:0/1837683638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557a8150e10 0x5557a816cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 >> 172.21.15.5:0/1837683638 conn(0x5557a801a630 msgr2=0x5557a815ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f5fd37fe700 0 client.27704 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 shutdown_connections 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 1 -- 172.21.15.5:0/1837683638 wait complete. 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.095+0000 7f6008cabb80 20 client.27704 trim_cache size 0 max 16384 2024-05-14T15:36:16.102 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:36:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5fdc0b2bc0 msgr2=0x7f5fdc0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5fdc0b2bc0 0x7f5fdc0b50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ff406fad0 tx=0x7f5fec00b700 comp rx=0 tx=0).stop 2024-05-14T15:36:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff410a930 msgr2=0x7f5ff410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff410a930 0x7f5ff410e220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e9c0 tx=0x7f5fe400ecd0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 shutdown_connections 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f5fdc0b2bc0 0x7f5fdc0b50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ff4111bd0 0x7f5ff410cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ff410a930 0x7f5ff410e220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 --2- 172.21.15.5:0/3270532539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff410b280 0x7f5ff410c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 >> 172.21.15.5:0/3270532539 conn(0x7f5ff406f4d0 msgr2=0x7f5ff4110570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 shutdown_connections 2024-05-14T15:36:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:16.096+0000 7f5ffa297700 1 -- 172.21.15.5:0/3270532539 wait complete. 2024-05-14T15:36:16.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1050860 9.22 MB/sec execute 457 sec latency 81.423 ms 2024-05-14T15:36:16.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:15 smithi144 ceph-mon[35650]: pgmap v360: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 65 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-05-14T15:36:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:15 smithi193 ceph-mon[36001]: pgmap v360: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 65 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2024-05-14T15:36:16.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:36:17.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1052101 9.22 MB/sec execute 458 sec latency 16.522 ms 2024-05-14T15:36:17.185 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 -- 172.21.15.5:0/2371155379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 msgr2=0x7f78ec103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 --2- 172.21.15.5:0/2371155379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec103d90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009900 tx=0x7f78e002a410 comp rx=0 tx=0).stop 2024-05-14T15:36:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 -- 172.21.15.5:0/2371155379 shutdown_connections 2024-05-14T15:36:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 --2- 172.21.15.5:0/2371155379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec104400 0x7f78ec1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 --2- 172.21.15.5:0/2371155379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec103d90 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 --2- 172.21.15.5:0/2371155379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 -- 172.21.15.5:0/2371155379 >> 172.21.15.5:0/2371155379 conn(0x7f78ec0793a0 msgr2=0x7f78ec0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 -- 172.21.15.5:0/2371155379 shutdown_connections 2024-05-14T15:36:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 -- 172.21.15.5:0/2371155379 wait complete. 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 Processor -- start 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.971+0000 7f78f4489640 1 -- start start 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f4489640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec104400 0x7f78ec074fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f4489640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f4489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f4489640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec113ea0 con 0x7f78ec10a150 2024-05-14T15:36:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f4489640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec113d20 con 0x7f78ec109cb0 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f4489640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec114020 con 0x7f78ec104400 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2225640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec104400 0x7f78ec074fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec075510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35484/0 (socket says 172.21.15.5:35484) 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec075a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60278/0 (socket says 172.21.15.5:60278) 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f1a24640 1 -- 172.21.15.5:0/3462968738 learned_addr learned my addr 172.21.15.5:0/3462968738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 -- 172.21.15.5:0/3462968738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec104400 msgr2=0x7f78ec074fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec104400 0x7f78ec074fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 -- 172.21.15.5:0/3462968738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 msgr2=0x7f78ec075510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec075510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 -- 172.21.15.5:0/3462968738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ec1a8c40 con 0x7f78ec10a150 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec075510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.972+0000 7f78f2a26640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec075a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009fd0 tx=0x7f78e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.973+0000 7f78df7fe640 1 -- 172.21.15.5:0/3462968738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400ce20 con 0x7f78ec10a150 2024-05-14T15:36:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.973+0000 7f78df7fe640 1 -- 172.21.15.5:0/3462968738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e4005020 con 0x7f78ec10a150 2024-05-14T15:36:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.973+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ec1a8f30 con 0x7f78ec10a150 2024-05-14T15:36:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.973+0000 7f78df7fe640 1 -- 172.21.15.5:0/3462968738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4010690 con 0x7f78ec10a150 2024-05-14T15:36:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.973+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ec1a9410 con 0x7f78ec10a150 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.975+0000 7f78fb41a740 1 Processor -- start 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78df7fe640 1 -- 172.21.15.5:0/3462968738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f78e40054d0 con 0x7f78ec10a150 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 -- start start 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bfa00075d0 con 0x55bf9fce04d0 2024-05-14T15:36:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf9ffc0890 con 0x55bf9fd011b0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.976+0000 7f78fb41a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bfa0003590 con 0x55bf9fcd82f0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35490/0 (socket says 172.21.15.5:35490) 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 -- 172.21.15.5:0/3790487200 learned_addr learned my addr 172.21.15.5:0/3790487200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f2225640 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78df7fe640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b29d0 0x7f78d80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f2a26640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b29d0 0x7f78d80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 -- 172.21.15.5:0/3790487200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 msgr2=0x55bfa00a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 -- 172.21.15.5:0/3790487200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 msgr2=0x55bfa00a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 -- 172.21.15.5:0/3790487200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9fd19210 con 0x55bf9fd011b0 2024-05-14T15:36:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78df7fe640 1 -- 172.21.15.5:0/3462968738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f78e40e30d0 con 0x7f78ec10a150 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00a1be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005f70 tx=0x7f78e0032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8991a14149c7fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.977+0000 7f78f2a26640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b29d0 0x7f78d80b4ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f78e40e0410 tx=0x7f78e401c000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.978+0000 7f78dcbfe640 1 -- 172.21.15.5:0/3790487200 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0035070 con 0x55bf9fd011b0 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.978+0000 7f78dcbfe640 1 -- 172.21.15.5:0/3790487200 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e00054d0 con 0x55bf9fd011b0 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.978+0000 7f78dcbfe640 1 -- 172.21.15.5:0/3790487200 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0004ac0 con 0x55bf9fd011b0 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.978+0000 7f78fb41a740 1 -- 172.21.15.5:0/3790487200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 msgr2=0x55bfa00a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.978+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00a1be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005f70 tx=0x7f78e0032f40 comp rx=0 tx=0).stop 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- 172.21.15.5:0/3790487200 shutdown_connections 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3790487200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00a1be0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- 172.21.15.5:0/3790487200 >> 172.21.15.5:0/3790487200 conn(0x55bf9fcf5300 msgr2=0x55bfa009e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- 172.21.15.5:0/3790487200 shutdown_connections 2024-05-14T15:36:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- 172.21.15.5:0/3790487200 wait complete. 2024-05-14T15:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- start start 2024-05-14T15:36:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 Processor -- start 2024-05-14T15:36:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 0x55bfa00ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bfa00aecb0 con 0x55bf9fcd82f0 2024-05-14T15:36:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf9ffc4370 con 0x55bf9fd011b0 2024-05-14T15:36:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bfa00aee30 con 0x55bf9fce04d0 2024-05-14T15:36:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.979+0000 7f78fb41a740 10 client.0 _ll_register_callbacks cb 0x55bf9fd18a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 0x55bfa00ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 0x55bfa00ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:47156/0 (socket says 172.21.15.5:47156) 2024-05-14T15:36:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 -- 172.21.15.5:0/3992428954 learned_addr learned my addr 172.21.15.5:0/3992428954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60296/0 (socket says 172.21.15.5:60296) 2024-05-14T15:36:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 -- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 msgr2=0x55bfa00b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 -- 172.21.15.5:0/3992428954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 msgr2=0x55bfa00ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 -- 172.21.15.5:0/3992428954 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9fde5620 con 0x55bf9fce04d0 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78f2225640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 0x55bfa00ac750 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78e8012700 tx=0x7f78e8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a042b03b7377d3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3992428954 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8004030 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e00054d0 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3992428954 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f78e80041d0 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3992428954 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f78e80043d0 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78fb41a740 10 client.27685 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.980+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bfa00bb090 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3992428954 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e00054d0 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3992428954 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f78e80039e0 con 0x55bf9fce04d0 2024-05-14T15:36:17.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78fb41a740 10 client.27685 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78fb41a740 10 client.27685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78fb41a740 20 client.27685 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fcbfd80 0x55bfa00be560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78fb41a740 4 client.27685 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa00151e0 con 0x55bf9fcbfd80 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.981+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fcbfd80 0x55bfa00be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78d5ffb640 10 client.27685 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fcbfd80 0x55bfa00be560 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394f57cd2ade47fd server_cookie=eb6486107badfc8 in_seq=0 out_seq=0 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3992428954 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfa00151e0 con 0x55bf9fcbfd80 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78d5ffb640 10 client.27685 handle_command_reply: tid=0 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78fb41a740 1 client.27685 shutdown 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 msgr2=0x55bfa00ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.982+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 0x55bfa00ac750 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78e8012700 tx=0x7f78e8012bc0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 shutdown_connections 2024-05-14T15:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fcbfd80 0x55bfa00be560 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fce04d0 0x55bfa00ac750 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78d5ffb640 0 client.27685 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3992428954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 >> 172.21.15.5:0/3992428954 conn(0x55bf9fcf5300 msgr2=0x55bfa00a19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 shutdown_connections 2024-05-14T15:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 1 -- 172.21.15.5:0/3992428954 wait complete. 2024-05-14T15:36:17.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.983+0000 7f78fb41a740 20 client.27685 trim_cache size 0 max 16384 2024-05-14T15:36:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.999+0000 7f78fb41a740 1 Processor -- start 2024-05-14T15:36:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:17.999+0000 7f78fb41a740 1 -- start start 2024-05-14T15:36:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60308/0 (socket says 172.21.15.5:60308) 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:47172/0 (socket says 172.21.15.5:47172) 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1b60c0 con 0x55bf9fce04d0 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 -- 172.21.15.5:0/2696954887 learned_addr learned my addr 172.21.15.5:0/2696954887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1b3990 con 0x55bf9fd011b0 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1b1800 con 0x55bf9fcd82f0 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f1a24640 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 -- 172.21.15.5:0/2696954887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 msgr2=0x55bfa00a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 -- 172.21.15.5:0/2696954887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 msgr2=0x55bfa00b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.000+0000 7f78f2225640 1 -- 172.21.15.5:0/2696954887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa00a7ba0 con 0x55bf9fce04d0 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.001+0000 7f78f1a24640 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.001+0000 7f78f2225640 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00aafe0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009e80 tx=0x7f78e801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c082cd36976e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.001+0000 7f78d7fff640 1 -- 172.21.15.5:0/2696954887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8004000 con 0x55bf9fce04d0 2024-05-14T15:36:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.001+0000 7f78d7fff640 1 -- 172.21.15.5:0/2696954887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e80041a0 con 0x55bf9fce04d0 2024-05-14T15:36:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.001+0000 7f78d7fff640 1 -- 172.21.15.5:0/2696954887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8003850 con 0x55bf9fce04d0 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 msgr2=0x55bfa00aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00aafe0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009e80 tx=0x7f78e801c4e0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 shutdown_connections 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fd011b0 0x55bfa00b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fce04d0 0x55bfa00aafe0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 --2- 172.21.15.5:0/2696954887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fcd82f0 0x55bfa00a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.002+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 >> 172.21.15.5:0/2696954887 conn(0x55bf9fcf5300 msgr2=0x55bfa00aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.003+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 shutdown_connections 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.003+0000 7f78fb41a740 1 -- 172.21.15.5:0/2696954887 wait complete. 2024-05-14T15:36:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.003+0000 7f78fb41a740 1 -- start start 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.003+0000 7f78fb41a740 1 Processor -- start 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.003+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 0x55bfa00b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78fb41a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fd011b0 0x55bfa00b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78fb41a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1b1e50 con 0x55bf9fcd82f0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78fb41a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1b32c0 con 0x55bf9fce04d0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78fb41a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78ec1b2520 con 0x55bf9fd011b0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78fb41a740 10 client.0 _ll_register_callbacks cb 0x7f78e8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 0x55bfa00b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f1a24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fd011b0 0x55bfa00b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 0x55bfa00b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35520/0 (socket says 172.21.15.5:35520) 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 -- 172.21.15.5:0/3776629003 learned_addr learned my addr 172.21.15.5:0/3776629003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 -- 172.21.15.5:0/3776629003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fd011b0 msgr2=0x55bfa00b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fd011b0 0x55bfa00b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 -- 172.21.15.5:0/3776629003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 msgr2=0x55bfa00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.004+0000 7f78f2225640 1 -- 172.21.15.5:0/3776629003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa0124980 con 0x55bf9fce04d0 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78f2225640 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 0x55bfa00b36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f78e8003f20 tx=0x7f78e801db20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73537ea51024477 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3776629003 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8004000 con 0x55bf9fce04d0 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e80041a0 con 0x55bf9fce04d0 2024-05-14T15:36:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3776629003 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7f78e80035f0 con 0x55bf9fce04d0 2024-05-14T15:36:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3776629003 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f78e800fe90 con 0x55bf9fce04d0 2024-05-14T15:36:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78fb41a740 10 client.27719 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bfa0125c10 con 0x55bf9fce04d0 2024-05-14T15:36:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3776629003 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e80041a0 con 0x55bf9fce04d0 2024-05-14T15:36:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.005+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3776629003 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f78e80055b0 con 0x55bf9fce04d0 2024-05-14T15:36:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78fb41a740 10 client.27719 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78fb41a740 10 client.27719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78fb41a740 20 client.27719 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fc951c0 0x55bfa01282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78fb41a740 4 client.27719 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:36:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfa000f050 con 0x55bf9fc951c0 2024-05-14T15:36:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fc951c0 0x55bfa01282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78d5ffb640 10 client.27719 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:18.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.006+0000 7f78f1a24640 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fc951c0 0x55bfa01282c0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac887b4db6c4a3a server_cookie=9f52f67d821f648 in_seq=0 out_seq=0 2024-05-14T15:36:18.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:17 smithi005 ceph-mon[31559]: pgmap v361: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 67 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-14T15:36:18.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1053488 9.21 MB/sec execute 459 sec latency 68.600 ms 2024-05-14T15:36:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.253+0000 7f78d5ffb640 1 -- 172.21.15.5:0/3776629003 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfa000f050 con 0x55bf9fc951c0 2024-05-14T15:36:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.253+0000 7f78d5ffb640 10 client.27719 handle_command_reply: tid=0 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.253+0000 7f78fb41a740 1 client.27719 shutdown 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.253+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 msgr2=0x55bfa00b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 0x55bfa00b36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f78e8003f20 tx=0x7f78e801db20 comp rx=0 tx=0).stop 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 shutdown_connections 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bf9fc951c0 0x55bfa01282c0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78d5ffb640 0 client.27719 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf9fd011b0 0x55bfa00b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf9fce04d0 0x55bfa00b36d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 --2- 172.21.15.5:0/3776629003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9fcd82f0 0x55bfa00b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 >> 172.21.15.5:0/3776629003 conn(0x55bf9fcf5300 msgr2=0x55bfa00a1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 shutdown_connections 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 1 -- 172.21.15.5:0/3776629003 wait complete. 2024-05-14T15:36:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.254+0000 7f78fb41a740 20 client.27719 trim_cache size 0 max 16384 2024-05-14T15:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b29d0 msgr2=0x7f78d80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b29d0 0x7f78d80b4ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f78e40e0410 tx=0x7f78e401c000 comp rx=0 tx=0).stop 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 msgr2=0x7f78ec075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec075a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009fd0 tx=0x7f78e400ef10 comp rx=0 tx=0).stop 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 shutdown_connections 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f78d80b29d0 0x7f78d80b4ee0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ec10a150 0x7f78ec075a50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec109cb0 0x7f78ec075510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 --2- 172.21.15.5:0/3462968738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78ec104400 0x7f78ec074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 >> 172.21.15.5:0/3462968738 conn(0x7f78ec0793a0 msgr2=0x7f78ec1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 shutdown_connections 2024-05-14T15:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:18.255+0000 7f78f4489640 1 -- 172.21.15.5:0/3462968738 wait complete. 2024-05-14T15:36:18.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:17 smithi144 ceph-mon[35650]: pgmap v361: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 67 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-14T15:36:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:17 smithi193 ceph-mon[36001]: pgmap v361: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 67 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-05-14T15:36:19.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1055212 9.22 MB/sec execute 460 sec latency 134.180 ms 2024-05-14T15:36:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 -- 172.21.15.5:0/1644380221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 msgr2=0x7fe95c105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1644380221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c105910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe950009990 tx=0x7fe950009ca0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 -- 172.21.15.5:0/1644380221 shutdown_connections 2024-05-14T15:36:19.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1644380221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c10e930 0x7fe95c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1644380221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c105910 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1644380221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe95c105e50 0x7fe95c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.560+0000 7fe9646d5700 1 -- 172.21.15.5:0/1644380221 >> 172.21.15.5:0/1644380221 conn(0x7fe95c0fedf0 msgr2=0x7fe95c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.561+0000 7fe9646d5700 1 -- 172.21.15.5:0/1644380221 shutdown_connections 2024-05-14T15:36:19.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.561+0000 7fe9646d5700 1 -- 172.21.15.5:0/1644380221 wait complete. 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.561+0000 7fe9646d5700 1 Processor -- start 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.561+0000 7fe9646d5700 1 -- start start 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe9646d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe9646d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c105e50 0x7fe95c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe9646d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe95c10e930 0x7fe95c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe9646d5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c114440 con 0x7fe95c105500 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe9646d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1142c0 con 0x7fe95c10e930 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe9646d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1145c0 con 0x7fe95c105e50 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60350/0 (socket says 172.21.15.5:60350) 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 -- 172.21.15.5:0/1883291099 learned_addr learned my addr 172.21.15.5:0/1883291099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962c72700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe95c10e930 0x7fe95c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe961c70700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c105e50 0x7fe95c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 -- 172.21.15.5:0/1883291099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c105e50 msgr2=0x7fe95c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c105e50 0x7fe95c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 -- 172.21.15.5:0/1883291099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe95c10e930 msgr2=0x7fe95c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe95c10e930 0x7fe95c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 -- 172.21.15.5:0/1883291099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c19fca0 con 0x7fe95c105500 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.562+0000 7fe962471700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c19add0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe9500047e0 tx=0x7fe950005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.563+0000 7fe94f7fe700 1 -- 172.21.15.5:0/1883291099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95000fb40 con 0x7fe95c105500 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.563+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c19ff30 con 0x7fe95c105500 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.563+0000 7fe94f7fe700 1 -- 172.21.15.5:0/1883291099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95000fce0 con 0x7fe95c105500 2024-05-14T15:36:19.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.563+0000 7fe94f7fe700 1 -- 172.21.15.5:0/1883291099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500054c0 con 0x7fe95c105500 2024-05-14T15:36:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.563+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c1a0400 con 0x7fe95c105500 2024-05-14T15:36:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.566+0000 7fe94f7fe700 1 -- 172.21.15.5:0/1883291099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe950009d10 con 0x7fe95c105500 2024-05-14T15:36:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.566+0000 7fe94f7fe700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe9480b29f0 0x7fe9480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 Processor -- start 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe961c70700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe9480b29f0 0x7fe9480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe94f7fe700 1 -- 172.21.15.5:0/1883291099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7fe9500e3570 con 0x7fe95c105500 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 -- start start 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc624e40 0x5653cc7a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe961c70700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe9480b29f0 0x7fe9480b4f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe95c1022b0 tx=0x7fe958009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7a7070 0x5653cc7ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653cc6c4860 con 0x5653cc7ab890 2024-05-14T15:36:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653cc6637c0 con 0x5653cc624e40 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5653cc6c0820 con 0x5653cc7a7070 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe962c72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe962c72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60366/0 (socket says 172.21.15.5:60366) 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe962c72700 1 -- 172.21.15.5:0/3572314500 learned_addr learned my addr 172.21.15.5:0/3572314500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe962471700 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc624e40 0x5653cc7a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.567+0000 7fe961c70700 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7a7070 0x5653cc7ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe962c72700 1 -- 172.21.15.5:0/3572314500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7a7070 msgr2=0x5653cc7ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe962c72700 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7a7070 0x5653cc7ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe962c72700 1 -- 172.21.15.5:0/3572314500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc624e40 msgr2=0x5653cc7a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe962c72700 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc624e40 0x5653cc7a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe962c72700 1 -- 172.21.15.5:0/3572314500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cc6118d0 con 0x5653cc7ab890 2024-05-14T15:36:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe962c72700 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7abca0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe954003fa0 tx=0x7fe954009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d9050e4c52e929 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe94cff9700 1 -- 172.21.15.5:0/3572314500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954004bb0 con 0x5653cc7ab890 2024-05-14T15:36:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe94cff9700 1 -- 172.21.15.5:0/3572314500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe954004d50 con 0x5653cc7ab890 2024-05-14T15:36:19.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.568+0000 7fe94cff9700 1 -- 172.21.15.5:0/3572314500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954007da0 con 0x5653cc7ab890 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3572314500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 msgr2=0x5653cc7abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7abca0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe954003fa0 tx=0x7fe954009ba0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3572314500 shutdown_connections 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7abca0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc624e40 0x5653cc7a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3572314500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7a7070 0x5653cc7ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3572314500 >> 172.21.15.5:0/3572314500 conn(0x5653cc66c600 msgr2=0x5653cc79d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:19.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3572314500 shutdown_connections 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.569+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3572314500 wait complete. 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 -- start start 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 Processor -- start 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653cc7b3840 con 0x5653cc7ab890 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653cc62ac30 con 0x5653cc7a7070 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5653cc7b39c0 con 0x5653cc624e40 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe9730e9b80 10 client.0 _ll_register_callbacks cb 0x5653cc65c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe962c72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe962c72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60372/0 (socket says 172.21.15.5:60372) 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe962c72700 1 -- 172.21.15.5:0/3194753577 learned_addr learned my addr 172.21.15.5:0/3194753577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:19.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe961c70700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.570+0000 7fe962471700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe962c72700 1 -- 172.21.15.5:0/3194753577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 msgr2=0x5653cc7b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe962c72700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe962c72700 1 -- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 msgr2=0x5653cc7b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe962c72700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe962c72700 1 -- 172.21.15.5:0/3194753577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cc660690 con 0x5653cc7ab890 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe962c72700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7b0fd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe954000f80 tx=0x7fe954007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd66e97a694d2ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe945ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe945ffb700 1 -- 172.21.15.5:0/3194753577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954022070 con 0x5653cc7ab890 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe945ffb700 1 -- 172.21.15.5:0/3194753577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7fe954004660 con 0x5653cc7ab890 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe954004d50 con 0x5653cc7ab890 2024-05-14T15:36:19.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe945ffb700 1 -- 172.21.15.5:0/3194753577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe95402e050 con 0x5653cc7ab890 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe9730e9b80 10 client.18144 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5653cc7be3f0 con 0x5653cc7ab890 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.571+0000 7fe945ffb700 1 -- 172.21.15.5:0/3194753577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe954004d50 con 0x5653cc7ab890 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe945ffb700 1 -- 172.21.15.5:0/3194753577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe954015ca0 con 0x5653cc7ab890 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe9730e9b80 10 client.18144 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe9730e9b80 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe9730e9b80 20 client.18144 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc7c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe9730e9b80 4 client.18144 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653cc6d2240 con 0x5653cc7c0d10 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe962471700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc7c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe945ffb700 10 client.18144 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.572+0000 7fe962471700 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc7c31c0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b27a26221a132d server_cookie=578d3caf94e2b9a2 in_seq=0 out_seq=0 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe945ffb700 1 -- 172.21.15.5:0/3194753577 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653cc6d2240 con 0x5653cc7c0d10 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe945ffb700 10 client.18144 handle_command_reply: tid=0 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 client.18144 shutdown 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 msgr2=0x5653cc7b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7b0fd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe954000f80 tx=0x7fe954007810 comp rx=0 tx=0).stop 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 shutdown_connections 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc7c31c0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7ab890 0x5653cc7b0fd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe945ffb700 0 client.18144 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.573+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.574+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/3194753577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.574+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 >> 172.21.15.5:0/3194753577 conn(0x5653cc66c600 msgr2=0x5653cc7a1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.574+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 shutdown_connections 2024-05-14T15:36:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.574+0000 7fe9730e9b80 1 -- 172.21.15.5:0/3194753577 wait complete. 2024-05-14T15:36:19.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.574+0000 7fe9730e9b80 20 client.18144 trim_cache size 0 max 16384 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 Processor -- start 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 -- start start 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 0x5653cc7abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7ab890 0x5653cc7ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1b0f10 con 0x5653cc624e40 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1ae7a0 con 0x5653cc7a7070 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1ac610 con 0x5653cc7ab890 2024-05-14T15:36:19.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe962471700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 0x5653cc7abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe962471700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 0x5653cc7abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60378/0 (socket says 172.21.15.5:60378) 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe962471700 1 -- 172.21.15.5:0/1226559672 learned_addr learned my addr 172.21.15.5:0/1226559672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.589+0000 7fe961c70700 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962c72700 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7ab890 0x5653cc7ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962471700 1 -- 172.21.15.5:0/1226559672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7ab890 msgr2=0x5653cc7ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962471700 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7ab890 0x5653cc7ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962471700 1 -- 172.21.15.5:0/1226559672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 msgr2=0x5653cc7bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962471700 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962471700 1 -- 172.21.15.5:0/1226559672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cc7af890 con 0x5653cc624e40 2024-05-14T15:36:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe962471700 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 0x5653cc7abca0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe9500a6590 tx=0x7fe95001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3064d21394e9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe947fff700 1 -- 172.21.15.5:0/1226559672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500920c0 con 0x5653cc624e40 2024-05-14T15:36:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe947fff700 1 -- 172.21.15.5:0/1226559672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9500ae8c0 con 0x5653cc624e40 2024-05-14T15:36:19.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.590+0000 7fe947fff700 1 -- 172.21.15.5:0/1226559672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9500b30b0 con 0x5653cc624e40 2024-05-14T15:36:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 -- 172.21.15.5:0/1226559672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 msgr2=0x5653cc7abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 0x5653cc7abca0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe9500a6590 tx=0x7fe95001a040 comp rx=0 tx=0).stop 2024-05-14T15:36:19.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 -- 172.21.15.5:0/1226559672 shutdown_connections 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc7ab890 0x5653cc7ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc624e40 0x5653cc7abca0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/1226559672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7a7070 0x5653cc7bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 -- 172.21.15.5:0/1226559672 >> 172.21.15.5:0/1226559672 conn(0x5653cc66c600 msgr2=0x5653cc7aab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 -- 172.21.15.5:0/1226559672 shutdown_connections 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 -- 172.21.15.5:0/1226559672 wait complete. 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.591+0000 7fe9730e9b80 1 -- start start 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 Processor -- start 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 0x5653cc7c3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7ab890 0x5653cc7c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1acc60 con 0x5653cc7a7070 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1ae0d0 con 0x5653cc7ab890 2024-05-14T15:36:19.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c1ad330 con 0x5653cc624e40 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe9730e9b80 10 client.0 _ll_register_callbacks cb 0x5653cc65c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 0x5653cc7c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 0x5653cc7c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60380/0 (socket says 172.21.15.5:60380) 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 -- 172.21.15.5:0/2856421875 learned_addr learned my addr 172.21.15.5:0/2856421875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe962471700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe962c72700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7ab890 0x5653cc7c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 -- 172.21.15.5:0/2856421875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 msgr2=0x5653cc7c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 -- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7ab890 msgr2=0x5653cc7c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7ab890 0x5653cc7c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.592+0000 7fe961c70700 1 -- 172.21.15.5:0/2856421875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cc7c3dd0 con 0x5653cc7a7070 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe962471700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe961c70700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 0x5653cc7c3160 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe9580117d0 tx=0x7fe9580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c710b1a795bf6d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:19.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe945ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe945ffb700 1 -- 172.21.15.5:0/2856421875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95802d060 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe945ffb700 1 -- 172.21.15.5:0/2856421875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9650+0+0 (secure 0 0 0) 0x7fe958028630 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9500ae8c0 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe945ffb700 1 -- 172.21.15.5:0/2856421875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe958028830 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe9730e9b80 10 client.18150 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5653cc82a370 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.593+0000 7fe945ffb700 1 -- 172.21.15.5:0/2856421875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9500ae8c0 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe945ffb700 1 -- 172.21.15.5:0/2856421875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe95801dac0 con 0x5653cc7a7070 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe9730e9b80 10 client.18150 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe9730e9b80 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe9730e9b80 20 client.18150 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc82cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe9730e9b80 4 client.18150 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:36:19.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653cc6cc2e0 con 0x5653cc7c0d10 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe962471700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc82cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe945ffb700 10 client.18150 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.594+0000 7fe962471700 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc82cb20 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297c50509b09169f server_cookie=1089c421473028cd in_seq=0 out_seq=0 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.595+0000 7fe945ffb700 1 -- 172.21.15.5:0/2856421875 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653cc6cc2e0 con 0x5653cc7c0d10 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.595+0000 7fe945ffb700 10 client.18150 handle_command_reply: tid=0 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 client.18150 shutdown 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 msgr2=0x5653cc7c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 0x5653cc7c3160 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe9580117d0 tx=0x7fe9580047c0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 shutdown_connections 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5653cc7c0d10 0x5653cc82cb20 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe945ffb700 0 client.18150 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653cc7ab890 0x5653cc7c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653cc624e40 0x5653cc7c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 --2- 172.21.15.5:0/2856421875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653cc7a7070 0x5653cc7c3160 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 >> 172.21.15.5:0/2856421875 conn(0x5653cc66c600 msgr2=0x5653cc79cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:19.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 shutdown_connections 2024-05-14T15:36:19.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 1 -- 172.21.15.5:0/2856421875 wait complete. 2024-05-14T15:36:19.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9730e9b80 20 client.18150 trim_cache size 0 max 16384 2024-05-14T15:36:19.603 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe9480b29f0 msgr2=0x7fe9480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe9480b29f0 0x7fe9480b4f00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe95c1022b0 tx=0x7fe958009260 comp rx=0 tx=0).stop 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 msgr2=0x7fe95c19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.596+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c19add0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe9500047e0 tx=0x7fe950005f10 comp rx=0 tx=0).stop 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 shutdown_connections 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe9480b29f0 0x7fe9480b4f00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe95c10e930 0x7fe95c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95c105500 0x7fe95c19add0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 --2- 172.21.15.5:0/1883291099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95c105e50 0x7fe95c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 >> 172.21.15.5:0/1883291099 conn(0x7fe95c0fedf0 msgr2=0x7fe95c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 shutdown_connections 2024-05-14T15:36:19.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:19.597+0000 7fe9646d5700 1 -- 172.21.15.5:0/1883291099 wait complete. 2024-05-14T15:36:20.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1056574 9.21 MB/sec execute 461 sec latency 34.063 ms 2024-05-14T15:36:20.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:19 smithi144 ceph-mon[35650]: pgmap v362: 129 pgs: 129 active+clean; 69 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-05-14T15:36:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:19 smithi193 ceph-mon[36001]: pgmap v362: 129 pgs: 129 active+clean; 69 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-05-14T15:36:20.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:19 smithi005 ceph-mon[31559]: pgmap v362: 129 pgs: 129 active+clean; 69 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-05-14T15:36:21.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1058404 9.21 MB/sec execute 462 sec latency 109.925 ms 2024-05-14T15:36:21.449 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:36:21.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:36:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:22.077 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:21 smithi005 ceph-mon[31559]: pgmap v363: 129 pgs: 129 active+clean; 69 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-05-14T15:36:22.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1059388 9.20 MB/sec execute 463 sec latency 25.515 ms 2024-05-14T15:36:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:21 smithi144 ceph-mon[35650]: pgmap v363: 129 pgs: 129 active+clean; 69 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-05-14T15:36:22.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:21 smithi193 ceph-mon[36001]: pgmap v363: 129 pgs: 129 active+clean; 69 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-05-14T15:36:23.134 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:22 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:36:23.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1061520 9.21 MB/sec execute 464 sec latency 29.688 ms 2024-05-14T15:36:23.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:22 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:36:23.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:22 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:36:24.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1062704 9.21 MB/sec execute 465 sec latency 57.212 ms 2024-05-14T15:36:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 -- 172.21.15.5:0/1752076893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f810e8f0 msgr2=0x7f17f8110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 --2- 172.21.15.5:0/1752076893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f810e8f0 0x7f17f8110dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f17f000b0a0 tx=0x7f17f000b3b0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 -- 172.21.15.5:0/1752076893 shutdown_connections 2024-05-14T15:36:24.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 --2- 172.21.15.5:0/1752076893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f810e8f0 0x7f17f8110dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 --2- 172.21.15.5:0/1752076893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 --2- 172.21.15.5:0/1752076893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8105dd0 0x7f17f810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 -- 172.21.15.5:0/1752076893 >> 172.21.15.5:0/1752076893 conn(0x7f17f80fef30 msgr2=0x7f17f8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 -- 172.21.15.5:0/1752076893 shutdown_connections 2024-05-14T15:36:24.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:23 smithi005 ceph-mon[31559]: pgmap v364: 129 pgs: 129 active+clean; 66 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-05-14T15:36:24.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:23 smithi005 ceph-mon[31559]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T15:36:24.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:23 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:36:24.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:23 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:36:24.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:23 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:36:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.158+0000 7f17fdef9700 1 -- 172.21.15.5:0/1752076893 wait complete. 2024-05-14T15:36:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 Processor -- start 2024-05-14T15:36:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 -- start start 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8105dd0 0x7f17f819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f810e8f0 0x7f17f819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8114450 con 0x7f17f810e8f0 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.159+0000 7f17fdef9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81142d0 con 0x7f17f8105dd0 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17fdef9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81145d0 con 0x7f17f8105480 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8105dd0 0x7f17f819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:47284/0 (socket says 172.21.15.5:47284) 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f77fe700 1 -- 172.21.15.5:0/55401328 learned_addr learned my addr 172.21.15.5:0/55401328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f7fff700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f810e8f0 0x7f17f819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 -- 172.21.15.5:0/55401328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 msgr2=0x7f17f819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 -- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f810e8f0 msgr2=0x7f17f819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f810e8f0 0x7f17f819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 -- 172.21.15.5:0/55401328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f819fc70 con 0x7f17f8105dd0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f77fe700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8105dd0 0x7f17f819b340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e9f0 tx=0x7f17e000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17f4ff9700 1 -- 172.21.15.5:0/55401328 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e000c9c0 con 0x7f17f8105dd0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.160+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f819ff60 con 0x7f17f8105dd0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.161+0000 7f17f4ff9700 1 -- 172.21.15.5:0/55401328 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e000cb60 con 0x7f17f8105dd0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.161+0000 7f17f4ff9700 1 -- 172.21.15.5:0/55401328 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0018680 con 0x7f17f8105dd0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.161+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f81a7880 con 0x7f17f8105dd0 2024-05-14T15:36:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.164+0000 7f17f4ff9700 1 -- 172.21.15.5:0/55401328 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f17e0010430 con 0x7f17f8105dd0 2024-05-14T15:36:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.164+0000 7f17f4ff9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17e40b29f0 0x7f17e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.165+0000 7f17f77fe700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17e40b29f0 0x7f17e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.165+0000 7f17f4ff9700 1 -- 172.21.15.5:0/55401328 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f17e00e2b10 con 0x7f17f8105dd0 2024-05-14T15:36:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.165+0000 7f17f77fe700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17e40b29f0 0x7f17e40b4f00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17e80104e0 tx=0x7f17e8007480 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.165+0000 7f180c90db80 1 Processor -- start 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.165+0000 7f180c90db80 1 -- start start 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f180c90db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d057ec0 0x55e92d1db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f180c90db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1d5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f180c90db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d1d5e90 0x55e92d1d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f180c90db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e92d0f79c0 con 0x55e92d057ec0 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f180c90db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e92d05ce20 con 0x55e92d1dbe00 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f180c90db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e92d0f3980 con 0x55e92d1d5e90 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d057ec0 0x55e92d1db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f7fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d1d5e90 0x55e92d1d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1d5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d057ec0 0x55e92d1db8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60416/0 (socket says 172.21.15.5:60416) 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f77fe700 1 -- 172.21.15.5:0/1295624798 learned_addr learned my addr 172.21.15.5:0/1295624798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f7fff700 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d1d5e90 0x55e92d1d62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:47296/0 (socket says 172.21.15.5:47296) 2024-05-14T15:36:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 -- 172.21.15.5:0/1295624798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d1d5e90 msgr2=0x55e92d1d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d1d5e90 0x55e92d1d62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 -- 172.21.15.5:0/1295624798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d057ec0 msgr2=0x55e92d1db8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d057ec0 0x55e92d1db8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 -- 172.21.15.5:0/1295624798 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92d044950 con 0x55e92d1dbe00 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1d5950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17e00dfe50 tx=0x7f17e00a51f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d4b900002a29ac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.166+0000 7f17edffb700 1 -- 172.21.15.5:0/1295624798 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e00b2190 con 0x55e92d1dbe00 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f17edffb700 1 -- 172.21.15.5:0/1295624798 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e00a99e0 con 0x55e92d1dbe00 2024-05-14T15:36:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f17edffb700 1 -- 172.21.15.5:0/1295624798 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e00cc340 con 0x55e92d1dbe00 2024-05-14T15:36:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 -- 172.21.15.5:0/1295624798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 msgr2=0x55e92d1d5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1d5950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17e00dfe50 tx=0x7f17e00a51f0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 -- 172.21.15.5:0/1295624798 shutdown_connections 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d1d5e90 0x55e92d1d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d057ec0 0x55e92d1db8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 --2- 172.21.15.5:0/1295624798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1d5950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 -- 172.21.15.5:0/1295624798 >> 172.21.15.5:0/1295624798 conn(0x55e92d09f6b0 msgr2=0x55e92d1d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.167+0000 7f180c90db80 1 -- 172.21.15.5:0/1295624798 shutdown_connections 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 -- 172.21.15.5:0/1295624798 wait complete. 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 -- start start 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 Processor -- start 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e92d1e28a0 con 0x55e92d1d5e90 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e92d0b0f50 con 0x55e92d1dbe00 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.168+0000 7f180c90db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e92d1e2a20 con 0x55e92d057ec0 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f180c90db80 10 client.0 _ll_register_callbacks cb 0x55e92d08f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1dfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60432/0 (socket says 172.21.15.5:60432) 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f6ffd700 1 -- 172.21.15.5:0/1390340156 learned_addr learned my addr 172.21.15.5:0/1390340156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f77fe700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 -- 172.21.15.5:0/1390340156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 msgr2=0x55e92d1df580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1df580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 -- 172.21.15.5:0/1390340156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 msgr2=0x55e92d1dfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1dfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 -- 172.21.15.5:0/1390340156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92d0ae7e0 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17f7fff700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e0000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17f000d010 tx=0x7f17f000be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfae4124d3d2e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17d6ffd700 1 -- 172.21.15.5:0/1390340156 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00077c0 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f17d6ffd700 1 -- 172.21.15.5:0/1390340156 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f17f0007960 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.169+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e00a99e0 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f17d6ffd700 1 -- 172.21.15.5:0/1390340156 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17f002f050 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 10 client.27749 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e92d1ed4f0 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f17d6ffd700 1 -- 172.21.15.5:0/1390340156 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e00a99e0 con 0x55e92d1dbe00 2024-05-14T15:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f17d6ffd700 1 -- 172.21.15.5:0/1390340156 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f17f0011070 con 0x55e92d1dbe00 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 10 client.27749 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 10 client.27749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 20 client.27749 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d1efa80 0x55e92d1f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 4 client.27749 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e92d1053a0 con 0x55e92d1efa80 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.170+0000 7f17f77fe700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d1efa80 0x55e92d1f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.171+0000 7f17d6ffd700 10 client.27749 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.171+0000 7f17f77fe700 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d1efa80 0x55e92d1f1ea0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d9402954a429a5 server_cookie=955b2732516a90d8 in_seq=0 out_seq=0 2024-05-14T15:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f17d6ffd700 1 -- 172.21.15.5:0/1390340156 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e92d1053a0 con 0x55e92d1efa80 2024-05-14T15:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f17d6ffd700 10 client.27749 handle_command_reply: tid=0 2024-05-14T15:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 client.27749 shutdown 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 msgr2=0x55e92d1e0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e0000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f17f000d010 tx=0x7f17f000be20 comp rx=0 tx=0).stop 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 shutdown_connections 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d1efa80 0x55e92d1f1ea0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e0000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f17d6ffd700 0 client.27749 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 --2- 172.21.15.5:0/1390340156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.172+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 >> 172.21.15.5:0/1390340156 conn(0x55e92d09f6b0 msgr2=0x55e92d1d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.173+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 shutdown_connections 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.173+0000 7f180c90db80 1 -- 172.21.15.5:0/1390340156 wait complete. 2024-05-14T15:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.173+0000 7f180c90db80 20 client.27749 trim_cache size 0 max 16384 2024-05-14T15:36:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 Processor -- start 2024-05-14T15:36:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 -- start start 2024-05-14T15:36:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1eb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81b3020 con 0x55e92d1d5e90 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81b08b0 con 0x55e92d1dbe00 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.194+0000 7f180c90db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81ae720 con 0x55e92d057ec0 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60438/0 (socket says 172.21.15.5:60438) 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f6ffd700 1 -- 172.21.15.5:0/2198727160 learned_addr learned my addr 172.21.15.5:0/2198727160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f7fff700 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 -- 172.21.15.5:0/2198727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 msgr2=0x55e92d1dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1dfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 -- 172.21.15.5:0/2198727160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 msgr2=0x55e92d1eb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1eb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 -- 172.21.15.5:0/2198727160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92d1d2060 con 0x55e92d057ec0 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1eb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17f77fe700 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e1660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55e92d1f2580 tx=0x7f17e8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5dcaa34fd215d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17ed7fa700 1 -- 172.21.15.5:0/2198727160 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e802a670 con 0x55e92d057ec0 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17ed7fa700 1 -- 172.21.15.5:0/2198727160 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e8030070 con 0x55e92d057ec0 2024-05-14T15:36:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.195+0000 7f17ed7fa700 1 -- 172.21.15.5:0/2198727160 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e801fde0 con 0x55e92d057ec0 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 -- 172.21.15.5:0/2198727160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 msgr2=0x55e92d1e1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e1660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55e92d1f2580 tx=0x7f17e8013000 comp rx=0 tx=0).stop 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 -- 172.21.15.5:0/2198727160 shutdown_connections 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e1660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 --2- 172.21.15.5:0/2198727160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1eb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 -- 172.21.15.5:0/2198727160 >> 172.21.15.5:0/2198727160 conn(0x55e92d09f6b0 msgr2=0x55e92d1d91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 -- 172.21.15.5:0/2198727160 shutdown_connections 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.196+0000 7f180c90db80 1 -- 172.21.15.5:0/2198727160 wait complete. 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 -- start start 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 Processor -- start 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81aed70 con 0x55e92d1d5e90 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81b01e0 con 0x55e92d1dbe00 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81af440 con 0x55e92d057ec0 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f180c90db80 10 client.0 _ll_register_callbacks cb 0x55e92d08f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.197+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60446/0 (socket says 172.21.15.5:60446) 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 -- 172.21.15.5:0/3949462586 learned_addr learned my addr 172.21.15.5:0/3949462586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f7fff700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 -- 172.21.15.5:0/3949462586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 msgr2=0x55e92d1e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 -- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 msgr2=0x55e92d1e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 -- 172.21.15.5:0/3949462586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92d1d0fc0 con 0x55e92d1d5e90 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f77fe700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17f6ffd700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1e81f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f17e00a96e0 tx=0x7f17e00b25e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea85a3fa2db5533 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17d6ffd700 1 -- 172.21.15.5:0/3949462586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e00c2070 con 0x55e92d1d5e90 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17d6ffd700 1 -- 172.21.15.5:0/3949462586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f17e00a8e50 con 0x55e92d1d5e90 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e8030070 con 0x55e92d1d5e90 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f17d6ffd700 1 -- 172.21.15.5:0/3949462586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17e00a99e0 con 0x55e92d1d5e90 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f180c90db80 10 client.18168 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.198+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e92d25a6e0 con 0x55e92d1d5e90 2024-05-14T15:36:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f17d6ffd700 1 -- 172.21.15.5:0/3949462586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e8030070 con 0x55e92d1d5e90 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f17d6ffd700 1 -- 172.21.15.5:0/3949462586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f17e006f140 con 0x55e92d1d5e90 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f180c90db80 10 client.18168 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f180c90db80 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f180c90db80 20 client.18168 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f180c90db80 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d25aa20 0x55e92d25ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f180c90db80 4 client.18168 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e92d0ff440 con 0x55e92d25aa20 2024-05-14T15:36:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.199+0000 7f17f77fe700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d25aa20 0x55e92d25ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.200+0000 7f17f77fe700 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d25aa20 0x55e92d25ce40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ba6f371ad0b491 server_cookie=2a39f344dc3d4f8f in_seq=0 out_seq=0 2024-05-14T15:36:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.200+0000 7f17d6ffd700 10 client.18168 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.202+0000 7f17d6ffd700 1 -- 172.21.15.5:0/3949462586 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e92d0ff440 con 0x55e92d25aa20 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.202+0000 7f17d6ffd700 10 client.18168 handle_command_reply: tid=0 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.202+0000 7f180c90db80 1 client.18168 shutdown 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 msgr2=0x55e92d1e81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1e81f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f17e00a96e0 tx=0x7f17e00b25e0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 shutdown_connections 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55e92d25aa20 0x55e92d25ce40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e92d1dbe00 0x55e92d1e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e92d057ec0 0x55e92d1e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 --2- 172.21.15.5:0/3949462586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e92d1d5e90 0x55e92d1e81f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 >> 172.21.15.5:0/3949462586 conn(0x55e92d09f6b0 msgr2=0x55e92d1d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f17d6ffd700 0 client.18168 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 shutdown_connections 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 1 -- 172.21.15.5:0/3949462586 wait complete. 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f180c90db80 20 client.18168 trim_cache size 0 max 16384 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "38f4e364-d572-4e5b-8030-4149bc86f791", 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:36:24.208 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17e40b29f0 msgr2=0x7f17e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f17fdef9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17e40b29f0 0x7f17e40b4f00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17e80104e0 tx=0x7f17e8007480 comp rx=0 tx=0).stop 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8105dd0 msgr2=0x7f17f819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.203+0000 7f17fdef9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8105dd0 0x7f17f819b340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e9f0 tx=0x7f17e000ed00 comp rx=0 tx=0).stop 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 shutdown_connections 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f17e40b29f0 0x7f17e40b4f00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f810e8f0 0x7f17f819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17f8105480 0x7f17f819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 --2- 172.21.15.5:0/55401328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8105dd0 0x7f17f819b340 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 >> 172.21.15.5:0/55401328 conn(0x7f17f80fef30 msgr2=0x7f17f80ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 shutdown_connections 2024-05-14T15:36:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:24.204+0000 7f17fdef9700 1 -- 172.21.15.5:0/55401328 wait complete. 2024-05-14T15:36:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:23 smithi144 ceph-mon[35650]: pgmap v364: 129 pgs: 129 active+clean; 66 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-05-14T15:36:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:23 smithi144 ceph-mon[35650]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T15:36:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:23 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:36:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:23 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:36:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:24 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:36:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:23 smithi193 ceph-mon[36001]: pgmap v364: 129 pgs: 129 active+clean; 66 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-05-14T15:36:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:23 smithi193 ceph-mon[36001]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T15:36:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:23 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:36:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:23 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:36:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:23 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:36:24.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:36:25.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1064590 9.20 MB/sec execute 466 sec latency 23.873 ms 2024-05-14T15:36:25.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:25 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:36:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:25 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:25 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:36:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:25 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:25.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:25 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:36:25.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:25 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:25 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:36:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:25 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:25.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:25 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:36:25.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:25 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:25.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:25 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:36:25.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:25 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:26.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1065829 9.20 MB/sec execute 467 sec latency 134.833 ms 2024-05-14T15:36:26.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:26 smithi144 ceph-mon[35650]: pgmap v366: 129 pgs: 129 active+clean; 66 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 272 op/s 2024-05-14T15:36:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:26 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:36:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:26 smithi193 ceph-mon[36001]: pgmap v366: 129 pgs: 129 active+clean; 66 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 272 op/s 2024-05-14T15:36:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:26 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:36:26.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:26 smithi005 ceph-mon[31559]: pgmap v366: 129 pgs: 129 active+clean; 66 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 272 op/s 2024-05-14T15:36:26.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:26 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:36:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.089+0000 7f74e4ab0700 1 -- 172.21.15.5:0/3303848288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e01034c0 msgr2=0x7f74e01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/3303848288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e01034c0 0x7f74e01038d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f74d00099c0 tx=0x7f74d0009cd0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 -- 172.21.15.5:0/3303848288 shutdown_connections 2024-05-14T15:36:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/3303848288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0108740 0x7f74e01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/3303848288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e01034c0 0x7f74e01038d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/3303848288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e0107df0 0x7f74e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 -- 172.21.15.5:0/3303848288 >> 172.21.15.5:0/3303848288 conn(0x7f74e00fedc0 msgr2=0x7f74e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 -- 172.21.15.5:0/3303848288 shutdown_connections 2024-05-14T15:36:27.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.090+0000 7f74e4ab0700 1 -- 172.21.15.5:0/3303848288 wait complete. 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 Processor -- start 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 -- start start 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e01034c0 0x7f74e019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0107df0 0x7f74e019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 0x7f74e01a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0113370 con 0x7f74e0108740 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01131f0 con 0x7f74e01034c0 2024-05-14T15:36:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74e4ab0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01134f0 con 0x7f74e0107df0 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 0x7f74e01a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 0x7f74e01a6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40036/0 (socket says 172.21.15.5:40036) 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74ded9d700 1 -- 172.21.15.5:0/247664237 learned_addr learned my addr 172.21.15.5:0/247664237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74ddd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0107df0 0x7f74e019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.091+0000 7f74de59c700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e01034c0 0x7f74e019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ded9d700 1 -- 172.21.15.5:0/247664237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0107df0 msgr2=0x7f74e019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ded9d700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0107df0 0x7f74e019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ded9d700 1 -- 172.21.15.5:0/247664237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e01034c0 msgr2=0x7f74e019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ded9d700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e01034c0 0x7f74e019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ded9d700 1 -- 172.21.15.5:0/247664237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e019ca00 con 0x7f74e0108740 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74de59c700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e01034c0 0x7f74e019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0107df0 0x7f74e019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74ded9d700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 0x7f74e01a6c90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f74d800ef30 tx=0x7f74d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74cf7fe700 1 -- 172.21.15.5:0/247664237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d80092e0 con 0x7f74e0108740 2024-05-14T15:36:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e01a7290 con 0x7f74e0108740 2024-05-14T15:36:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74cf7fe700 1 -- 172.21.15.5:0/247664237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d8004bb0 con 0x7f74e0108740 2024-05-14T15:36:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e01a77a0 con 0x7f74e0108740 2024-05-14T15:36:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.092+0000 7f74cf7fe700 1 -- 172.21.15.5:0/247664237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8004020 con 0x7f74e0108740 2024-05-14T15:36:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.095+0000 7f74cf7fe700 1 -- 172.21.15.5:0/247664237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f74d801b030 con 0x7f74e0108740 2024-05-14T15:36:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74cf7fe700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f74c80b29f0 0x7f74c80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 Processor -- start 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74de59c700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f74c80b29f0 0x7f74c80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74cf7fe700 1 -- 172.21.15.5:0/247664237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f74d80e24d0 con 0x7f74e0108740 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 -- start start 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74de59c700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f74c80b29f0 0x7f74c80b4f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005b40 tx=0x7f74d001a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca921cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca9221350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca9221890 0x55bca9221ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bca913a860 con 0x55bca921d070 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bca90d97c0 con 0x55bca909ae40 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.096+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bca9136820 con 0x55bca9221890 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca921cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca9221350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca921cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35478/0 (socket says 172.21.15.5:35478) 2024-05-14T15:36:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 -- 172.21.15.5:0/1267707806 learned_addr learned my addr 172.21.15.5:0/1267707806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74ded9d700 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca9221890 0x55bca9221ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 -- 172.21.15.5:0/1267707806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca9221890 msgr2=0x55bca9221ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca9221890 0x55bca9221ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 -- 172.21.15.5:0/1267707806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 msgr2=0x55bca9221350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca9221350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 -- 172.21.15.5:0/1267707806 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca90878d0 con 0x55bca909ae40 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74ded9d700 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca9221890 0x55bca9221ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74de59c700 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca921cb30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74d0015000 tx=0x7f74d0022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e80f958b0e1b434 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.097+0000 7f74ccff9700 1 -- 172.21.15.5:0/1267707806 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00095b0 con 0x55bca909ae40 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74ccff9700 1 -- 172.21.15.5:0/1267707806 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d0003c70 con 0x55bca909ae40 2024-05-14T15:36:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74ccff9700 1 -- 172.21.15.5:0/1267707806 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0030ca0 con 0x55bca909ae40 2024-05-14T15:36:27.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1267707806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 msgr2=0x55bca921cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca921cb30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74d0015000 tx=0x7f74d0022000 comp rx=0 tx=0).stop 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1267707806 shutdown_connections 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca9221890 0x55bca9221ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca921cb30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1267707806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca9221350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.098+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1267707806 >> 172.21.15.5:0/1267707806 conn(0x55bca90e2600 msgr2=0x55bca92134b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.099+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1267707806 shutdown_connections 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.099+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1267707806 wait complete. 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.099+0000 7f74f34c4b80 1 -- start start 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.099+0000 7f74f34c4b80 1 Processor -- start 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca9226500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca9226a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca9226f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bca92297e0 con 0x55bca9221890 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bca90a0c30 con 0x55bca909ae40 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bca9229960 con 0x55bca921d070 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74f34c4b80 10 client.0 _ll_register_callbacks cb 0x55bca90d2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca9226f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74de59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca9226500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca9226a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74de59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca9226500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35482/0 (socket says 172.21.15.5:35482) 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74de59c700 1 -- 172.21.15.5:0/688890833 learned_addr learned my addr 172.21.15.5:0/688890833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 -- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 msgr2=0x55bca9226500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca9226500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 -- 172.21.15.5:0/688890833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 msgr2=0x55bca9226f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca9226f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 -- 172.21.15.5:0/688890833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca90d6690 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca9226a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74e01024b0 tx=0x7f74d4009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ca29accbc6068f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.100+0000 7f74c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74c5ffb700 1 -- 172.21.15.5:0/688890833 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004050 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74c5ffb700 1 -- 172.21.15.5:0/688890833 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f74d40041f0 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d0003c70 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74c5ffb700 1 -- 172.21.15.5:0/688890833 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f74d4025050 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 10 client.27739 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bca9234430 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74c5ffb700 1 -- 172.21.15.5:0/688890833 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d0003c70 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74c5ffb700 1 -- 172.21.15.5:0/688890833 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f74d40038e0 con 0x55bca921d070 2024-05-14T15:36:27.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 10 client.27739 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 10 client.27739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 20 client.27739 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca9236a70 0x55bca9238e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 4 client.27739 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.101+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bca9148240 con 0x55bca9236a70 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.102+0000 7f74de59c700 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca9236a70 0x55bca9238e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.102+0000 7f74c5ffb700 10 client.27739 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.102+0000 7f74de59c700 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca9236a70 0x55bca9238e90 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16fb36b2319b8053 server_cookie=81844ca5a37928bc in_seq=0 out_seq=0 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.103+0000 7f74c5ffb700 1 -- 172.21.15.5:0/688890833 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bca9148240 con 0x55bca9236a70 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.103+0000 7f74c5ffb700 10 client.27739 handle_command_reply: tid=0 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.103+0000 7f74f34c4b80 1 client.27739 shutdown 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.103+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 msgr2=0x55bca9226a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.103+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca9226a40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74e01024b0 tx=0x7f74d4009e20 comp rx=0 tx=0).stop 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.103+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 shutdown_connections 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca9236a70 0x55bca9238e90 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca9226f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca9226500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74c5ffb700 0 client.27739 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/688890833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca9226a40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 >> 172.21.15.5:0/688890833 conn(0x55bca90e2600 msgr2=0x55bca9217450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 shutdown_connections 2024-05-14T15:36:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 1 -- 172.21.15.5:0/688890833 wait complete. 2024-05-14T15:36:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.104+0000 7f74f34c4b80 20 client.27739 trim_cache size 0 max 16384 2024-05-14T15:36:27.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.124+0000 7f74f34c4b80 1 Processor -- start 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.124+0000 7f74f34c4b80 1 -- start start 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca909ae40 0x55bca9226dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca92132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca9221890 0x55bca92328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01b0c50 con 0x55bca921d070 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01ae4e0 con 0x55bca9221890 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01ac350 con 0x55bca909ae40 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca92132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca92132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40058/0 (socket says 172.21.15.5:40058) 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 -- 172.21.15.5:0/2154571731 learned_addr learned my addr 172.21.15.5:0/2154571731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:27.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ded9d700 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca9221890 0x55bca92328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74de59c700 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca909ae40 0x55bca9226dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 -- 172.21.15.5:0/2154571731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca909ae40 msgr2=0x55bca9226dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca909ae40 0x55bca9226dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 -- 172.21.15.5:0/2154571731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca9221890 msgr2=0x55bca92328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca9221890 0x55bca92328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 -- 172.21.15.5:0/2154571731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bca9220520 con 0x55bca921d070 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.125+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca92132f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f74d4003e50 tx=0x7f74d4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631ffa170439bd92 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74c7fff700 1 -- 172.21.15.5:0/2154571731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d40104f0 con 0x55bca921d070 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74c7fff700 1 -- 172.21.15.5:0/2154571731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d401ac10 con 0x55bca921d070 2024-05-14T15:36:27.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74c7fff700 1 -- 172.21.15.5:0/2154571731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4005570 con 0x55bca921d070 2024-05-14T15:36:27.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74f34c4b80 1 -- 172.21.15.5:0/2154571731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 msgr2=0x55bca92132f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca92132f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f74d4003e50 tx=0x7f74d4005900 comp rx=0 tx=0).stop 2024-05-14T15:36:27.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74f34c4b80 1 -- 172.21.15.5:0/2154571731 shutdown_connections 2024-05-14T15:36:27.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca9221890 0x55bca92328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.126+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca909ae40 0x55bca9226dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/2154571731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca921d070 0x55bca92132f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- 172.21.15.5:0/2154571731 >> 172.21.15.5:0/2154571731 conn(0x55bca90e2600 msgr2=0x55bca9212be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- 172.21.15.5:0/2154571731 shutdown_connections 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- 172.21.15.5:0/2154571731 wait complete. 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- start start 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 Processor -- start 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca922ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca923d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca923dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01ac9a0 con 0x55bca9221890 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01ade10 con 0x55bca909ae40 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01ad070 con 0x55bca921d070 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.127+0000 7f74f34c4b80 10 client.0 _ll_register_callbacks cb 0x55bca90d2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca923dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74de59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca922ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca923dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40072/0 (socket says 172.21.15.5:40072) 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 -- 172.21.15.5:0/1144289704 learned_addr learned my addr 172.21.15.5:0/1144289704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca923d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 -- 172.21.15.5:0/1144289704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 msgr2=0x55bca923d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca923d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 -- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 msgr2=0x55bca922ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca922ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 -- 172.21.15.5:0/1144289704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bca923e230 con 0x55bca9221890 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ddd9b700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca923d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74de59c700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca922ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74ded9d700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca923dcf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f74d800ef00 tx=0x7f74d80e2330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f8d0a16eaf58b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74c5ffb700 1 -- 172.21.15.5:0/1144289704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d80c2070 con 0x55bca9221890 2024-05-14T15:36:27.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74c5ffb700 1 -- 172.21.15.5:0/1144289704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f74d80b1f20 con 0x55bca9221890 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.128+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d401ac10 con 0x55bca9221890 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74c5ffb700 1 -- 172.21.15.5:0/1144289704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f74d80a93d0 con 0x55bca9221890 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 10 client.18186 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bca92a0da0 con 0x55bca9221890 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74c5ffb700 1 -- 172.21.15.5:0/1144289704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d80ad780 con 0x55bca9221890 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74c5ffb700 1 -- 172.21.15.5:0/1144289704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f74d80a4d40 con 0x55bca9221890 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 10 client.18186 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 20 client.18186 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca92a14a0 0x55bca92a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 4 client.18186 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:36:27.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f74d401ac10 con 0x55bca92a14a0 2024-05-14T15:36:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.129+0000 7f74de59c700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca92a14a0 0x55bca92a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.130+0000 7f74de59c700 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca92a14a0 0x55bca92a38c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd4fbc53269f88a server_cookie=bb529ac5eb16dc92 in_seq=0 out_seq=0 2024-05-14T15:36:27.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.130+0000 7f74c5ffb700 10 client.18186 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74c5ffb700 1 -- 172.21.15.5:0/1144289704 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f74d401ac10 con 0x55bca92a14a0 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74c5ffb700 10 client.18186 handle_command_reply: tid=0 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 client.18186 shutdown 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 msgr2=0x55bca923dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca923dcf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f74d800ef00 tx=0x7f74d80e2330 comp rx=0 tx=0).stop 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 shutdown_connections 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55bca92a14a0 0x55bca92a38c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bca9221890 0x55bca923dcf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bca909ae40 0x55bca922ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 --2- 172.21.15.5:0/1144289704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bca921d070 0x55bca923d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74c5ffb700 0 client.18186 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 >> 172.21.15.5:0/1144289704 conn(0x55bca90e2600 msgr2=0x55bca9227b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 shutdown_connections 2024-05-14T15:36:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 1 -- 172.21.15.5:0/1144289704 wait complete. 2024-05-14T15:36:27.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.132+0000 7f74f34c4b80 20 client.18186 trim_cache size 0 max 16384 2024-05-14T15:36:27.138 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:36:27.139 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:36:27.139 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:36:27.139 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:36:27.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f74c80b29f0 msgr2=0x7f74c80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f74c80b29f0 0x7f74c80b4f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005b40 tx=0x7f74d001a040 comp rx=0 tx=0).stop 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 msgr2=0x7f74e01a6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 0x7f74e01a6c90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f74d800ef30 tx=0x7f74d800c3a0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 shutdown_connections 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f74c80b29f0 0x7f74c80b4f00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e0108740 0x7f74e01a6c90 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e01034c0 0x7f74e019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 --2- 172.21.15.5:0/247664237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74e0107df0 0x7f74e019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 >> 172.21.15.5:0/247664237 conn(0x7f74e00fedc0 msgr2=0x7f74e01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 shutdown_connections 2024-05-14T15:36:27.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:27.133+0000 7f74e4ab0700 1 -- 172.21.15.5:0/247664237 wait complete. 2024-05-14T15:36:27.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1067184 9.20 MB/sec execute 468 sec latency 25.058 ms 2024-05-14T15:36:27.924 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:36:27.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:36:28.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1069324 9.20 MB/sec execute 469 sec latency 19.989 ms 2024-05-14T15:36:28.182 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:28.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:28 smithi144 ceph-mon[35650]: pgmap v367: 129 pgs: 129 active+clean; 68 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-05-14T15:36:28.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:28 smithi193 ceph-mon[36001]: pgmap v367: 129 pgs: 129 active+clean; 68 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-05-14T15:36:28.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:28 smithi005 ceph-mon[31559]: pgmap v367: 129 pgs: 129 active+clean; 68 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-05-14T15:36:29.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1070623 9.20 MB/sec execute 470 sec latency 18.319 ms 2024-05-14T15:36:30.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:30 smithi005 ceph-mon[31559]: pgmap v368: 129 pgs: 129 active+clean; 72 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-05-14T15:36:30.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1072755 9.21 MB/sec execute 471 sec latency 45.429 ms 2024-05-14T15:36:30.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:30 smithi144 ceph-mon[35650]: pgmap v368: 129 pgs: 129 active+clean; 72 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-05-14T15:36:30.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:30 smithi193 ceph-mon[36001]: pgmap v368: 129 pgs: 129 active+clean; 72 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-05-14T15:36:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 -- 172.21.15.5:0/2304450626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c107df0 msgr2=0x7f476c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 --2- 172.21.15.5:0/2304450626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c107df0 0x7f476c108200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f47580099c0 tx=0x7f4758009cd0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 -- 172.21.15.5:0/2304450626 shutdown_connections 2024-05-14T15:36:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 --2- 172.21.15.5:0/2304450626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f476c108740 0x7f476c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 --2- 172.21.15.5:0/2304450626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f476c1034c0 0x7f476c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 --2- 172.21.15.5:0/2304450626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c107df0 0x7f476c108200 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 -- 172.21.15.5:0/2304450626 >> 172.21.15.5:0/2304450626 conn(0x7f476c0fedc0 msgr2=0x7f476c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 -- 172.21.15.5:0/2304450626 shutdown_connections 2024-05-14T15:36:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.389+0000 7f4772fb9700 1 -- 172.21.15.5:0/2304450626 wait complete. 2024-05-14T15:36:30.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 Processor -- start 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 -- start start 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 0x7f476c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f476c107df0 0x7f476c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f476c108740 0x7f476c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f476c114480 con 0x7f476c1034c0 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f476c114300 con 0x7f476c107df0 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4772fb9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f476c114600 con 0x7f476c108740 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4770d55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 0x7f476c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4770d55700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 0x7f476c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40094/0 (socket says 172.21.15.5:40094) 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4770d55700 1 -- 172.21.15.5:0/3812331989 learned_addr learned my addr 172.21.15.5:0/3812331989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f476bfff700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f476c107df0 0x7f476c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.390+0000 7f4771556700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f476c108740 0x7f476c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4770d55700 1 -- 172.21.15.5:0/3812331989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f476c108740 msgr2=0x7f476c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4770d55700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f476c108740 0x7f476c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4770d55700 1 -- 172.21.15.5:0/3812331989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f476c107df0 msgr2=0x7f476c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4770d55700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f476c107df0 0x7f476c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4770d55700 1 -- 172.21.15.5:0/3812331989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c19fca0 con 0x7f476c1034c0 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4770d55700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 0x7f476c19add0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f476000e9a0 tx=0x7f476000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476000ca10 con 0x7f476c1034c0 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476000cbb0 con 0x7f476c1034c0 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760018680 con 0x7f476c1034c0 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c19ff90 con 0x7f476c1034c0 2024-05-14T15:36:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.391+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1109b0 con 0x7f476c1034c0 2024-05-14T15:36:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.392+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c104970 con 0x7f476c1034c0 2024-05-14T15:36:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.394+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4760010430 con 0x7f476c1034c0 2024-05-14T15:36:30.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.395+0000 7f4769ffb700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f475c0b2a70 0x7f475c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:30.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.395+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f47600e2e60 con 0x7f476c1034c0 2024-05-14T15:36:30.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.395+0000 7f476bfff700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f475c0b2a70 0x7f475c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:30.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.396+0000 7f476bfff700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f475c0b2a70 0x7f475c0b4f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f475801a040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:30.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.400+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47600aa430 con 0x7f476c1034c0 2024-05-14T15:36:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.726+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f476c02e5f0 con 0x7f476c1034c0 2024-05-14T15:36:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:36:30.736 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:35:24.885756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27547":{"gid":27547,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3068356254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3068356254},{"type":"v1","addr":"172.21.15.144:6835","nonce":3068356254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:36:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.734+0000 7f4769ffb700 1 -- 172.21.15.5:0/3812331989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11864 (secure 0 0 0) 0x7f47600a9d30 con 0x7f476c1034c0 2024-05-14T15:36:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.741+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f475c0b2a70 msgr2=0x7f475c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.741+0000 7f4772fb9700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f475c0b2a70 0x7f475c0b4f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f475801a040 comp rx=0 tx=0).stop 2024-05-14T15:36:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.742+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 msgr2=0x7f476c19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.742+0000 7f4772fb9700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 0x7f476c19add0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f476000e9a0 tx=0x7f476000ecb0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 shutdown_connections 2024-05-14T15:36:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f475c0b2a70 0x7f475c0b4f80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f476c108740 0x7f476c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f476c1034c0 0x7f476c19add0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 --2- 172.21.15.5:0/3812331989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f476c107df0 0x7f476c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 >> 172.21.15.5:0/3812331989 conn(0x7f476c0fedc0 msgr2=0x7f476c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.744+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 shutdown_connections 2024-05-14T15:36:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:30.745+0000 7f4772fb9700 1 -- 172.21.15.5:0/3812331989 wait complete. 2024-05-14T15:36:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-14T15:36:31.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1073317 9.19 MB/sec execute 472 sec latency 106.462 ms 2024-05-14T15:36:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:31 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3812331989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:36:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:31 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3812331989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:36:31.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:31 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3812331989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:36:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.374+0000 7fb1037a9640 1 -- 172.21.15.5:0/595459695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc10e6d0 msgr2=0x7fb0fc110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.374+0000 7fb1037a9640 1 --2- 172.21.15.5:0/595459695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc10e6d0 0x7fb0fc110af0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009ca0 tx=0x7fb0f402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 -- 172.21.15.5:0/595459695 shutdown_connections 2024-05-14T15:36:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 --2- 172.21.15.5:0/595459695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc10e6d0 0x7fb0fc110af0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 --2- 172.21.15.5:0/595459695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 --2- 172.21.15.5:0/595459695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0fc109d20 0x7fb0fc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 -- 172.21.15.5:0/595459695 >> 172.21.15.5:0/595459695 conn(0x7fb0fc0779e0 msgr2=0x7fb0fc077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 -- 172.21.15.5:0/595459695 shutdown_connections 2024-05-14T15:36:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 -- 172.21.15.5:0/595459695 wait complete. 2024-05-14T15:36:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 Processor -- start 2024-05-14T15:36:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.375+0000 7fb1037a9640 1 -- start start 2024-05-14T15:36:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb1037a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb1037a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc109d20 0x7fb0fc1033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb1037a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0fc10e6d0 0x7fb0fc1038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb1037a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc113ee0 con 0x7fb0fc10e6d0 2024-05-14T15:36:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb1037a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc113d60 con 0x7fb0fc109d20 2024-05-14T15:36:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb1037a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc114060 con 0x7fb0fc1021d0 2024-05-14T15:36:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb100d44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc109d20 0x7fb0fc1033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:46752/0 (socket says 172.21.15.5:46752) 2024-05-14T15:36:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101d46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0fc10e6d0 0x7fb0fc1038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 -- 172.21.15.5:0/4164026401 learned_addr learned my addr 172.21.15.5:0/4164026401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 -- 172.21.15.5:0/4164026401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc109d20 msgr2=0x7fb0fc1033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc109d20 0x7fb0fc1033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 -- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0fc10e6d0 msgr2=0x7fb0fc1038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0fc10e6d0 0x7fb0fc1038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb101545640 1 -- 172.21.15.5:0/4164026401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc1a8c00 con 0x7fb0fc1021d0 2024-05-14T15:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.376+0000 7fb100d44640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc109d20 0x7fb0fc1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.377+0000 7fb101545640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc102e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e9f0 tx=0x7fb0f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.377+0000 7fb0f27fc640 1 -- 172.21.15.5:0/4164026401 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800ccc0 con 0x7fb0fc1021d0 2024-05-14T15:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.377+0000 7fb0f27fc640 1 -- 172.21.15.5:0/4164026401 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f80035f0 con 0x7fb0fc1021d0 2024-05-14T15:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.378+0000 7fb0f27fc640 1 -- 172.21.15.5:0/4164026401 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800ce60 con 0x7fb0fc1021d0 2024-05-14T15:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.378+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc1a8ef0 con 0x7fb0fc1021d0 2024-05-14T15:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.378+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc1a9360 con 0x7fb0fc1021d0 2024-05-14T15:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.381+0000 7fb0f27fc640 1 -- 172.21.15.5:0/4164026401 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb0f8003aa0 con 0x7fb0fc1021d0 2024-05-14T15:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.381+0000 7fb10a65e740 1 Processor -- start 2024-05-14T15:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.381+0000 7fb10a65e740 1 -- start start 2024-05-14T15:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c9b1b0 0x5654e303bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e303c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c7a4d0 0x5654e303c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb10a65e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654e2fa15d0 con 0x5654e2c722f0 2024-05-14T15:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb10a65e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5654e2f5a890 con 0x5654e2c9b1b0 2024-05-14T15:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb10a65e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5654e2f9d590 con 0x5654e2c7a4d0 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb101d46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c9b1b0 0x5654e303bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e303c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e303c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40140/0 (socket says 172.21.15.5:40140) 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb101545640 1 -- 172.21.15.5:0/2516320360 learned_addr learned my addr 172.21.15.5:0/2516320360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb100d44640 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c7a4d0 0x5654e303c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.382+0000 7fb0f27fc640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb0e80b29d0 0x7fb0e80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 -- 172.21.15.5:0/2516320360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c7a4d0 msgr2=0x5654e303c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c7a4d0 0x5654e303c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 -- 172.21.15.5:0/2516320360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 msgr2=0x5654e303c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e303c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 -- 172.21.15.5:0/2516320360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e2cb3210 con 0x5654e2c9b1b0 2024-05-14T15:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb0f27fc640 1 -- 172.21.15.5:0/4164026401 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fb0f80e2c30 con 0x7fb0fc1021d0 2024-05-14T15:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb0e80b29d0 0x7fb0e80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb101d46640 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c9b1b0 0x5654e303bad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0f402ade0 tx=0x7fb0f4009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72975b5093a5dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb0e77fe640 1 -- 172.21.15.5:0/2516320360 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4042040 con 0x5654e2c9b1b0 2024-05-14T15:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb0e77fe640 1 -- 172.21.15.5:0/2516320360 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f4036070 con 0x5654e2c9b1b0 2024-05-14T15:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.383+0000 7fb0e77fe640 1 -- 172.21.15.5:0/2516320360 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40046f0 con 0x5654e2c9b1b0 2024-05-14T15:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 -- 172.21.15.5:0/2516320360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c9b1b0 msgr2=0x5654e303bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb101d46640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb0e80b29d0 0x7fb0e80b4ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005e30 tx=0x7fb0f404e000 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c9b1b0 0x5654e303bad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0f402ade0 tx=0x7fb0f4009a00 comp rx=0 tx=0).stop 2024-05-14T15:36:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 -- 172.21.15.5:0/2516320360 shutdown_connections 2024-05-14T15:36:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c7a4d0 0x5654e303c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e303c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2516320360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c9b1b0 0x5654e303bad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 -- 172.21.15.5:0/2516320360 >> 172.21.15.5:0/2516320360 conn(0x5654e2c8f300 msgr2=0x5654e3038340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 -- 172.21.15.5:0/2516320360 shutdown_connections 2024-05-14T15:36:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.384+0000 7fb10a65e740 1 -- 172.21.15.5:0/2516320360 wait complete. 2024-05-14T15:36:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.385+0000 7fb10a65e740 1 -- start start 2024-05-14T15:36:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.385+0000 7fb10a65e740 1 Processor -- start 2024-05-14T15:36:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e304e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e304ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c9b1b0 0x5654e305a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654e3051520 con 0x5654e2c722f0 2024-05-14T15:36:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5654e2f5e370 con 0x5654e2c7a4d0 2024-05-14T15:36:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5654e30516a0 con 0x5654e2c9b1b0 2024-05-14T15:36:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb10a65e740 10 client.0 _ll_register_callbacks cb 0x5654e2cb2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e304e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101d46640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c9b1b0 0x5654e305a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e304e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40150/0 (socket says 172.21.15.5:40150) 2024-05-14T15:36:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101545640 1 -- 172.21.15.5:0/2832272009 learned_addr learned my addr 172.21.15.5:0/2832272009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb100d44640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e304ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101d46640 1 -- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 msgr2=0x5654e304ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101d46640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e304ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101d46640 1 -- 172.21.15.5:0/2832272009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 msgr2=0x5654e304e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101d46640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e304e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101d46640 1 -- 172.21.15.5:0/2832272009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e2d7f620 con 0x5654e2c9b1b0 2024-05-14T15:36:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.386+0000 7fb101545640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e304e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb101d46640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c9b1b0 0x5654e305a9d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0f402aeb0 tx=0x7fb0f4003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99387f1c3362bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb0e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/2832272009 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4042070 con 0x5654e2c9b1b0 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/2832272009 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fb0f4004450 con 0x5654e2c9b1b0 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f4036070 con 0x5654e2c9b1b0 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/2832272009 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb0f4062050 con 0x5654e2c9b1b0 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb10a65e740 10 client.27763 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5654e3051520 con 0x5654e2c9b1b0 2024-05-14T15:36:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/2832272009 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f4036070 con 0x5654e2c9b1b0 2024-05-14T15:36:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.387+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/2832272009 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb0f40305c0 con 0x5654e2c9b1b0 2024-05-14T15:36:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb10a65e740 10 client.27763 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb10a65e740 10 client.27763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb10a65e740 20 client.27763 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c59d80 0x5654e3060d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb10a65e740 4 client.27763 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654e2faf1e0 con 0x5654e2c59d80 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.388+0000 7fb101545640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c59d80 0x5654e3060d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.389+0000 7fb0e4ff9640 10 client.27763 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.389+0000 7fb101545640 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c59d80 0x5654e3060d70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d9ce5ad18e9dbb server_cookie=fcf5ae58d73fc656 in_seq=0 out_seq=0 2024-05-14T15:36:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.390+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/2832272009 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654e2faf1e0 con 0x5654e2c59d80 2024-05-14T15:36:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.390+0000 7fb0e4ff9640 10 client.27763 handle_command_reply: tid=0 2024-05-14T15:36:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.390+0000 7fb10a65e740 1 client.27763 shutdown 2024-05-14T15:36:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c9b1b0 msgr2=0x5654e305a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c9b1b0 0x5654e305a9d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0f402aeb0 tx=0x7fb0f4003e10 comp rx=0 tx=0).stop 2024-05-14T15:36:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 shutdown_connections 2024-05-14T15:36:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c59d80 0x5654e3060d70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c9b1b0 0x5654e305a9d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e304ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb0e4ff9640 0 client.27763 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 --2- 172.21.15.5:0/2832272009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c722f0 0x5654e304e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 >> 172.21.15.5:0/2832272009 conn(0x5654e2c8f300 msgr2=0x5654e303b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 shutdown_connections 2024-05-14T15:36:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 1 -- 172.21.15.5:0/2832272009 wait complete. 2024-05-14T15:36:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.391+0000 7fb10a65e740 20 client.27763 trim_cache size 0 max 16384 2024-05-14T15:36:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 Processor -- start 2024-05-14T15:36:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 -- start start 2024-05-14T15:36:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3050600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e3037980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e3037ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1b1b50 con 0x5654e2c59d80 2024-05-14T15:36:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1af420 con 0x5654e2c7a4d0 2024-05-14T15:36:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb10a65e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1ad2b0 con 0x5654e2c722f0 2024-05-14T15:36:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3050600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3050600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40166/0 (socket says 172.21.15.5:40166) 2024-05-14T15:36:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb101d46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e3037ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb101545640 1 -- 172.21.15.5:0/1749627727 learned_addr learned my addr 172.21.15.5:0/1749627727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.408+0000 7fb100d44640 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e3037980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101d46640 1 -- 172.21.15.5:0/1749627727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 msgr2=0x5654e3037980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101d46640 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e3037980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101d46640 1 -- 172.21.15.5:0/1749627727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 msgr2=0x5654e3050600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101d46640 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3050600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101d46640 1 -- 172.21.15.5:0/1749627727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e304f690 con 0x5654e2c7a4d0 2024-05-14T15:36:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb100d44640 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e3037980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:36:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101545640 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3050600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb101d46640 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e3037ec0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004af0 tx=0x7fb0f4003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a38e1e6101b7a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb0e6ffd640 1 -- 172.21.15.5:0/1749627727 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4042070 con 0x5654e2c7a4d0 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb0e6ffd640 1 -- 172.21.15.5:0/1749627727 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f4036040 con 0x5654e2c7a4d0 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.409+0000 7fb0e6ffd640 1 -- 172.21.15.5:0/1749627727 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f402f610 con 0x5654e2c7a4d0 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 -- 172.21.15.5:0/1749627727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 msgr2=0x5654e3037ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e3037ec0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004af0 tx=0x7fb0f4003f90 comp rx=0 tx=0).stop 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 -- 172.21.15.5:0/1749627727 shutdown_connections 2024-05-14T15:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e3037980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e3037ec0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 --2- 172.21.15.5:0/1749627727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3050600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 -- 172.21.15.5:0/1749627727 >> 172.21.15.5:0/1749627727 conn(0x5654e2c8f300 msgr2=0x5654e304fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 -- 172.21.15.5:0/1749627727 shutdown_connections 2024-05-14T15:36:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.410+0000 7fb10a65e740 1 -- 172.21.15.5:0/1749627727 wait complete. 2024-05-14T15:36:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 -- start start 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 Processor -- start 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3060fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e30614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e30c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1ad900 con 0x5654e2c59d80 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1aed50 con 0x5654e2c7a4d0 2024-05-14T15:36:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1adfb0 con 0x5654e2c722f0 2024-05-14T15:36:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.411+0000 7fb10a65e740 10 client.0 _ll_register_callbacks cb 0x7fb0f80a61d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3060fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e30614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb101545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3060fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40178/0 (socket says 172.21.15.5:40178) 2024-05-14T15:36:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e30614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:46806/0 (socket says 172.21.15.5:46806) 2024-05-14T15:36:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb101545640 1 -- 172.21.15.5:0/3483249476 learned_addr learned my addr 172.21.15.5:0/3483249476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb101d46640 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e30c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 -- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 msgr2=0x5654e30c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e30c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 -- 172.21.15.5:0/3483249476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 msgr2=0x5654e3060fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3060fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 -- 172.21.15.5:0/3483249476 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e30c75d0 con 0x5654e2c722f0 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb100d44640 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e30614f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5654e303cb30 tx=0x7fb0ec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b8a881739e70ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb0e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/3483249476 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec022060 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/3483249476 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fb0ec004050 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.412+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f4036040 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/3483249476 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb0ec029050 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 10 client.27769 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5654e30c8790 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/3483249476 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f4036040 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/3483249476 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ec01c3d0 con 0x5654e2c722f0 2024-05-14T15:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 10 client.27769 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 10 client.27769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 20 client.27769 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c9b1b0 0x5654e30caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 4 client.27769 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654e2fa9050 con 0x5654e2c9b1b0 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.413+0000 7fb101545640 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c9b1b0 0x5654e30caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.414+0000 7fb0e4ff9640 10 client.27769 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.414+0000 7fb101545640 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c9b1b0 0x5654e30caf50 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d518c1cbc456d40b server_cookie=52190bb68c3d2110 in_seq=0 out_seq=0 2024-05-14T15:36:31.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb0e4ff9640 1 -- 172.21.15.5:0/3483249476 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654e2fa9050 con 0x5654e2c9b1b0 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb0e4ff9640 10 client.27769 handle_command_reply: tid=0 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 client.27769 shutdown 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 msgr2=0x5654e30614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e30614f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5654e303cb30 tx=0x7fb0ec00f790 comp rx=0 tx=0).stop 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 shutdown_connections 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x5654e2c9b1b0 0x5654e30caf50 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5654e2c722f0 0x5654e30614f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5654e2c7a4d0 0x5654e30c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 --2- 172.21.15.5:0/3483249476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654e2c59d80 0x5654e3060fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 >> 172.21.15.5:0/3483249476 conn(0x5654e2c8f300 msgr2=0x5654e303c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.787+0000 7fb0e4ff9640 0 client.27769 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.788+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 shutdown_connections 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.788+0000 7fb10a65e740 1 -- 172.21.15.5:0/3483249476 wait complete. 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.788+0000 7fb10a65e740 20 client.27769 trim_cache size 0 max 16384 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:36:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:36:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.789+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb0e80b29d0 msgr2=0x7fb0e80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.789+0000 7fb1037a9640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb0e80b29d0 0x7fb0e80b4ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005e30 tx=0x7fb0f404e000 comp rx=0 tx=0).stop 2024-05-14T15:36:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.789+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 msgr2=0x7fb0fc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.789+0000 7fb1037a9640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc102e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e9f0 tx=0x7fb0f800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.790+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 shutdown_connections 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.790+0000 7fb1037a9640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb0e80b29d0 0x7fb0e80b4ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.790+0000 7fb1037a9640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0fc10e6d0 0x7fb0fc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.790+0000 7fb1037a9640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0fc109d20 0x7fb0fc1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.790+0000 7fb1037a9640 1 --2- 172.21.15.5:0/4164026401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0fc1021d0 0x7fb0fc102e70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.790+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 >> 172.21.15.5:0/4164026401 conn(0x7fb0fc0779e0 msgr2=0x7fb0fc10a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.792+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 shutdown_connections 2024-05-14T15:36:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:31.792+0000 7fb1037a9640 1 -- 172.21.15.5:0/4164026401 wait complete. 2024-05-14T15:36:31.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:32.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1074304 9.18 MB/sec execute 473 sec latency 262.805 ms 2024-05-14T15:36:32.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:32 smithi005 ceph-mon[31559]: pgmap v369: 129 pgs: 129 active+clean; 72 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-05-14T15:36:32.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:32 smithi144 ceph-mon[35650]: pgmap v369: 129 pgs: 129 active+clean; 72 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-05-14T15:36:32.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:32 smithi193 ceph-mon[36001]: pgmap v369: 129 pgs: 129 active+clean; 72 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 286 op/s 2024-05-14T15:36:33.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1076336 9.19 MB/sec execute 474 sec latency 95.632 ms 2024-05-14T15:36:34.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:34 smithi005 ceph-mon[31559]: pgmap v370: 129 pgs: 129 active+clean; 66 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-05-14T15:36:34.159 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1077242 9.17 MB/sec execute 475 sec latency 40.397 ms 2024-05-14T15:36:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 -- 172.21.15.5:0/3198057546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04281045d0 msgr2=0x7f04281049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3198057546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04281045d0 0x7f04281049e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f04180099c0 tx=0x7f0418009cd0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 -- 172.21.15.5:0/3198057546 shutdown_connections 2024-05-14T15:36:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3198057546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0428109850 0x7f04281111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3198057546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04281045d0 0x7f04281049e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3198057546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0428108f00 0x7f0428109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 -- 172.21.15.5:0/3198057546 >> 172.21.15.5:0/3198057546 conn(0x7f04280ffea0 msgr2=0x7f04281022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 -- 172.21.15.5:0/3198057546 shutdown_connections 2024-05-14T15:36:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.190+0000 7f042d3c7700 1 -- 172.21.15.5:0/3198057546 wait complete. 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.191+0000 7f042d3c7700 1 Processor -- start 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.191+0000 7f042d3c7700 1 -- start start 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.191+0000 7f042d3c7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 0x7f042819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.191+0000 7f042d3c7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 0x7f042819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.191+0000 7f042d3c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0428109850 0x7f042819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.191+0000 7f042d3c7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04281144d0 con 0x7f0428108f00 2024-05-14T15:36:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f042d3c7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0428114350 con 0x7f04281045d0 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 0x7f042819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f042d3c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0428114650 con 0x7f0428109850 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 0x7f042819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 0x7f042819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40204/0 (socket says 172.21.15.5:40204) 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 -- 172.21.15.5:0/3236374647 learned_addr learned my addr 172.21.15.5:0/3236374647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 0x7f042819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04277fe700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0428109850 0x7f042819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 -- 172.21.15.5:0/3236374647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0428109850 msgr2=0x7f042819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0428109850 0x7f042819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 -- 172.21.15.5:0/3236374647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 msgr2=0x7f042819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 0x7f042819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 -- 172.21.15.5:0/3236374647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042819fcd0 con 0x7f0428108f00 2024-05-14T15:36:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f0426ffd700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 0x7f042819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f04267fc700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 0x7f042819b3a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f041000ea30 tx=0x7f041000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f041ffff700 1 -- 172.21.15.5:0/3236374647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000cbd0 con 0x7f0428108f00 2024-05-14T15:36:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f041ffff700 1 -- 172.21.15.5:0/3236374647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04100044d0 con 0x7f0428108f00 2024-05-14T15:36:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042819ffc0 con 0x7f0428108f00 2024-05-14T15:36:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f041ffff700 1 -- 172.21.15.5:0/3236374647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410003dc0 con 0x7f0428108f00 2024-05-14T15:36:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.192+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04281a78e0 con 0x7f0428108f00 2024-05-14T15:36:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.196+0000 7f041ffff700 1 -- 172.21.15.5:0/3236374647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0410018430 con 0x7f0428108f00 2024-05-14T15:36:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 Processor -- start 2024-05-14T15:36:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 -- start start 2024-05-14T15:36:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f041ffff700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f04140b29f0 0x7f04140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f041ffff700 1 -- 172.21.15.5:0/3236374647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f04100e2b50 con 0x7f0428108f00 2024-05-14T15:36:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f04277fe700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f04140b29f0 0x7f04140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 0x55b7ede0b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede05e10 0x55b7ede06220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7edd27940 con 0x55b7ede05e10 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7edc8cda0 con 0x55b7ede0bd80 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f043bddbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b7edd23900 con 0x55b7edc87e40 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.197+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 0x55b7ede0b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 0x55b7ede0b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:46838/0 (socket says 172.21.15.5:46838) 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f04277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede05e10 0x55b7ede06220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 -- 172.21.15.5:0/1292362551 learned_addr learned my addr 172.21.15.5:0/1292362551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f04277fe700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f04140b29f0 0x7f04140b4f00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f042000f590 tx=0x7f0420009920 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f04267fc700 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 -- 172.21.15.5:0/1292362551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 msgr2=0x55b7ede058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 -- 172.21.15.5:0/1292362551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede05e10 msgr2=0x55b7ede06220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede05e10 0x55b7ede06220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 -- 172.21.15.5:0/1292362551 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7edc748d0 con 0x55b7edc87e40 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f04267fc700 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f0426ffd700 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 0x55b7ede0b840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0418015000 tx=0x7f0418005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1662c5bd03ecaa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f041d7fa700 1 -- 172.21.15.5:0/1292362551 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418027070 con 0x55b7edc87e40 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f041d7fa700 1 -- 172.21.15.5:0/1292362551 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0418005550 con 0x55b7edc87e40 2024-05-14T15:36:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.198+0000 7f041d7fa700 1 -- 172.21.15.5:0/1292362551 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418003cb0 con 0x55b7edc87e40 2024-05-14T15:36:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 -- 172.21.15.5:0/1292362551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 msgr2=0x55b7ede0b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 0x55b7ede0b840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0418015000 tx=0x7f0418005fb0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 -- 172.21.15.5:0/1292362551 shutdown_connections 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede05e10 0x55b7ede06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7edc87e40 0x55b7ede0b840 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1292362551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 -- 172.21.15.5:0/1292362551 >> 172.21.15.5:0/1292362551 conn(0x55b7edccf630 msgr2=0x55b7ede00a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 -- 172.21.15.5:0/1292362551 shutdown_connections 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.199+0000 7f043bddbb80 1 -- 172.21.15.5:0/1292362551 wait complete. 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 -- start start 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 Processor -- start 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7edc87e40 0x55b7ede0f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede0ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7ede127c0 con 0x55b7edc87e40 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7edce0ed0 con 0x55b7ede0bd80 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b7ede12940 con 0x55b7ede05e10 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.200+0000 7f043bddbb80 10 client.0 _ll_register_callbacks cb 0x55b7edcbf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7edc87e40 0x55b7ede0f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede0ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede0ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35606/0 (socket says 172.21.15.5:35606) 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 -- 172.21.15.5:0/349343089 learned_addr learned my addr 172.21.15.5:0/349343089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04267fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 -- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 msgr2=0x55b7ede0f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 -- 172.21.15.5:0/349343089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7edc87e40 msgr2=0x55b7ede0f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7edc87e40 0x55b7ede0f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 -- 172.21.15.5:0/349343089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7edcde760 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04267fc700 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f04277fe700 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede0ff30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0420019e60 tx=0x7f0420004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e71ffacbac986089 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f040e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.201+0000 7f040e7fc700 1 -- 172.21.15.5:0/349343089 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420026030 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0418005550 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f040e7fc700 1 -- 172.21.15.5:0/349343089 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f0420004bb0 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f040e7fc700 1 -- 172.21.15.5:0/349343089 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0420031050 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 10 client.27806 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b7ede1d3e0 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f040e7fc700 1 -- 172.21.15.5:0/349343089 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0418005550 con 0x55b7ede0bd80 2024-05-14T15:36:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f040e7fc700 1 -- 172.21.15.5:0/349343089 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0420004020 con 0x55b7ede0bd80 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 10 client.27806 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 10 client.27806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 20 client.27806 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede1f970 0x55b7ede21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 4 client.27806 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.202+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7edd35320 con 0x55b7ede1f970 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.203+0000 7f0426ffd700 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede1f970 0x55b7ede21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.203+0000 7f0426ffd700 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede1f970 0x55b7ede21d90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c51b5faffbe4138c server_cookie=6434e34c079c0c0 in_seq=0 out_seq=0 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.203+0000 7f040e7fc700 10 client.27806 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f040e7fc700 1 -- 172.21.15.5:0/349343089 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7edd35320 con 0x55b7ede1f970 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f040e7fc700 10 client.27806 handle_command_reply: tid=0 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 client.27806 shutdown 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 msgr2=0x55b7ede0ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede0ff30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0420019e60 tx=0x7f0420004b40 comp rx=0 tx=0).stop 2024-05-14T15:36:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 shutdown_connections 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede1f970 0x55b7ede21d90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7ede0bd80 0x55b7ede0ff30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7edc87e40 0x55b7ede0f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 --2- 172.21.15.5:0/349343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 >> 172.21.15.5:0/349343089 conn(0x55b7edccf630 msgr2=0x55b7ede048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f040e7fc700 0 client.27806 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 shutdown_connections 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 1 -- 172.21.15.5:0/349343089 wait complete. 2024-05-14T15:36:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.204+0000 7f043bddbb80 20 client.27806 trim_cache size 0 max 16384 2024-05-14T15:36:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 Processor -- start 2024-05-14T15:36:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 -- start start 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede1b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04281b3010 con 0x55b7ede0bd80 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04281b08a0 con 0x55b7edc87e40 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f043bddbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04281ae710 con 0x55b7ede05e10 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f04277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede1b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35610/0 (socket says 172.21.15.5:35610) 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.221+0000 7f0426ffd700 1 -- 172.21.15.5:0/1657271920 learned_addr learned my addr 172.21.15.5:0/1657271920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f04267fc700 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f0426ffd700 1 -- 172.21.15.5:0/1657271920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 msgr2=0x55b7ede0fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f0426ffd700 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f0426ffd700 1 -- 172.21.15.5:0/1657271920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 msgr2=0x55b7ede1b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f0426ffd700 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede1b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f0426ffd700 1 -- 172.21.15.5:0/1657271920 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ede0b100 con 0x55b7edc87e40 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f04277fe700 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede1b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f0426ffd700 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede058d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0418015000 tx=0x7f0418000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83e693b31e6a7e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f041cff9700 1 -- 172.21.15.5:0/1657271920 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418027040 con 0x55b7edc87e40 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f041cff9700 1 -- 172.21.15.5:0/1657271920 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0418028040 con 0x55b7edc87e40 2024-05-14T15:36:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.222+0000 7f041cff9700 1 -- 172.21.15.5:0/1657271920 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418005510 con 0x55b7edc87e40 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 -- 172.21.15.5:0/1657271920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 msgr2=0x55b7ede058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede058d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0418015000 tx=0x7f0418000bc0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 -- 172.21.15.5:0/1657271920 shutdown_connections 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede1b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede058d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 --2- 172.21.15.5:0/1657271920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede0fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 -- 172.21.15.5:0/1657271920 >> 172.21.15.5:0/1657271920 conn(0x55b7edccf630 msgr2=0x55b7ede0dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 -- 172.21.15.5:0/1657271920 shutdown_connections 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.223+0000 7f043bddbb80 1 -- 172.21.15.5:0/1657271920 wait complete. 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 -- start start 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 Processor -- start 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede26500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede26a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04281aed60 con 0x55b7ede0bd80 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04281b01d0 con 0x55b7edc87e40 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04281af430 con 0x55b7ede05e10 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f043bddbb80 10 client.0 _ll_register_callbacks cb 0x55b7edcbf950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.224+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede26500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede26a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede26500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:35624/0 (socket says 172.21.15.5:35624) 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f0426ffd700 1 -- 172.21.15.5:0/3886012044 learned_addr learned my addr 172.21.15.5:0/3886012044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04277fe700 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 -- 172.21.15.5:0/3886012044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 msgr2=0x55b7ede26500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede26500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 -- 172.21.15.5:0/3886012044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 msgr2=0x55b7ede26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 -- 172.21.15.5:0/3886012044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ede27780 con 0x55b7ede05e10 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f04267fc700 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede26a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04100dfe90 tx=0x7f04100a9eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772a992bbeb38d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f040e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f040e7fc700 1 -- 172.21.15.5:0/3886012044 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410003680 con 0x55b7ede05e10 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0418028040 con 0x55b7ede05e10 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f040e7fc700 1 -- 172.21.15.5:0/3886012044 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f04100cc300 con 0x55b7ede05e10 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f040e7fc700 1 -- 172.21.15.5:0/3886012044 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f041000ce00 con 0x55b7ede05e10 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f043bddbb80 10 client.27787 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.225+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b7ede17500 con 0x55b7ede05e10 2024-05-14T15:36:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f040e7fc700 1 -- 172.21.15.5:0/3886012044 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0418028040 con 0x55b7ede05e10 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f040e7fc700 1 -- 172.21.15.5:0/3886012044 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f04100a5390 con 0x55b7ede05e10 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f043bddbb80 10 client.27787 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f043bddbb80 10 client.27787 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f043bddbb80 20 client.27787 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f043bddbb80 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede17bf0 0x55b7ede18000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f043bddbb80 4 client.27787 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7edd2f3c0 con 0x55b7ede17bf0 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f0426ffd700 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede17bf0 0x55b7ede18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f0426ffd700 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede17bf0 0x55b7ede18000 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6836d1762d09a140 server_cookie=54375e412d3ddec5 in_seq=0 out_seq=0 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.226+0000 7f040e7fc700 10 client.27787 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f040e7fc700 1 -- 172.21.15.5:0/3886012044 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7edd2f3c0 con 0x55b7ede17bf0 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f040e7fc700 10 client.27787 handle_command_reply: tid=0 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 client.27787 shutdown 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 msgr2=0x55b7ede26a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede26a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04100dfe90 tx=0x7f04100a9eb0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 shutdown_connections 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7ede0bd80 0x55b7ede26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55b7ede17bf0 0x55b7ede18000 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7edc87e40 0x55b7ede26500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 --2- 172.21.15.5:0/3886012044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7ede05e10 0x55b7ede26a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f040e7fc700 0 client.27787 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 >> 172.21.15.5:0/3886012044 conn(0x55b7edccf630 msgr2=0x55b7eddfa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 shutdown_connections 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 1 -- 172.21.15.5:0/3886012044 wait complete. 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.227+0000 7f043bddbb80 20 client.27787 trim_cache size 0 max 16384 2024-05-14T15:36:34.233 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f04140b29f0 msgr2=0x7f04140b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f04140b29f0 0x7f04140b4f00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f042000f590 tx=0x7f0420009920 comp rx=0 tx=0).stop 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 msgr2=0x7f042819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 0x7f042819b3a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f041000ea30 tx=0x7f041000ed40 comp rx=0 tx=0).stop 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 shutdown_connections 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f04140b29f0 0x7f04140b4f00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0428109850 0x7f042819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04281045d0 0x7f042819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 --2- 172.21.15.5:0/3236374647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428108f00 0x7f042819b3a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 >> 172.21.15.5:0/3236374647 conn(0x7f04280ffea0 msgr2=0x7f042810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 shutdown_connections 2024-05-14T15:36:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:34.228+0000 7f042d3c7700 1 -- 172.21.15.5:0/3236374647 wait complete. 2024-05-14T15:36:34.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:34 smithi144 ceph-mon[35650]: pgmap v370: 129 pgs: 129 active+clean; 66 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-05-14T15:36:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:34 smithi193 ceph-mon[36001]: pgmap v370: 129 pgs: 129 active+clean; 66 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-05-14T15:36:35.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1079631 9.18 MB/sec execute 476 sec latency 19.476 ms 2024-05-14T15:36:35.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:36:35.732 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:36.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:36 smithi005 ceph-mon[31559]: pgmap v371: 129 pgs: 129 active+clean; 66 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T15:36:36.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1080404 9.17 MB/sec execute 477 sec latency 41.686 ms 2024-05-14T15:36:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:36 smithi144 ceph-mon[35650]: pgmap v371: 129 pgs: 129 active+clean; 66 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T15:36:36.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:36 smithi193 ceph-mon[36001]: pgmap v371: 129 pgs: 129 active+clean; 66 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-05-14T15:36:37.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1082265 9.17 MB/sec execute 478 sec latency 16.955 ms 2024-05-14T15:36:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 -- 172.21.15.5:0/330946867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 msgr2=0x7f29241038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 --2- 172.21.15.5:0/330946867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29241038d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f29140099c0 tx=0x7f2914009cd0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 -- 172.21.15.5:0/330946867 shutdown_connections 2024-05-14T15:36:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 --2- 172.21.15.5:0/330946867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924108740 0x7f29241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 --2- 172.21.15.5:0/330946867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29241038d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 --2- 172.21.15.5:0/330946867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924107df0 0x7f2924108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 -- 172.21.15.5:0/330946867 >> 172.21.15.5:0/330946867 conn(0x7f29240fedc0 msgr2=0x7f29241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 -- 172.21.15.5:0/330946867 shutdown_connections 2024-05-14T15:36:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.572+0000 7f292a9ad700 1 -- 172.21.15.5:0/330946867 wait complete. 2024-05-14T15:36:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.573+0000 7f292a9ad700 1 Processor -- start 2024-05-14T15:36:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.573+0000 7f292a9ad700 1 -- start start 2024-05-14T15:36:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.573+0000 7f292a9ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29240ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.573+0000 7f292a9ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924107df0 0x7f29240ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.573+0000 7f292a9ad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924108740 0x7f29240ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.573+0000 7f292a9ad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29241133c0 con 0x7f29241034c0 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f292a9ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113240 con 0x7f2924107df0 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f292a9ad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113540 con 0x7f2924108740 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29240ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f29237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924107df0 0x7f29240ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29240ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52564/0 (socket says 172.21.15.5:52564) 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2928f4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924108740 0x7f29240ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 -- 172.21.15.5:0/3345266640 learned_addr learned my addr 172.21.15.5:0/3345266640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 -- 172.21.15.5:0/3345266640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924108740 msgr2=0x7f29240ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924108740 0x7f29240ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 -- 172.21.15.5:0/3345266640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924107df0 msgr2=0x7f29240ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924107df0 0x7f29240ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 -- 172.21.15.5:0/3345266640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29241a91a0 con 0x7f29241034c0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f29237fe700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924107df0 0x7f29240ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2928f4a700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924108740 0x7f29240ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f2923fff700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29240ff2a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2914005910 tx=0x7f2914004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.574+0000 7f29217fa700 1 -- 172.21.15.5:0/3345266640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914026070 con 0x7f29241034c0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.575+0000 7f29217fa700 1 -- 172.21.15.5:0/3345266640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2914003c70 con 0x7f29241034c0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.575+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29241a9430 con 0x7f29241034c0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.575+0000 7f29217fa700 1 -- 172.21.15.5:0/3345266640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400fa40 con 0x7f29241034c0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.575+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a9880 con 0x7f29241034c0 2024-05-14T15:36:37.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.577+0000 7f29217fa700 1 -- 172.21.15.5:0/3345266640 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2914029070 con 0x7f29241034c0 2024-05-14T15:36:37.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.578+0000 7f29217fa700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f290c0b29f0 0x7f290c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.578+0000 7f29237fe700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f290c0b29f0 0x7f290c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.578+0000 7f29217fa700 1 -- 172.21.15.5:0/3345266640 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f29140e3410 con 0x7f29241034c0 2024-05-14T15:36:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.578+0000 7f29393c1b80 1 Processor -- start 2024-05-14T15:36:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 -- start start 2024-05-14T15:36:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f57f38e40 0x557f580bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29237fe700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f290c0b29f0 0x7f290c0b4f00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f29241086b0 tx=0x7f2918009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580b7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f57fd8940 con 0x557f580bcd80 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557f57f3dda0 con 0x557f580b6e10 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29393c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f57fd4900 con 0x557f57f38e40 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f2923fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f57f38e40 0x557f580bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f2928f4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52572/0 (socket says 172.21.15.5:52572) 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29237fe700 1 -- 172.21.15.5:0/313046199 learned_addr learned my addr 172.21.15.5:0/313046199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29237fe700 1 -- 172.21.15.5:0/313046199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f57f38e40 msgr2=0x557f580bc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.579+0000 7f29237fe700 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f57f38e40 0x557f580bc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f29237fe700 1 -- 172.21.15.5:0/313046199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 msgr2=0x557f580b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f29237fe700 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f29237fe700 1 -- 172.21.15.5:0/313046199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f57f258d0 con 0x557f580bcd80 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f2923fff700 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f57f38e40 0x557f580bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f2928f4a700 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f29237fe700 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580b68d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f29180043c0 tx=0x7f2918014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f43f82a11a43ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f290a7fc700 1 -- 172.21.15.5:0/313046199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918003d10 con 0x557f580bcd80 2024-05-14T15:36:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f290a7fc700 1 -- 172.21.15.5:0/313046199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291801e8a0 con 0x557f580bcd80 2024-05-14T15:36:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.580+0000 7f290a7fc700 1 -- 172.21.15.5:0/313046199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800e7a0 con 0x557f580bcd80 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 -- 172.21.15.5:0/313046199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 msgr2=0x557f580b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580b68d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f29180043c0 tx=0x7f2918014000 comp rx=0 tx=0).stop 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 -- 172.21.15.5:0/313046199 shutdown_connections 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f57f38e40 0x557f580bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 --2- 172.21.15.5:0/313046199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580b68d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 -- 172.21.15.5:0/313046199 >> 172.21.15.5:0/313046199 conn(0x557f57f80630 msgr2=0x557f580b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 -- 172.21.15.5:0/313046199 shutdown_connections 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 -- 172.21.15.5:0/313046199 wait complete. 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.581+0000 7f29393c1b80 1 -- start start 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 Processor -- start 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580bcd80 0x557f580c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f580c3820 con 0x557f57f38e40 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557f57f91ed0 con 0x557f580b6e10 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557f580c39a0 con 0x557f580bcd80 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f2923fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29393c1b80 10 client.0 _ll_register_callbacks cb 0x557f57f70950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f2928f4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580bcd80 0x557f580c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f2923fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52576/0 (socket says 172.21.15.5:52576) 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f2923fff700 1 -- 172.21.15.5:0/2795422781 learned_addr learned my addr 172.21.15.5:0/2795422781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.582+0000 7f29237fe700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580c0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44904/0 (socket says 172.21.15.5:44904) 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2928f4a700 1 -- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 msgr2=0x557f580c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2928f4a700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580c0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2928f4a700 1 -- 172.21.15.5:0/2795422781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 msgr2=0x557f580c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2928f4a700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2928f4a700 1 -- 172.21.15.5:0/2795422781 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f57f8f760 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2923fff700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f2928f4a700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580bcd80 0x557f580c0f70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f291c0046a0 tx=0x7f291c0049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34c986526231503 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29037fe700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29037fe700 1 -- 172.21.15.5:0/2795422781 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c004db0 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291801e8a0 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29037fe700 1 -- 172.21.15.5:0/2795422781 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f291c00ea20 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29037fe700 1 -- 172.21.15.5:0/2795422781 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f291c026050 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29393c1b80 10 client.27799 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557f580ce4b0 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.583+0000 7f29037fe700 1 -- 172.21.15.5:0/2795422781 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291801e8a0 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29037fe700 1 -- 172.21.15.5:0/2795422781 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f291c007ce0 con 0x557f580bcd80 2024-05-14T15:36:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29393c1b80 10 client.27799 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29393c1b80 10 client.27799 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29393c1b80 20 client.27799 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f580d0af0 0x557f580d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29393c1b80 4 client.27799 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f57fe6320 con 0x557f580d0af0 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f2923fff700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f580d0af0 0x557f580d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f29037fe700 10 client.27799 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.584+0000 7f2923fff700 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f580d0af0 0x557f580d2f60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77896905de77ef0 server_cookie=b5ed6345106dc8bf in_seq=0 out_seq=0 2024-05-14T15:36:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29037fe700 1 -- 172.21.15.5:0/2795422781 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f57fe6320 con 0x557f580d0af0 2024-05-14T15:36:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29037fe700 10 client.27799 handle_command_reply: tid=0 2024-05-14T15:36:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 client.27799 shutdown 2024-05-14T15:36:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580bcd80 msgr2=0x557f580c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580bcd80 0x557f580c0f70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f291c0046a0 tx=0x7f291c0049b0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 shutdown_connections 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f580d0af0 0x557f580d2f60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580bcd80 0x557f580c0f70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.585+0000 7f29037fe700 0 client.27799 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.586+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2795422781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580b6e10 0x557f580c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.586+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 >> 172.21.15.5:0/2795422781 conn(0x557f57f80630 msgr2=0x557f580b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.586+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 shutdown_connections 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.586+0000 7f29393c1b80 1 -- 172.21.15.5:0/2795422781 wait complete. 2024-05-14T15:36:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.586+0000 7f29393c1b80 20 client.27799 trim_cache size 0 max 16384 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 Processor -- start 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 -- start start 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 0x557f580b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580be1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b5130 con 0x557f580bcd80 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b29c0 con 0x557f57f38e40 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29393c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b0830 con 0x557f580b6e10 2024-05-14T15:36:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f2928f4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f2923fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 0x557f580b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f2928f4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52584/0 (socket says 172.21.15.5:52584) 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f2923fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 0x557f580b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44908/0 (socket says 172.21.15.5:44908) 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f2928f4a700 1 -- 172.21.15.5:0/3859419531 learned_addr learned my addr 172.21.15.5:0/3859419531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.602+0000 7f29237fe700 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580be1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2928f4a700 1 -- 172.21.15.5:0/3859419531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 msgr2=0x557f580be1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2928f4a700 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580be1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2928f4a700 1 -- 172.21.15.5:0/3859419531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 msgr2=0x557f580b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2928f4a700 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 0x557f580b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2928f4a700 1 -- 172.21.15.5:0/3859419531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f580bea80 con 0x557f580bcd80 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2923fff700 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 0x557f580b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2928f4a700 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580cc8b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f291c01e400 tx=0x7f291c01e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e08d86fd58b8e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2909ffb700 1 -- 172.21.15.5:0/3859419531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c004ce0 con 0x557f580bcd80 2024-05-14T15:36:37.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2909ffb700 1 -- 172.21.15.5:0/3859419531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291c031070 con 0x557f580bcd80 2024-05-14T15:36:37.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.603+0000 7f2909ffb700 1 -- 172.21.15.5:0/3859419531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c007500 con 0x557f580bcd80 2024-05-14T15:36:37.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 -- 172.21.15.5:0/3859419531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 msgr2=0x557f580cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580cc8b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f291c01e400 tx=0x7f291c01e770 comp rx=0 tx=0).stop 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 -- 172.21.15.5:0/3859419531 shutdown_connections 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f580bcd80 0x557f580cc8b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f57f38e40 0x557f580b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 --2- 172.21.15.5:0/3859419531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580be1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 -- 172.21.15.5:0/3859419531 >> 172.21.15.5:0/3859419531 conn(0x557f57f80630 msgr2=0x557f580bb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 -- 172.21.15.5:0/3859419531 shutdown_connections 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 -- 172.21.15.5:0/3859419531 wait complete. 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.604+0000 7f29393c1b80 1 -- start start 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 Processor -- start 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580d7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580bcd80 0x557f580d7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b0e80 con 0x557f57f38e40 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b22f0 con 0x557f580bcd80 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b1550 con 0x557f580b6e10 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29393c1b80 10 client.0 _ll_register_callbacks cb 0x557f57f70950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f2923fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f2923fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52588/0 (socket says 172.21.15.5:52588) 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f2923fff700 1 -- 172.21.15.5:0/2495840837 learned_addr learned my addr 172.21.15.5:0/2495840837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f29237fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580d7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f2928f4a700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580bcd80 0x557f580d7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.605+0000 7f2923fff700 1 -- 172.21.15.5:0/2495840837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 msgr2=0x557f580d7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f2923fff700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580d7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f2923fff700 1 -- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580bcd80 msgr2=0x557f580d7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f2923fff700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580bcd80 0x557f580d7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f2923fff700 1 -- 172.21.15.5:0/2495840837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f580c95f0 con 0x557f57f38e40 2024-05-14T15:36:37.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f2928f4a700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580bcd80 0x557f580d7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29237fe700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580d7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f2923fff700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c8e30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f29140aa070 tx=0x7f29140036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c7ad104f47dfa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29037fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29037fe700 1 -- 172.21.15.5:0/2495840837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140aaa70 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29037fe700 1 -- 172.21.15.5:0/2495840837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f29140aa7e0 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291c031070 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29037fe700 1 -- 172.21.15.5:0/2495840837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f29140aa370 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29393c1b80 10 client.18246 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557f5813ad70 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.606+0000 7f29037fe700 1 -- 172.21.15.5:0/2495840837 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291c031070 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29037fe700 1 -- 172.21.15.5:0/2495840837 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f29140a95d0 con 0x557f57f38e40 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 10 client.18246 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 10 client.18246 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 20 client.18246 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:37.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f5813b450 0x557f5813d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 4 client.18246 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f57fe03c0 con 0x557f5813b450 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29237fe700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f5813b450 0x557f5813d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29237fe700 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f5813b450 0x557f5813d870 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcaa0c0b0d54f17c server_cookie=1a6b6083156d4b9e in_seq=0 out_seq=0 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29037fe700 10 client.18246 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29037fe700 1 -- 172.21.15.5:0/2495840837 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f57fe03c0 con 0x557f5813b450 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29037fe700 10 client.18246 handle_command_reply: tid=0 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.607+0000 7f29393c1b80 1 client.18246 shutdown 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 msgr2=0x557f580c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c8e30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f29140aa070 tx=0x7f29140036f0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 shutdown_connections 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x557f5813b450 0x557f5813d870 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557f580bcd80 0x557f580d7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f57f38e40 0x557f580c8e30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 --2- 172.21.15.5:0/2495840837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557f580b6e10 0x557f580d7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 >> 172.21.15.5:0/2495840837 conn(0x557f57f80630 msgr2=0x557f580bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29037fe700 0 client.18246 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 shutdown_connections 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 1 -- 172.21.15.5:0/2495840837 wait complete. 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.608+0000 7f29393c1b80 20 client.18246 trim_cache size 0 max 16384 2024-05-14T15:36:37.615 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f290c0b29f0 msgr2=0x7f290c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f290c0b29f0 0x7f290c0b4f00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f29241086b0 tx=0x7f2918009260 comp rx=0 tx=0).stop 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 msgr2=0x7f29240ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29240ff2a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2914005910 tx=0x7f2914004270 comp rx=0 tx=0).stop 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 shutdown_connections 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f290c0b29f0 0x7f290c0b4f00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2924108740 0x7f29240ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29241034c0 0x7f29240ff2a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 --2- 172.21.15.5:0/3345266640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2924107df0 0x7f29240ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:37.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 >> 172.21.15.5:0/3345266640 conn(0x7f29240fedc0 msgr2=0x7f292406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:37.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 shutdown_connections 2024-05-14T15:36:37.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:37.609+0000 7f292a9ad700 1 -- 172.21.15.5:0/3345266640 wait complete. 2024-05-14T15:36:38.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1083782 9.18 MB/sec execute 479 sec latency 38.954 ms 2024-05-14T15:36:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:38 smithi144 ceph-mon[35650]: pgmap v372: 129 pgs: 129 active+clean; 62 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-05-14T15:36:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:38 smithi193 ceph-mon[36001]: pgmap v372: 129 pgs: 129 active+clean; 62 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-05-14T15:36:38.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:38 smithi005 ceph-mon[31559]: pgmap v372: 129 pgs: 129 active+clean; 62 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 277 op/s 2024-05-14T15:36:38.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:36:38.760 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:39.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1085011 9.17 MB/sec execute 480 sec latency 25.002 ms 2024-05-14T15:36:40.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1086410 9.16 MB/sec execute 481 sec latency 39.347 ms 2024-05-14T15:36:40.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:40 smithi144 ceph-mon[35650]: pgmap v373: 129 pgs: 129 active+clean; 67 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 312 op/s 2024-05-14T15:36:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:40 smithi193 ceph-mon[36001]: pgmap v373: 129 pgs: 129 active+clean; 67 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 312 op/s 2024-05-14T15:36:40.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:40 smithi005 ceph-mon[31559]: pgmap v373: 129 pgs: 129 active+clean; 67 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 312 op/s 2024-05-14T15:36:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 -- 172.21.15.5:0/1983436303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44108740 msgr2=0x7fbc441100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/1983436303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44108740 0x7fbc441100a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00b0d0 tx=0x7fbc3c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 -- 172.21.15.5:0/1983436303 shutdown_connections 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/1983436303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44108740 0x7fbc441100a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/1983436303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc441034c0 0x7fbc441038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/1983436303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc44107df0 0x7fbc44108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 -- 172.21.15.5:0/1983436303 >> 172.21.15.5:0/1983436303 conn(0x7fbc440fedc0 msgr2=0x7fbc441011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 -- 172.21.15.5:0/1983436303 shutdown_connections 2024-05-14T15:36:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.081+0000 7fbc48a6b700 1 -- 172.21.15.5:0/1983436303 wait complete. 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.082+0000 7fbc48a6b700 1 Processor -- start 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.082+0000 7fbc48a6b700 1 -- start start 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc48a6b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc441034c0 0x7fbc4419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc48a6b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44107df0 0x7fbc4419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc48a6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 0x7fbc4419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc48a6b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44113390 con 0x7fbc44107df0 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc48a6b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44113210 con 0x7fbc441034c0 2024-05-14T15:36:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc48a6b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44113510 con 0x7fbc44108740 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 0x7fbc4419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 0x7fbc4419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36124/0 (socket says 172.21.15.5:36124) 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 -- 172.21.15.5:0/2631928745 learned_addr learned my addr 172.21.15.5:0/2631928745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44107df0 0x7fbc4419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc441034c0 0x7fbc4419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 -- 172.21.15.5:0/2631928745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc441034c0 msgr2=0x7fbc4419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc441034c0 0x7fbc4419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 -- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44107df0 msgr2=0x7fbc4419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44107df0 0x7fbc4419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.083+0000 7fbc42d9d700 1 -- 172.21.15.5:0/2631928745 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4419fca0 con 0x7fbc44108740 2024-05-14T15:36:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.084+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 0x7fbc4419f600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007a00 tx=0x7fbc3c007ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.084+0000 7fbc337fe700 1 -- 172.21.15.5:0/2631928745 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c01f070 con 0x7fbc44108740 2024-05-14T15:36:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.084+0000 7fbc337fe700 1 -- 172.21.15.5:0/2631928745 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc3c005520 con 0x7fbc44108740 2024-05-14T15:36:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.084+0000 7fbc337fe700 1 -- 172.21.15.5:0/2631928745 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c01b670 con 0x7fbc44108740 2024-05-14T15:36:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.084+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4419ff30 con 0x7fbc44108740 2024-05-14T15:36:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.084+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc441a78b0 con 0x7fbc44108740 2024-05-14T15:36:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.087+0000 7fbc337fe700 1 -- 172.21.15.5:0/2631928745 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbc3c022070 con 0x7fbc44108740 2024-05-14T15:36:41.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.088+0000 7fbc337fe700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.088+0000 7fbc337fe700 1 -- 172.21.15.5:0/2631928745 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fbc3c00f020 con 0x7fbc44108740 2024-05-14T15:36:41.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.088+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.088+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009b30 tx=0x7fbc34009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:41.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.089+0000 7fbc5747fb80 1 Processor -- start 2024-05-14T15:36:41.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.089+0000 7fbc5747fb80 1 -- start start 2024-05-14T15:36:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.089+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e88a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.089+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e88ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.089+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e88f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.089+0000 7fbc41d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e88ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc41d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e88ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52618/0 (socket says 172.21.15.5:52618) 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc41d9b700 1 -- 172.21.15.5:0/1111661613 learned_addr learned my addr 172.21.15.5:0/1111661613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a4e7ac860 con 0x557a4e88ac60 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a4e74b7c0 con 0x557a4e88f470 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a4e7a8820 con 0x557a4e70ce40 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e88a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e88f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 -- 172.21.15.5:0/1111661613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 msgr2=0x557a4e88f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e88f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 -- 172.21.15.5:0/1111661613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 msgr2=0x557a4e88ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e88ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 -- 172.21.15.5:0/1111661613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4e6f98d0 con 0x557a4e70ce40 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e88ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.090+0000 7fbc4259c700 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e88a720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc34010b80 tx=0x7fbc34013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6203e64953f998e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.091+0000 7fbc30ff9700 1 -- 172.21.15.5:0/1111661613 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34023070 con 0x557a4e70ce40 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.091+0000 7fbc30ff9700 1 -- 172.21.15.5:0/1111661613 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc34010e90 con 0x557a4e70ce40 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.091+0000 7fbc30ff9700 1 -- 172.21.15.5:0/1111661613 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3401e3b0 con 0x557a4e70ce40 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 msgr2=0x557a4e88a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e88a720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc34010b80 tx=0x7fbc34013000 comp rx=0 tx=0).stop 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 shutdown_connections 2024-05-14T15:36:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e88f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e88a720 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/1111661613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e88ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 >> 172.21.15.5:0/1111661613 conn(0x557a4e754600 msgr2=0x557a4e8854b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 shutdown_connections 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 -- 172.21.15.5:0/1111661613 wait complete. 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.092+0000 7fbc5747fb80 1 -- start start 2024-05-14T15:36:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 Processor -- start 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e894190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e894c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a4e897420 con 0x557a4e88ac60 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a4e712c30 con 0x557a4e88f470 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a4e8975a0 con 0x557a4e70ce40 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc5747fb80 10 client.0 _ll_register_callbacks cb 0x557a4e744920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc41d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc41d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52632/0 (socket says 172.21.15.5:52632) 2024-05-14T15:36:41.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc41d9b700 1 -- 172.21.15.5:0/2090528277 learned_addr learned my addr 172.21.15.5:0/2090528277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e894c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e894190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc4259c700 1 -- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 msgr2=0x557a4e894c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e894c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc4259c700 1 -- 172.21.15.5:0/2090528277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 msgr2=0x557a4e8946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc4259c700 1 -- 172.21.15.5:0/2090528277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4e748690 con 0x557a4e70ce40 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.093+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e894c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e894190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbc34010650 tx=0x7fbc3401f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a263500e42af01ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2090528277 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34037060 con 0x557a4e70ce40 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2090528277 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fbc3401fdb0 con 0x557a4e70ce40 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc34010e90 con 0x557a4e70ce40 2024-05-14T15:36:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2090528277 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc3403e050 con 0x557a4e70ce40 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc5747fb80 10 client.27817 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a4e8a2040 con 0x557a4e70ce40 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.094+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2090528277 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc34010e90 con 0x557a4e70ce40 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2090528277 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fbc3400f040 con 0x557a4e70ce40 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc5747fb80 10 client.27817 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc5747fb80 10 client.27817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc5747fb80 20 client.27817 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a45c0 0x557a4e8a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc5747fb80 4 client.27817 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4e7ba240 con 0x557a4e8a45c0 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a45c0 0x557a4e8a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc29ffb700 10 client.27817 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.095+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a45c0 0x557a4e8a69e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b864ab9350e8dd78 server_cookie=7a06fcd759692235 in_seq=0 out_seq=0 2024-05-14T15:36:41.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2090528277 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a4e7ba240 con 0x557a4e8a45c0 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc29ffb700 10 client.27817 handle_command_reply: tid=0 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 client.27817 shutdown 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 msgr2=0x557a4e894190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e894190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbc34010650 tx=0x7fbc3401f680 comp rx=0 tx=0).stop 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 shutdown_connections 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a45c0 0x557a4e8a69e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88f470 0x557a4e894c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e70ce40 0x557a4e894190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc29ffb700 0 client.27817 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2090528277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 >> 172.21.15.5:0/2090528277 conn(0x557a4e754600 msgr2=0x557a4e889450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 shutdown_connections 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2090528277 wait complete. 2024-05-14T15:36:41.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.097+0000 7fbc5747fb80 20 client.27817 trim_cache size 0 max 16384 2024-05-14T15:36:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.114+0000 7fbc5747fb80 1 Processor -- start 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.114+0000 7fbc5747fb80 1 -- start start 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e70ce40 0x557a4e8934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e894a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441b2f40 con 0x557a4e88ac60 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441b07d0 con 0x557a4e70ce40 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441ae640 con 0x557a4e88f470 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc42d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e894a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc4259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e70ce40 0x557a4e8934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52634/0 (socket says 172.21.15.5:52634) 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc42d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e894a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36152/0 (socket says 172.21.15.5:36152) 2024-05-14T15:36:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 -- 172.21.15.5:0/3697834046 learned_addr learned my addr 172.21.15.5:0/3697834046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc4259c700 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e70ce40 0x557a4e8934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44948/0 (socket says 172.21.15.5:44948) 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 -- 172.21.15.5:0/3697834046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 msgr2=0x557a4e894a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e894a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 -- 172.21.15.5:0/3697834046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e70ce40 msgr2=0x557a4e8934b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e70ce40 0x557a4e8934b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 -- 172.21.15.5:0/3697834046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4e87fdf0 con 0x557a4e88ac60 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e894a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.115+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8a0840 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x557a4e87f060 tx=0x7fbc38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b1f6c694ba3a0b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.116+0000 7fbc2bfff700 1 -- 172.21.15.5:0/3697834046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38026070 con 0x557a4e88ac60 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.116+0000 7fbc2bfff700 1 -- 172.21.15.5:0/3697834046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38020660 con 0x557a4e88ac60 2024-05-14T15:36:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.116+0000 7fbc2bfff700 1 -- 172.21.15.5:0/3697834046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc380116e0 con 0x557a4e88ac60 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 -- 172.21.15.5:0/3697834046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 msgr2=0x557a4e8a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8a0840 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x557a4e87f060 tx=0x7fbc38009260 comp rx=0 tx=0).stop 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 -- 172.21.15.5:0/3697834046 shutdown_connections 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e894a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e70ce40 0x557a4e8934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/3697834046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e88ac60 0x557a4e8a0840 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 -- 172.21.15.5:0/3697834046 >> 172.21.15.5:0/3697834046 conn(0x557a4e754600 msgr2=0x557a4e884be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 -- 172.21.15.5:0/3697834046 shutdown_connections 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 -- 172.21.15.5:0/3697834046 wait complete. 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 -- start start 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.117+0000 7fbc5747fb80 1 Processor -- start 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 0x557a4e908330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88ac60 0x557a4e906ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e907000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441aec90 con 0x557a4e70ce40 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441b0100 con 0x557a4e88ac60 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441af360 con 0x557a4e88f470 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc5747fb80 10 client.0 _ll_register_callbacks cb 0x557a4e744920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 0x557a4e908330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 0x557a4e908330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52636/0 (socket says 172.21.15.5:52636) 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 -- 172.21.15.5:0/2934995157 learned_addr learned my addr 172.21.15.5:0/2934995157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88ac60 0x557a4e906ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc42d9d700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e907000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 -- 172.21.15.5:0/2934995157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 msgr2=0x557a4e907000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e907000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 -- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88ac60 msgr2=0x557a4e906ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88ac60 0x557a4e906ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 -- 172.21.15.5:0/2934995157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4e9076e0 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc4259c700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 0x557a4e908330 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x557a4e893a10 tx=0x7fbc3401fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f454e9f894499817 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.118+0000 7fbc29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2934995157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34047060 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2934995157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fbc34028660 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc38020660 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2934995157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc3404e050 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 10 client.18264 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a4e8a9880 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2934995157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38020660 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2934995157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fbc34032590 con 0x557a4e70ce40 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 10 client.18264 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 20 client.18264 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.119+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a9f30 0x557a4e914f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc5747fb80 4 client.18264 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a4e7b42e0 con 0x557a4e8a9f30 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a9f30 0x557a4e914f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc29ffb700 10 client.18264 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc41d9b700 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a9f30 0x557a4e914f20 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcd5c37338d6ec4 server_cookie=3dd925327ece5ad6 in_seq=0 out_seq=0 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc29ffb700 1 -- 172.21.15.5:0/2934995157 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a4e7b42e0 con 0x557a4e8a9f30 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.120+0000 7fbc29ffb700 10 client.18264 handle_command_reply: tid=0 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 client.18264 shutdown 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 msgr2=0x557a4e908330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 0x557a4e908330 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x557a4e893a10 tx=0x7fbc3401fb00 comp rx=0 tx=0).stop 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 shutdown_connections 2024-05-14T15:36:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x557a4e8a9f30 0x557a4e914f20 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a4e88f470 0x557a4e907000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a4e70ce40 0x557a4e908330 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 --2- 172.21.15.5:0/2934995157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a4e88ac60 0x557a4e906ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 >> 172.21.15.5:0/2934995157 conn(0x557a4e754600 msgr2=0x557a4e893210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc29ffb700 0 client.18264 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 shutdown_connections 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 1 -- 172.21.15.5:0/2934995157 wait complete. 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.121+0000 7fbc5747fb80 20 client.18264 trim_cache size 0 max 16384 2024-05-14T15:36:41.127 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbc2c0b2a70 msgr2=0x7fbc2c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009b30 tx=0x7fbc34009260 comp rx=0 tx=0).stop 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 msgr2=0x7fbc4419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 0x7fbc4419f600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007a00 tx=0x7fbc3c007ae0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 shutdown_connections 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc44108740 0x7fbc4419f600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc441034c0 0x7fbc4419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 --2- 172.21.15.5:0/2631928745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc44107df0 0x7fbc4419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 >> 172.21.15.5:0/2631928745 conn(0x7fbc440fedc0 msgr2=0x7fbc4406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 shutdown_connections 2024-05-14T15:36:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:41.122+0000 7fbc48a6b700 1 -- 172.21.15.5:0/2631928745 wait complete. 2024-05-14T15:36:41.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1087462 9.16 MB/sec execute 482 sec latency 31.801 ms 2024-05-14T15:36:41.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:41 smithi144 ceph-mon[35650]: pgmap v374: 129 pgs: 129 active+clean; 67 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 249 op/s 2024-05-14T15:36:41.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:41 smithi193 ceph-mon[36001]: pgmap v374: 129 pgs: 129 active+clean; 67 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 249 op/s 2024-05-14T15:36:41.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:41 smithi005 ceph-mon[31559]: pgmap v374: 129 pgs: 129 active+clean; 67 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 249 op/s 2024-05-14T15:36:42.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1089308 9.15 MB/sec execute 483 sec latency 32.071 ms 2024-05-14T15:36:43.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-14T15:36:43.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-14T15:36:43.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1090724 9.16 MB/sec execute 484 sec latency 170.512 ms 2024-05-14T15:36:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:44.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:43 smithi005 ceph-mon[31559]: pgmap v375: 129 pgs: 129 active+clean; 75 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-05-14T15:36:44.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1092081 9.15 MB/sec execute 485 sec latency 39.591 ms 2024-05-14T15:36:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:43 smithi144 ceph-mon[35650]: pgmap v375: 129 pgs: 129 active+clean; 75 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-05-14T15:36:44.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:43 smithi193 ceph-mon[36001]: pgmap v375: 129 pgs: 129 active+clean; 75 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-05-14T15:36:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 -- 172.21.15.5:0/1251831218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 msgr2=0x7f4ae8102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 --2- 172.21.15.5:0/1251831218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae8102740 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009900 tx=0x7f4ae402a410 comp rx=0 tx=0).stop 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 -- 172.21.15.5:0/1251831218 shutdown_connections 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 --2- 172.21.15.5:0/1251831218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae81032f0 0x7f4ae810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 --2- 172.21.15.5:0/1251831218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ae8109040 0x7f4ae8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 --2- 172.21.15.5:0/1251831218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae8102740 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 -- 172.21.15.5:0/1251831218 >> 172.21.15.5:0/1251831218 conn(0x7f4ae80782e0 msgr2=0x7f4ae8078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 -- 172.21.15.5:0/1251831218 shutdown_connections 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.912+0000 7f4aefad8640 1 -- 172.21.15.5:0/1251831218 wait complete. 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 Processor -- start 2024-05-14T15:36:44.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 -- start start 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae81032f0 0x7f4ae80ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae80ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ae8109040 0x7f4ae80ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8112de0 con 0x7f4ae8108ba0 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8112c60 con 0x7f4ae8109040 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aefad8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8112f60 con 0x7f4ae81032f0 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae80ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae80ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52652/0 (socket says 172.21.15.5:52652) 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 -- 172.21.15.5:0/3267296781 learned_addr learned my addr 172.21.15.5:0/3267296781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed874640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae81032f0 0x7f4ae80ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aee075640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ae8109040 0x7f4ae80ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 -- 172.21.15.5:0/3267296781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae81032f0 msgr2=0x7f4ae80ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae81032f0 0x7f4ae80ff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 -- 172.21.15.5:0/3267296781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ae8109040 msgr2=0x7f4ae80ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ae8109040 0x7f4ae80ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.913+0000 7f4aed073640 1 -- 172.21.15.5:0/3267296781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae81a8c50 con 0x7f4ae8108ba0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.914+0000 7f4aed073640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae80ff8c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800e9f0 tx=0x7f4ad800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.914+0000 7f4adeffd640 1 -- 172.21.15.5:0/3267296781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800ccc0 con 0x7f4ae8108ba0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.914+0000 7f4adeffd640 1 -- 172.21.15.5:0/3267296781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad80035f0 con 0x7f4ae8108ba0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.914+0000 7f4adeffd640 1 -- 172.21.15.5:0/3267296781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800ce60 con 0x7f4ae8108ba0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.914+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae81a8f40 con 0x7f4ae8108ba0 2024-05-14T15:36:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.914+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae81a9400 con 0x7f4ae8108ba0 2024-05-14T15:36:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.917+0000 7f4adeffd640 1 -- 172.21.15.5:0/3267296781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4ad8003aa0 con 0x7f4ae8108ba0 2024-05-14T15:36:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.917+0000 7f4af6983740 1 Processor -- start 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.917+0000 7f4af6983740 1 -- start start 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4af6983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec611b0 0x55c8ef001be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4af6983740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef002120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4af6983740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef002660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4af6983740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8eef675d0 con 0x55c8eec404d0 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4af6983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8eef20890 con 0x55c8eec611b0 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4af6983740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c8eef63590 con 0x55c8eec382f0 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef002120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef002660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aee075640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec611b0 0x55c8ef001be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef002120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36182/0 (socket says 172.21.15.5:36182) 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aee075640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec611b0 0x55c8ef001be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:44990/0 (socket says 172.21.15.5:44990) 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 -- 172.21.15.5:0/3657591867 learned_addr learned my addr 172.21.15.5:0/3657591867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 -- 172.21.15.5:0/3657591867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec611b0 msgr2=0x55c8ef001be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4adeffd640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ad40b29d0 0x7f4ad40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec611b0 0x55c8ef001be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 -- 172.21.15.5:0/3657591867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 msgr2=0x55c8ef002660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef002660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aed874640 1 -- 172.21.15.5:0/3657591867 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eec79210 con 0x55c8eec382f0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.918+0000 7f4aee075640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ad40b29d0 0x7f4ad40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.919+0000 7f4adeffd640 1 -- 172.21.15.5:0/3267296781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f4ad80e2c30 con 0x7f4ae8108ba0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.919+0000 7f4aed874640 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef002120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8103880 tx=0x7f4ae4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88ffd0a701ba1d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.919+0000 7f4ad3fff640 1 -- 172.21.15.5:0/3657591867 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4035040 con 0x55c8eec382f0 2024-05-14T15:36:44.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.919+0000 7f4ad3fff640 1 -- 172.21.15.5:0/3657591867 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae40054f0 con 0x55c8eec382f0 2024-05-14T15:36:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.919+0000 7f4ad3fff640 1 -- 172.21.15.5:0/3657591867 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40035f0 con 0x55c8eec382f0 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4aee075640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ad40b29d0 0x7f4ad40b4ee0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009fd0 tx=0x7f4ae0004a40 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 -- 172.21.15.5:0/3657591867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 msgr2=0x55c8ef002120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef002120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8103880 tx=0x7f4ae4004040 comp rx=0 tx=0).stop 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 -- 172.21.15.5:0/3657591867 shutdown_connections 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef002120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef002660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 --2- 172.21.15.5:0/3657591867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec611b0 0x55c8ef001be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 -- 172.21.15.5:0/3657591867 >> 172.21.15.5:0/3657591867 conn(0x55c8eec55300 msgr2=0x55c8eeffe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 -- 172.21.15.5:0/3657591867 shutdown_connections 2024-05-14T15:36:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.920+0000 7f4af6983740 1 -- 172.21.15.5:0/3657591867 wait complete. 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 -- start start 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 Processor -- start 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec404d0 0x55c8ef00c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec611b0 0x55c8ef012130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8ef00ecb0 con 0x55c8eec611b0 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8eef24370 con 0x55c8eec404d0 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c8ef00ee30 con 0x55c8eec382f0 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4af6983740 10 client.0 _ll_register_callbacks cb 0x55c8eec78a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4aed874640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4aee075640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec611b0 0x55c8ef012130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4aed874640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36188/0 (socket says 172.21.15.5:36188) 2024-05-14T15:36:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4aed874640 1 -- 172.21.15.5:0/3363850389 learned_addr learned my addr 172.21.15.5:0/3363850389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.921+0000 7f4aed073640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec404d0 0x55c8ef00c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aed874640 1 -- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec404d0 msgr2=0x55c8ef00c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aed874640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec404d0 0x55c8ef00c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aed874640 1 -- 172.21.15.5:0/3363850389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec611b0 msgr2=0x55c8ef012130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aed874640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec611b0 0x55c8ef012130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aed874640 1 -- 172.21.15.5:0/3363850389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8eed45620 con 0x55c8eec382f0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aee075640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec611b0 0x55c8ef012130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4aed874640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00c230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4031440 tx=0x7f4ae4004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78ac8c37ada2f437 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4ad17fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4ad17fa640 1 -- 172.21.15.5:0/3363850389 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4035050 con 0x55c8eec382f0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4ad17fa640 1 -- 172.21.15.5:0/3363850389 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f4ae4004d10 con 0x55c8eec382f0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae40054f0 con 0x55c8eec382f0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4ad17fa640 1 -- 172.21.15.5:0/3363850389 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4ae4009520 con 0x55c8eec382f0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4af6983740 10 client.27835 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.922+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8ef00ecb0 con 0x55c8eec382f0 2024-05-14T15:36:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4ad17fa640 1 -- 172.21.15.5:0/3363850389 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae40054f0 con 0x55c8eec382f0 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4ad17fa640 1 -- 172.21.15.5:0/3363850389 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae4003a80 con 0x55c8eec382f0 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4af6983740 10 client.27835 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4af6983740 10 client.27835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4af6983740 20 client.27835 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4af6983740 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec1fd80 0x55c8ef01e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4af6983740 4 client.27835 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8eef751e0 con 0x55c8eec1fd80 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4aed073640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec1fd80 0x55c8ef01e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4ad17fa640 10 client.27835 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.923+0000 7f4aed073640 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec1fd80 0x55c8ef01e1f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7885c8d14f463b server_cookie=289b3e852ddece65 in_seq=0 out_seq=0 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4ad17fa640 1 -- 172.21.15.5:0/3363850389 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c8eef751e0 con 0x55c8eec1fd80 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4ad17fa640 10 client.27835 handle_command_reply: tid=0 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 client.27835 shutdown 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 msgr2=0x55c8ef00c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00c230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4031440 tx=0x7f4ae4004520 comp rx=0 tx=0).stop 2024-05-14T15:36:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 shutdown_connections 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec1fd80 0x55c8ef01e1f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec611b0 0x55c8ef012130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec404d0 0x55c8ef00c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 --2- 172.21.15.5:0/3363850389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00c230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4ad17fa640 0 client.27835 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 >> 172.21.15.5:0/3363850389 conn(0x55c8eec55300 msgr2=0x55c8ef001a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 shutdown_connections 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 1 -- 172.21.15.5:0/3363850389 wait complete. 2024-05-14T15:36:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.926+0000 7f4af6983740 20 client.27835 trim_cache size 0 max 16384 2024-05-14T15:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 Processor -- start 2024-05-14T15:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 -- start start 2024-05-14T15:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec55300 0x55c8ef0023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef00e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b5f80 con 0x55c8eec404d0 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b3850 con 0x55c8eec55300 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4af6983740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b16c0 con 0x55c8eec382f0 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef00e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef00e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52680/0 (socket says 172.21.15.5:52680) 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 -- 172.21.15.5:0/4142429365 learned_addr learned my addr 172.21.15.5:0/4142429365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aed874640 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec55300 0x55c8ef0023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aed073640 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 -- 172.21.15.5:0/4142429365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 msgr2=0x55c8ef00de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 -- 172.21.15.5:0/4142429365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec55300 msgr2=0x55c8ef0023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec55300 0x55c8ef0023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aee075640 1 -- 172.21.15.5:0/4142429365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ef00cf40 con 0x55c8eec404d0 2024-05-14T15:36:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.942+0000 7f4aed874640 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec55300 0x55c8ef0023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.943+0000 7f4aee075640 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef00e390 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000fcc0 tx=0x7f4ae0013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9796552c3f0ccb66 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.943+0000 7f4ad37fe640 1 -- 172.21.15.5:0/4142429365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae002e070 con 0x55c8eec404d0 2024-05-14T15:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.943+0000 7f4ad37fe640 1 -- 172.21.15.5:0/4142429365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae001e0d0 con 0x55c8eec404d0 2024-05-14T15:36:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.943+0000 7f4ad37fe640 1 -- 172.21.15.5:0/4142429365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae001c680 con 0x55c8eec404d0 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 -- 172.21.15.5:0/4142429365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 msgr2=0x55c8ef00e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef00e390 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000fcc0 tx=0x7f4ae0013030 comp rx=0 tx=0).stop 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 -- 172.21.15.5:0/4142429365 shutdown_connections 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec382f0 0x55c8ef00de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef00e390 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 --2- 172.21.15.5:0/4142429365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec55300 0x55c8ef0023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 -- 172.21.15.5:0/4142429365 >> 172.21.15.5:0/4142429365 conn(0x55c8ef0060a0 msgr2=0x55c8ef00d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 -- 172.21.15.5:0/4142429365 shutdown_connections 2024-05-14T15:36:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.944+0000 7f4af6983740 1 -- 172.21.15.5:0/4142429365 wait complete. 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.945+0000 7f4af6983740 1 -- start start 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.945+0000 7f4af6983740 1 Processor -- start 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec382f0 0x55c8ef0184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef018a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 0x55c8ef0159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b1d10 con 0x55c8eec404d0 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b3180 con 0x55c8eec382f0 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b23e0 con 0x55c8eec55300 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4af6983740 10 client.0 _ll_register_callbacks cb 0x7f4ad80e6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aed073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef018a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 0x55c8ef0159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 0x55c8ef0159d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:36218/0 (socket says 172.21.15.5:36218) 2024-05-14T15:36:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aed073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef018a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52696/0 (socket says 172.21.15.5:52696) 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 -- 172.21.15.5:0/1025122719 learned_addr learned my addr 172.21.15.5:0/1025122719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aed874640 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec382f0 0x55c8ef0184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 -- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec382f0 msgr2=0x55c8ef0184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec382f0 0x55c8ef0184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 -- 172.21.15.5:0/1025122719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 msgr2=0x55c8ef018a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef018a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.946+0000 7f4aee075640 1 -- 172.21.15.5:0/1025122719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ef016010 con 0x55c8eec55300 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4aee075640 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 0x55c8ef0159d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009fa0 tx=0x7f4ae001e310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524e6e44ed1a89f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4ad17fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4ad17fa640 1 -- 172.21.15.5:0/1025122719 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0038020 con 0x55c8eec55300 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4ad17fa640 1 -- 172.21.15.5:0/1025122719 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f4ae0004ab0 con 0x55c8eec55300 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae001e0d0 con 0x55c8eec55300 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4ad17fa640 1 -- 172.21.15.5:0/1025122719 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4ae0043050 con 0x55c8eec55300 2024-05-14T15:36:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4af6983740 10 client.27841 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8ef08e400 con 0x55c8eec55300 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.947+0000 7f4ad17fa640 1 -- 172.21.15.5:0/1025122719 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae001e0d0 con 0x55c8eec55300 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4ad17fa640 1 -- 172.21.15.5:0/1025122719 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae0005200 con 0x55c8eec55300 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4af6983740 10 client.27841 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4af6983740 10 client.27841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4af6983740 20 client.27841 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4af6983740 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec611b0 0x55c8ef090ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4af6983740 4 client.27841 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8eef6f050 con 0x55c8eec611b0 2024-05-14T15:36:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4aed874640 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec611b0 0x55c8ef090ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4ad17fa640 10 client.27841 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:44.948+0000 7f4aed874640 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec611b0 0x55c8ef090ad0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebfeff83b47f8f0 server_cookie=4a12bb40ff3db65e in_seq=0 out_seq=0 2024-05-14T15:36:45.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1094053 9.16 MB/sec execute 486 sec latency 93.287 ms 2024-05-14T15:36:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.324+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/772614351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0108d40 msgr2=0x7fd1c0109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.324+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/772614351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0108d40 0x7fd1c0109150 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800b0d0 tx=0x7fd1b800b3e0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.328+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/772614351 shutdown_connections 2024-05-14T15:36:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.328+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/772614351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0109690 0x7fd1c0110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.328+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/772614351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0104410 0x7fd1c0104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.328+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/772614351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0108d40 0x7fd1c0109150 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.328+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/772614351 >> 172.21.15.5:0/772614351 conn(0x7fd1c0079c20 msgr2=0x7fd1c007a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.332+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/772614351 shutdown_connections 2024-05-14T15:36:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.332+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/772614351 wait complete. 2024-05-14T15:36:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.334+0000 7fd1c4d9d700 1 Processor -- start 2024-05-14T15:36:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.334+0000 7fd1c4d9d700 1 -- start start 2024-05-14T15:36:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1c4d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0104410 0x7fd1c019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1c4d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0108d40 0x7fd1c019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1c4d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0109690 0x7fd1c0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0104410 0x7fd1c019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0104410 0x7fd1c019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:45040/0 (socket says 172.21.15.5:45040) 2024-05-14T15:36:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1bf7fe700 1 -- 172.21.15.5:0/233325929 learned_addr learned my addr 172.21.15.5:0/233325929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.335+0000 7fd1beffd700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0108d40 0x7fd1c019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.336+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0112200 con 0x7fd1c0109690 2024-05-14T15:36:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.336+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0112080 con 0x7fd1c0104410 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.336+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0112380 con 0x7fd1c0108d40 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.336+0000 7fd1bffff700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0109690 0x7fd1c0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.337+0000 7fd1beffd700 1 -- 172.21.15.5:0/233325929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0104410 msgr2=0x7fd1c019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.337+0000 7fd1beffd700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0104410 0x7fd1c019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.337+0000 7fd1beffd700 1 -- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0109690 msgr2=0x7fd1c0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.337+0000 7fd1beffd700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0109690 0x7fd1c0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.337+0000 7fd1beffd700 1 -- 172.21.15.5:0/233325929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c0198ff0 con 0x7fd1c0108d40 2024-05-14T15:36:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.338+0000 7fd1beffd700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0108d40 0x7fd1c019ec60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009fd0 tx=0x7fd1b8007a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.338+0000 7fd1bcff9700 1 -- 172.21.15.5:0/233325929 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8021070 con 0x7fd1c0108d40 2024-05-14T15:36:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.339+0000 7fd1bcff9700 1 -- 172.21.15.5:0/233325929 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b801db00 con 0x7fd1c0108d40 2024-05-14T15:36:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.339+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c0199280 con 0x7fd1c0108d40 2024-05-14T15:36:45.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.339+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c0199750 con 0x7fd1c0108d40 2024-05-14T15:36:45.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.339+0000 7fd1bcff9700 1 -- 172.21.15.5:0/233325929 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b801b9b0 con 0x7fd1c0108d40 2024-05-14T15:36:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.343+0000 7fd1bcff9700 1 -- 172.21.15.5:0/233325929 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fd1b8024070 con 0x7fd1c0108d40 2024-05-14T15:36:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1d4a13b80 1 Processor -- start 2024-05-14T15:36:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1bcff9700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd1a80b6eb0 0x7fd1a80b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1d4a13b80 1 -- start start 2024-05-14T15:36:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1bcff9700 1 -- 172.21.15.5:0/233325929 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fd1b800f020 con 0x7fd1c0108d40 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97d77ec0 0x55cf97ef5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1bffff700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd1a80b6eb0 0x7fd1a80b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97effb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.345+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 0x55cf97f02050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.346+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf97e179c0 con 0x55cf97ef68d0 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.346+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf97d7ce20 con 0x55cf97ef64c0 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.346+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf97e13980 con 0x55cf97d77ec0 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.346+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 0x55cf97f02050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.346+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 0x55cf97f02050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52710/0 (socket says 172.21.15.5:52710) 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.346+0000 7fd1bffff700 1 -- 172.21.15.5:0/2566658593 learned_addr learned my addr 172.21.15.5:0/2566658593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 -- 172.21.15.5:0/2566658593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97d77ec0 msgr2=0x55cf97ef5f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97d77ec0 0x55cf97ef5f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 -- 172.21.15.5:0/2566658593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 msgr2=0x55cf97effb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:36:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97effb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 -- 172.21.15.5:0/2566658593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf97d64950 con 0x55cf97ef68d0 2024-05-14T15:36:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 0x55cf97f02050 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0011a30 tx=0x7fd1b001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5403c69837b8ce5c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.347+0000 7fd1bffff700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd1a80b6eb0 0x7fd1a80b93c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00149c0 tx=0x7fd1b000bbf0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.348+0000 7fd1a5ffb700 1 -- 172.21.15.5:0/2566658593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0025070 con 0x55cf97ef68d0 2024-05-14T15:36:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.348+0000 7fd1a5ffb700 1 -- 172.21.15.5:0/2566658593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b0014e90 con 0x55cf97ef68d0 2024-05-14T15:36:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.348+0000 7fd1a5ffb700 1 -- 172.21.15.5:0/2566658593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0014330 con 0x55cf97ef68d0 2024-05-14T15:36:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2566658593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 msgr2=0x55cf97f02050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 0x55cf97f02050 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0011a30 tx=0x7fd1b001b000 comp rx=0 tx=0).stop 2024-05-14T15:36:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2566658593 shutdown_connections 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97ef68d0 0x55cf97f02050 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97d77ec0 0x55cf97ef5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2566658593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97effb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2566658593 >> 172.21.15.5:0/2566658593 conn(0x55cf97dbf6b0 msgr2=0x55cf97ef0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.350+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2566658593 shutdown_connections 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.351+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2566658593 wait complete. 2024-05-14T15:36:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.351+0000 7fd1d4a13b80 1 -- start start 2024-05-14T15:36:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 Processor -- start 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97eff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97effb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97eff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97eff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52718/0 (socket says 172.21.15.5:52718) 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1bf7fe700 1 -- 172.21.15.5:0/2107339994 learned_addr learned my addr 172.21.15.5:0/2107339994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf97f0b1f0 con 0x55cf97d77ec0 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf97dd0f50 con 0x55cf97ef68d0 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.352+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cf97f0b370 con 0x55cf97ef64c0 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1beffd700 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97effb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1d4a13b80 10 client.0 _ll_register_callbacks cb 0x55cf97daf9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1beffd700 1 -- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 msgr2=0x55cf97f00080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T15:36:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1beffd700 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f00080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1beffd700 1 -- 172.21.15.5:0/2107339994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 msgr2=0x55cf97eff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1beffd700 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97eff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.353+0000 7fd1beffd700 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf97dce7e0 con 0x55cf97ef64c0 2024-05-14T15:36:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.354+0000 7fd1beffd700 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97effb40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80e1230 tx=0x7fd1b80aa670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca3598bfe0c3e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.354+0000 7fd19effd700 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-05-14T15:36:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.354+0000 7fd19effd700 1 -- 172.21.15.5:0/2107339994 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80aae50 con 0x55cf97ef64c0 2024-05-14T15:36:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.354+0000 7fd19effd700 1 -- 172.21.15.5:0/2107339994 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fd1b80c5440 con 0x55cf97ef64c0 2024-05-14T15:36:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.354+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b0014e90 con 0x55cf97ef64c0 2024-05-14T15:36:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.354+0000 7fd19effd700 1 -- 172.21.15.5:0/2107339994 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1b80af200 con 0x55cf97ef64c0 2024-05-14T15:36:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 10 client.27847 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cf97f0b250 con 0x55cf97ef64c0 2024-05-14T15:36:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd19effd700 1 -- 172.21.15.5:0/2107339994 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b80ad030 con 0x55cf97ef64c0 2024-05-14T15:36:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd19effd700 1 -- 172.21.15.5:0/2107339994 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fd1b8011070 con 0x55cf97ef64c0 2024-05-14T15:36:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 10 client.27847 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 10 client.27847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 20 client.27847 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 4 client.27847 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.355+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd1b0014e90 con 0x55cf97f18580 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.356+0000 7fd1bf7fe700 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.356+0000 7fd19effd700 10 client.27847 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:45.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.356+0000 7fd1bf7fe700 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f1a9d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9afee0133a85c98 server_cookie=a17842af36e58667 in_seq=0 out_seq=0 2024-05-14T15:36:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.398+0000 7f4ad17fa640 1 -- 172.21.15.5:0/1025122719 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8eef6f050 con 0x55c8eec611b0 2024-05-14T15:36:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.398+0000 7f4ad17fa640 10 client.27841 handle_command_reply: tid=0 2024-05-14T15:36:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.398+0000 7f4af6983740 1 client.27841 shutdown 2024-05-14T15:36:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 msgr2=0x55c8ef0159d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 0x55c8ef0159d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009fa0 tx=0x7f4ae001e310 comp rx=0 tx=0).stop 2024-05-14T15:36:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 shutdown_connections 2024-05-14T15:36:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55c8eec611b0 0x55c8ef090ad0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c8eec55300 0x55c8ef0159d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4ad17fa640 0 client.27841 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8eec404d0 0x55c8ef018a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.399+0000 7f4af6983740 1 --2- 172.21.15.5:0/1025122719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8eec382f0 0x55c8ef0184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.400+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 >> 172.21.15.5:0/1025122719 conn(0x55c8ef0060a0 msgr2=0x55c8ef002100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.400+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 shutdown_connections 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.399+0000 7fd19effd700 1 -- 172.21.15.5:0/2107339994 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fd1b0014e90 con 0x55cf97f18580 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.399+0000 7fd19effd700 10 client.27847 handle_command_reply: tid=0 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.399+0000 7fd1d4a13b80 1 client.27847 shutdown 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.399+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 msgr2=0x55cf97effb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.399+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97effb40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80e1230 tx=0x7fd1b80aa670 comp rx=0 tx=0).stop 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 shutdown_connections 2024-05-14T15:36:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f1a9d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd19effd700 0 client.27847 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97eff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/2107339994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97effb40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 >> 172.21.15.5:0/2107339994 conn(0x55cf97dbf6b0 msgr2=0x55cf97ef4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 shutdown_connections 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/2107339994 wait complete. 2024-05-14T15:36:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.400+0000 7f4af6983740 1 -- 172.21.15.5:0/1025122719 wait complete. 2024-05-14T15:36:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.400+0000 7fd1d4a13b80 20 client.27847 trim_cache size 0 max 16384 2024-05-14T15:36:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.400+0000 7f4af6983740 20 client.27841 trim_cache size 0 max 16384 2024-05-14T15:36:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:36:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:36:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:36:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:36:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ad40b29d0 msgr2=0x7f4ad40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ad40b29d0 0x7f4ad40b4ee0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009fd0 tx=0x7f4ae0004a40 comp rx=0 tx=0).stop 2024-05-14T15:36:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 msgr2=0x7f4ae80ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae80ff8c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800e9f0 tx=0x7f4ad800eeb0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 shutdown_connections 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f4ad40b29d0 0x7f4ad40b4ee0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ae8109040 0x7f4ae80ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae8108ba0 0x7f4ae80ff8c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 --2- 172.21.15.5:0/3267296781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae81032f0 0x7f4ae80ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 >> 172.21.15.5:0/3267296781 conn(0x7f4ae80782e0 msgr2=0x7f4ae806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 shutdown_connections 2024-05-14T15:36:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:45.401+0000 7f4aefad8640 1 -- 172.21.15.5:0/3267296781 wait complete. 2024-05-14T15:36:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 Processor -- start 2024-05-14T15:36:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 -- start start 2024-05-14T15:36:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f01aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97f13d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f20150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac0116e0 con 0x55cf97d77ec0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac00ef70 con 0x55cf97ef68d0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.434+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac00cde0 con 0x55cf97ef64c0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f01aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97f13d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f01aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52734/0 (socket says 172.21.15.5:52734) 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1bf7fe700 1 -- 172.21.15.5:0/3976306791 learned_addr learned my addr 172.21.15.5:0/3976306791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1bffff700 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f20150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 -- 172.21.15.5:0/3976306791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 msgr2=0x55cf97f20150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f20150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 -- 172.21.15.5:0/3976306791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 msgr2=0x55cf97f01aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f01aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 -- 172.21.15.5:0/3976306791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf97ef56c0 con 0x55cf97ef64c0 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1bf7fe700 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f01aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.435+0000 7fd1beffd700 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97f13d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80a6ed0 tx=0x7fd1b80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c101dd3d3ec38119 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.436+0000 7fd1a57fa700 1 -- 172.21.15.5:0/3976306791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80aae50 con 0x55cf97ef64c0 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.436+0000 7fd1a57fa700 1 -- 172.21.15.5:0/3976306791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b80ad070 con 0x55cf97ef64c0 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.436+0000 7fd1a57fa700 1 -- 172.21.15.5:0/3976306791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80ab2c0 con 0x55cf97ef64c0 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/3976306791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 msgr2=0x55cf97f13d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97f13d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80a6ed0 tx=0x7fd1b80099e0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/3976306791 shutdown_connections 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef68d0 0x55cf97f20150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f01aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/3976306791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef64c0 0x55cf97f13d30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/3976306791 >> 172.21.15.5:0/3976306791 conn(0x55cf97dbf6b0 msgr2=0x55cf97ef4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/3976306791 shutdown_connections 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.437+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/3976306791 wait complete. 2024-05-14T15:36:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 -- start start 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 Processor -- start 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f10910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97f0eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef68d0 0x55cf97f0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac00d430 con 0x55cf97d77ec0 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.438+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac00e8a0 con 0x55cf97ef64c0 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1d4a13b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac00db00 con 0x55cf97ef68d0 2024-05-14T15:36:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f10910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1d4a13b80 10 client.0 _ll_register_callbacks cb 0x55cf97daf9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97f0eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef68d0 0x55cf97f0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97f0eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:45074/0 (socket says 172.21.15.5:45074) 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 -- 172.21.15.5:0/4016156271 learned_addr learned my addr 172.21.15.5:0/4016156271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 -- 172.21.15.5:0/4016156271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef68d0 msgr2=0x55cf97f0f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef68d0 0x55cf97f0f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 -- 172.21.15.5:0/4016156271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 msgr2=0x55cf97f10910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f10910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 -- 172.21.15.5:0/4016156271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf97f0f720 con 0x55cf97ef64c0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1bf7fe700 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f10910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1beffd700 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97f0eca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80e1830 tx=0x7fd1b801d890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1adcdec4ca04f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd19effd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd19effd700 1 -- 172.21.15.5:0/4016156271 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80aae50 con 0x55cf97ef64c0 2024-05-14T15:36:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1b80ad070 con 0x55cf97ef64c0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.439+0000 7fd19effd700 1 -- 172.21.15.5:0/4016156271 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fd1b80911b0 con 0x55cf97ef64c0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd19effd700 1 -- 172.21.15.5:0/4016156271 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1b80b3c20 con 0x55cf97ef64c0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 10 client.27875 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cf97f0f720 con 0x55cf97ef64c0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd19effd700 1 -- 172.21.15.5:0/4016156271 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b80ad070 con 0x55cf97ef64c0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd19effd700 1 -- 172.21.15.5:0/4016156271 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fd1b8011050 con 0x55cf97ef64c0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 10 client.27875 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 10 client.27875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 20 client.27875 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f85bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 4 client.27875 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cf97e253a0 con 0x55cf97f18580 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.440+0000 7fd1bf7fe700 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f85bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.441+0000 7fd19effd700 10 client.27875 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.441+0000 7fd1bf7fe700 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f85bb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef7b7f976c1a51a server_cookie=33b7b3bb74139f0c in_seq=0 out_seq=0 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd19effd700 1 -- 172.21.15.5:0/4016156271 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf97e253a0 con 0x55cf97f18580 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd19effd700 10 client.27875 handle_command_reply: tid=0 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 client.27875 shutdown 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 msgr2=0x55cf97f0eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97f0eca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80e1830 tx=0x7fd1b801d890 comp rx=0 tx=0).stop 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 shutdown_connections 2024-05-14T15:36:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55cf97f18580 0x55cf97f85bb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cf97ef68d0 0x55cf97f0f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf97d77ec0 0x55cf97f10910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd19effd700 0 client.27875 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 --2- 172.21.15.5:0/4016156271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf97ef64c0 0x55cf97f0eca0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.443+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 >> 172.21.15.5:0/4016156271 conn(0x55cf97dbf6b0 msgr2=0x55cf97ef4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 shutdown_connections 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1d4a13b80 1 -- 172.21.15.5:0/4016156271 wait complete. 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1d4a13b80 20 client.27875 trim_cache size 0 max 16384 2024-05-14T15:36:45.447 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:36:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-14T15:36:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d27813c4-32f3-47e0-a645-6b9033e47981", 2024-05-14T15:36:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-14T15:36:45.448 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd1a80b6eb0 msgr2=0x7fd1a80b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd1a80b6eb0 0x7fd1a80b93c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00149c0 tx=0x7fd1b000bbf0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0108d40 msgr2=0x7fd1c019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.444+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0108d40 0x7fd1c019ec60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009fd0 tx=0x7fd1b8007a50 comp rx=0 tx=0).stop 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 shutdown_connections 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd1a80b6eb0 0x7fd1a80b93c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0109690 0x7fd1c0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1c0104410 0x7fd1c019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 --2- 172.21.15.5:0/233325929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1c0108d40 0x7fd1c019ec60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 >> 172.21.15.5:0/233325929 conn(0x7fd1c0079c20 msgr2=0x7fd1c0101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 shutdown_connections 2024-05-14T15:36:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:45.445+0000 7fd1c4d9d700 1 -- 172.21.15.5:0/233325929 wait complete. 2024-05-14T15:36:46.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:45 smithi005 ceph-mon[31559]: pgmap v376: 129 pgs: 129 active+clean; 75 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-14T15:36:46.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:45 smithi005 ceph-mon[31559]: scrub queued for path: / 2024-05-14T15:36:46.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:45 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:46.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:45 smithi005 ceph-mon[31559]: scrub summary: active paths [/] 2024-05-14T15:36:46.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:45 smithi005 ceph-mon[31559]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:46.160 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1094732 9.14 MB/sec execute 487 sec latency 92.707 ms 2024-05-14T15:36:46.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:36:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:45 smithi144 ceph-mon[35650]: pgmap v376: 129 pgs: 129 active+clean; 75 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-14T15:36:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:45 smithi144 ceph-mon[35650]: scrub queued for path: / 2024-05-14T15:36:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:45 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:45 smithi144 ceph-mon[35650]: scrub summary: active paths [/] 2024-05-14T15:36:46.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:45 smithi144 ceph-mon[35650]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:45 smithi193 ceph-mon[36001]: pgmap v376: 129 pgs: 129 active+clean; 75 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-05-14T15:36:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:45 smithi193 ceph-mon[36001]: scrub queued for path: / 2024-05-14T15:36:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:45 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:45 smithi193 ceph-mon[36001]: scrub summary: active paths [/] 2024-05-14T15:36:46.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:45 smithi193 ceph-mon[36001]: scrub summary: idle+waiting paths [/] 2024-05-14T15:36:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:36:47.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1096532 9.14 MB/sec execute 488 sec latency 23.046 ms 2024-05-14T15:36:47.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:47 smithi005 ceph-mon[31559]: pgmap v377: 129 pgs: 129 active+clean; 80 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-05-14T15:36:48.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1097994 9.14 MB/sec execute 489 sec latency 19.505 ms 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 -- 172.21.15.5:0/2533585758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 msgr2=0x7f47641111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 --2- 172.21.15.5:0/2533585758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 0x7f47641111c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f475c00b0d0 tx=0x7f475c00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 -- 172.21.15.5:0/2533585758 shutdown_connections 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 --2- 172.21.15.5:0/2533585758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 0x7f47641111c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 --2- 172.21.15.5:0/2533585758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47641045e0 0x7f47641049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 --2- 172.21.15.5:0/2533585758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4764108f10 0x7f4764109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.250+0000 7f47697b6700 1 -- 172.21.15.5:0/2533585758 >> 172.21.15.5:0/2533585758 conn(0x7f47640ffe70 msgr2=0x7f47641022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.251+0000 7f47697b6700 1 -- 172.21.15.5:0/2533585758 shutdown_connections 2024-05-14T15:36:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.251+0000 7f47697b6700 1 -- 172.21.15.5:0/2533585758 wait complete. 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.251+0000 7f47697b6700 1 Processor -- start 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 -- start start 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 0x7f476419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 0x7f476419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 0x7f476419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4764114490 con 0x7f4764109860 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4764114310 con 0x7f47641045e0 2024-05-14T15:36:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47697b6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4764114610 con 0x7f4764108f10 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 0x7f476419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 0x7f476419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 0x7f476419ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48310/0 (socket says 172.21.15.5:48310) 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 0x7f476419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:34924/0 (socket says 172.21.15.5:34924) 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 -- 172.21.15.5:0/3014772782 learned_addr learned my addr 172.21.15.5:0/3014772782 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f47637fe700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 0x7f476419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 -- 172.21.15.5:0/3014772782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 msgr2=0x7f476419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 0x7f476419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 -- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 msgr2=0x7f476419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.252+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 0x7f476419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f4762ffd700 1 -- 172.21.15.5:0/3014772782 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476419fc50 con 0x7f47641045e0 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 0x7f476419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 0x7f476419ad80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f475400d950 tx=0x7f475400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f4760ff9700 1 -- 172.21.15.5:0/3014772782 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754016070 con 0x7f47641045e0 2024-05-14T15:36:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f4760ff9700 1 -- 172.21.15.5:0/3014772782 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475400fd40 con 0x7f47641045e0 2024-05-14T15:36:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476419ff40 con 0x7f47641045e0 2024-05-14T15:36:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f4760ff9700 1 -- 172.21.15.5:0/3014772782 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475400e3b0 con 0x7f47641045e0 2024-05-14T15:36:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.253+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641a7860 con 0x7f47641045e0 2024-05-14T15:36:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.256+0000 7f4760ff9700 1 -- 172.21.15.5:0/3014772782 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4754019070 con 0x7f47641045e0 2024-05-14T15:36:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f4760ff9700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f47500b29f0 0x7f47500b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 Processor -- start 2024-05-14T15:36:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f47500b29f0 0x7f47500b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 -- start start 2024-05-14T15:36:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f4760ff9700 1 -- 172.21.15.5:0/3014772782 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f47540e2580 con 0x7f47641045e0 2024-05-14T15:36:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f47500b29f0 0x7f47500b4f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47641037c0 tx=0x7f474c0073d0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ecc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ec68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ec7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f858de8940 con 0x55f858d48e40 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f858d4dda0 con 0x55f858ec6e10 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.257+0000 7f47781cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f858de4900 con 0x55f858eccd80 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ecc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ecc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58300/0 (socket says 172.21.15.5:58300) 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 -- 172.21.15.5:0/4007115887 learned_addr learned my addr 172.21.15.5:0/4007115887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f47637fe700 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ec7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f475a7fc700 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ec68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 -- 172.21.15.5:0/4007115887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 msgr2=0x55f858ec68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ec68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 -- 172.21.15.5:0/4007115887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 msgr2=0x55f858ec7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ec7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 -- 172.21.15.5:0/4007115887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f858d358d0 con 0x55f858d48e40 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f475a7fc700 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ec68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f4762ffd700 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ecc840 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f47540a5470 tx=0x7f47540a8e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501c0eac839fdc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f47597fa700 1 -- 172.21.15.5:0/4007115887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754020070 con 0x55f858d48e40 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f47597fa700 1 -- 172.21.15.5:0/4007115887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47540ad7a0 con 0x55f858d48e40 2024-05-14T15:36:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.258+0000 7f47597fa700 1 -- 172.21.15.5:0/4007115887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540a9870 con 0x55f858d48e40 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 -- 172.21.15.5:0/4007115887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 msgr2=0x55f858ecc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ecc840 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f47540a5470 tx=0x7f47540a8e70 comp rx=0 tx=0).stop 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 -- 172.21.15.5:0/4007115887 shutdown_connections 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ec7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ecc840 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 --2- 172.21.15.5:0/4007115887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ec68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 -- 172.21.15.5:0/4007115887 >> 172.21.15.5:0/4007115887 conn(0x55f858d90630 msgr2=0x55f858ec1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 -- 172.21.15.5:0/4007115887 shutdown_connections 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.259+0000 7f47781cab80 1 -- 172.21.15.5:0/4007115887 wait complete. 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 -- start start 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 Processor -- start 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ed04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ed0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ed0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f858ed37c0 con 0x55f858d48e40 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f858da1ed0 con 0x55f858ec6e10 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f858ed3940 con 0x55f858eccd80 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f47781cab80 10 client.0 _ll_register_callbacks cb 0x55f858d80950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ed04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ed0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ed04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58314/0 (socket says 172.21.15.5:58314) 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.260+0000 7f4762ffd700 1 -- 172.21.15.5:0/3870211130 learned_addr learned my addr 172.21.15.5:0/3870211130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f47637fe700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ed0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f4762ffd700 1 -- 172.21.15.5:0/3870211130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 msgr2=0x55f858ed0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ed0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f4762ffd700 1 -- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 msgr2=0x55f858ed0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ed0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f4762ffd700 1 -- 172.21.15.5:0/3870211130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f858d9f760 con 0x55f858d48e40 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ed0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ed04c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f47540dfde0 tx=0x7f4754010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd565f286db43f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f474a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f474a7fc700 1 -- 172.21.15.5:0/3870211130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754020070 con 0x55f858d48e40 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f474a7fc700 1 -- 172.21.15.5:0/3870211130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f47540a9650 con 0x55f858d48e40 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47540ad7a0 con 0x55f858d48e40 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f474a7fc700 1 -- 172.21.15.5:0/3870211130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f47540b2250 con 0x55f858d48e40 2024-05-14T15:36:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f47781cab80 10 client.18312 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.261+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f858ede3f0 con 0x55f858d48e40 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f474a7fc700 1 -- 172.21.15.5:0/3870211130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47540ad7a0 con 0x55f858d48e40 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f474a7fc700 1 -- 172.21.15.5:0/3870211130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f47540c3ce0 con 0x55f858d48e40 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f47781cab80 10 client.18312 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f47781cab80 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f47781cab80 20 client.18312 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f47781cab80 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858ee08c0 0x55f858ee2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f47781cab80 4 client.18312 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f858df6320 con 0x55f858ee08c0 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858ee08c0 0x55f858ee2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f474a7fc700 10 client.18312 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.262+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858ee08c0 0x55f858ee2ce0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d509d2719f1e2 server_cookie=964d7e0b19e875e5 in_seq=0 out_seq=0 2024-05-14T15:36:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f474a7fc700 1 -- 172.21.15.5:0/3870211130 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f858df6320 con 0x55f858ee08c0 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f474a7fc700 10 client.18312 handle_command_reply: tid=0 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 client.18312 shutdown 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 msgr2=0x55f858ed04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ed04c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f47540dfde0 tx=0x7f4754010000 comp rx=0 tx=0).stop 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 shutdown_connections 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858ee08c0 0x55f858ee2ce0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ed0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858d48e40 0x55f858ed04c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 --2- 172.21.15.5:0/3870211130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858ec6e10 0x55f858ed0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f474a7fc700 0 client.18312 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 >> 172.21.15.5:0/3870211130 conn(0x55f858d90630 msgr2=0x55f858ec58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 shutdown_connections 2024-05-14T15:36:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 1 -- 172.21.15.5:0/3870211130 wait complete. 2024-05-14T15:36:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.264+0000 7f47781cab80 20 client.18312 trim_cache size 0 max 16384 2024-05-14T15:36:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 Processor -- start 2024-05-14T15:36:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 -- start start 2024-05-14T15:36:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ec68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ecbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858edcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b2f60 con 0x55f858ec6e10 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b07f0 con 0x55f858d48e40 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f47781cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47641ae660 con 0x55f858eccd80 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ecbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ecbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58318/0 (socket says 172.21.15.5:58318) 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f475a7fc700 1 -- 172.21.15.5:0/1100314354 learned_addr learned my addr 172.21.15.5:0/1100314354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.285+0000 7f4762ffd700 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ec68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f47637fe700 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858edcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f475a7fc700 1 -- 172.21.15.5:0/1100314354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 msgr2=0x55f858edcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f475a7fc700 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858edcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f475a7fc700 1 -- 172.21.15.5:0/1100314354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 msgr2=0x55f858ec68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f475a7fc700 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ec68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f475a7fc700 1 -- 172.21.15.5:0/1100314354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f858ecf480 con 0x55f858ec6e10 2024-05-14T15:36:48.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f475a7fc700 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ecbd70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f474c005fd0 tx=0x7f474c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd45a205580d7c05 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f4758ff9700 1 -- 172.21.15.5:0/1100314354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c02e070 con 0x55f858ec6e10 2024-05-14T15:36:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f4758ff9700 1 -- 172.21.15.5:0/1100314354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c015070 con 0x55f858ec6e10 2024-05-14T15:36:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.286+0000 7f4758ff9700 1 -- 172.21.15.5:0/1100314354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c0118a0 con 0x55f858ec6e10 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 -- 172.21.15.5:0/1100314354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 msgr2=0x55f858ecbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ecbd70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f474c005fd0 tx=0x7f474c016000 comp rx=0 tx=0).stop 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 -- 172.21.15.5:0/1100314354 shutdown_connections 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858edcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ec68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 --2- 172.21.15.5:0/1100314354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ecbd70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 -- 172.21.15.5:0/1100314354 >> 172.21.15.5:0/1100314354 conn(0x55f858d90630 msgr2=0x55f858ed1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 -- 172.21.15.5:0/1100314354 shutdown_connections 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.287+0000 7f47781cab80 1 -- 172.21.15.5:0/1100314354 wait complete. 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 -- start start 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 Processor -- start 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ee5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ee3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ee3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47641aecb0 con 0x55f858ec6e10 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47641b0120 con 0x55f858d48e40 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47641af380 con 0x55f858eccd80 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47781cab80 10 client.0 _ll_register_callbacks cb 0x55f858d80950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ee3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ee5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f475a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ee3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58330/0 (socket says 172.21.15.5:58330) 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ee5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:48348/0 (socket says 172.21.15.5:48348) 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f475a7fc700 1 -- 172.21.15.5:0/3561239370 learned_addr learned my addr 172.21.15.5:0/3561239370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.288+0000 7f47637fe700 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ee3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f475a7fc700 1 -- 172.21.15.5:0/3561239370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 msgr2=0x55f858ee3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ee3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f475a7fc700 1 -- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 msgr2=0x55f858ee5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ee5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f475a7fc700 1 -- 172.21.15.5:0/3561239370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f858ee3dc0 con 0x55f858ec6e10 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f475a7fc700 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ee3340 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f474c003f10 tx=0x7f474c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90252342ac3861 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f474a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f474a7fc700 1 -- 172.21.15.5:0/3561239370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c037070 con 0x55f858ec6e10 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f474a7fc700 1 -- 172.21.15.5:0/3561239370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f474c029540 con 0x55f858ec6e10 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474c015070 con 0x55f858ec6e10 2024-05-14T15:36:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f474a7fc700 1 -- 172.21.15.5:0/3561239370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f474c02e040 con 0x55f858ec6e10 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f47781cab80 10 client.18318 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f858f4aa20 con 0x55f858ec6e10 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.289+0000 7f474a7fc700 1 -- 172.21.15.5:0/3561239370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c015070 con 0x55f858ec6e10 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f474a7fc700 1 -- 172.21.15.5:0/3561239370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f474c029ce0 con 0x55f858ec6e10 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f47781cab80 10 client.18318 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f47781cab80 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f47781cab80 20 client.18318 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f47781cab80 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858f4b070 0x55f858f4d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f47781cab80 4 client.18318 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f858df03c0 con 0x55f858f4b070 2024-05-14T15:36:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.290+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858f4b070 0x55f858f4d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:48.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.291+0000 7f4762ffd700 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858f4b070 0x55f858f4d490 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66322480332dbe12 server_cookie=f650e490185b906b in_seq=0 out_seq=0 2024-05-14T15:36:48.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.291+0000 7f474a7fc700 10 client.18318 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f474a7fc700 1 -- 172.21.15.5:0/3561239370 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f858df03c0 con 0x55f858f4b070 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f474a7fc700 10 client.18318 handle_command_reply: tid=0 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 client.18318 shutdown 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 msgr2=0x55f858ee3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ee3340 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f474c003f10 tx=0x7f474c003f40 comp rx=0 tx=0).stop 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 shutdown_connections 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55f858f4b070 0x55f858f4d490 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f858eccd80 0x55f858ee3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f858d48e40 0x55f858ee5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 --2- 172.21.15.5:0/3561239370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f858ec6e10 0x55f858ee3340 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f474a7fc700 0 client.18318 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 >> 172.21.15.5:0/3561239370 conn(0x55f858d90630 msgr2=0x55f858ec1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:48.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 shutdown_connections 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 1 -- 172.21.15.5:0/3561239370 wait complete. 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.292+0000 7f47781cab80 20 client.18318 trim_cache size 0 max 16384 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: "d27813c4-32f3-47e0-a645-6b9033e47981": { 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d27813c4-32f3-47e0-a645-6b9033e47981", 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-05-14T15:36:48.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:36:48.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:36:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:36:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f47500b29f0 msgr2=0x7f47500b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f47500b29f0 0x7f47500b4f00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47641037c0 tx=0x7f474c0073d0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 msgr2=0x7f476419ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 0x7f476419ad80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f475400d950 tx=0x7f475400dc60 comp rx=0 tx=0).stop 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 shutdown_connections 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f47500b29f0 0x7f47500b4f00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764109860 0x7f476419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47641045e0 0x7f476419ad80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 --2- 172.21.15.5:0/3014772782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4764108f10 0x7f476419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 >> 172.21.15.5:0/3014772782 conn(0x7f47640ffe70 msgr2=0x7f4764108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 shutdown_connections 2024-05-14T15:36:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:36:48.293+0000 7f47697b6700 1 -- 172.21.15.5:0/3014772782 wait complete. 2024-05-14T15:36:48.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:47 smithi144 ceph-mon[35650]: pgmap v377: 129 pgs: 129 active+clean; 80 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-05-14T15:36:48.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:47 smithi193 ceph-mon[36001]: pgmap v377: 129 pgs: 129 active+clean; 80 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-05-14T15:36:49.095 INFO:tasks.cephfs.filesystem:scrub status for tag:d27813c4-32f3-47e0-a645-6b9033e47981 - {'path': '/', 'tag': 'd27813c4-32f3-47e0-a645-6b9033e47981', 'options': 'recursive,force'} 2024-05-14T15:36:49.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1099936 9.14 MB/sec execute 490 sec latency 16.526 ms 2024-05-14T15:36:50.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1101416 9.15 MB/sec execute 491 sec latency 22.763 ms 2024-05-14T15:36:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:49 smithi144 ceph-mon[35650]: pgmap v378: 129 pgs: 129 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-05-14T15:36:50.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:49 smithi193 ceph-mon[36001]: pgmap v378: 129 pgs: 129 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-05-14T15:36:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:49 smithi005 ceph-mon[31559]: pgmap v378: 129 pgs: 129 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-05-14T15:36:51.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1102812 9.14 MB/sec execute 492 sec latency 42.877 ms 2024-05-14T15:36:51.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:50 smithi144 ceph-mon[35650]: scrub summary: idle 2024-05-14T15:36:51.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:50 smithi193 ceph-mon[36001]: scrub summary: idle 2024-05-14T15:36:51.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:50 smithi005 ceph-mon[31559]: scrub summary: idle 2024-05-14T15:36:52.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1104752 9.14 MB/sec execute 493 sec latency 115.661 ms 2024-05-14T15:36:52.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:51 smithi144 ceph-mon[35650]: pgmap v379: 129 pgs: 129 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-05-14T15:36:52.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:51 smithi193 ceph-mon[36001]: pgmap v379: 129 pgs: 129 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-05-14T15:36:52.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:51 smithi005 ceph-mon[31559]: pgmap v379: 129 pgs: 129 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-05-14T15:36:53.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1105506 9.13 MB/sec execute 494 sec latency 50.706 ms 2024-05-14T15:36:54.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1108042 9.14 MB/sec execute 495 sec latency 14.108 ms 2024-05-14T15:36:54.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:53 smithi144 ceph-mon[35650]: pgmap v380: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-05-14T15:36:54.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:53 smithi193 ceph-mon[36001]: pgmap v380: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-05-14T15:36:54.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:53 smithi005 ceph-mon[31559]: pgmap v380: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 306 op/s 2024-05-14T15:36:55.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1108972 9.13 MB/sec execute 496 sec latency 27.352 ms 2024-05-14T15:36:56.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1111135 9.14 MB/sec execute 497 sec latency 17.653 ms 2024-05-14T15:36:56.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:55 smithi005 ceph-mon[31559]: pgmap v381: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-05-14T15:36:56.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:55 smithi144 ceph-mon[35650]: pgmap v381: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-05-14T15:36:56.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:55 smithi193 ceph-mon[36001]: pgmap v381: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-05-14T15:36:57.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1112157 9.13 MB/sec execute 498 sec latency 65.833 ms 2024-05-14T15:36:58.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1113452 9.12 MB/sec execute 499 sec latency 26.035 ms 2024-05-14T15:36:58.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:57 smithi144 ceph-mon[35650]: pgmap v382: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-05-14T15:36:58.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:57 smithi193 ceph-mon[36001]: pgmap v382: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-05-14T15:36:58.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:57 smithi005 ceph-mon[31559]: pgmap v382: 129 pgs: 129 active+clean; 79 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-05-14T15:36:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 -- 172.21.15.5:0/1726267461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824109030 msgr2=0x7fc824102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 --2- 172.21.15.5:0/1726267461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824109030 0x7fc824102c70 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802a410 comp rx=0 tx=0).stop 2024-05-14T15:36:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 -- 172.21.15.5:0/1726267461 shutdown_connections 2024-05-14T15:36:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 --2- 172.21.15.5:0/1726267461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 0x7fc82410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 --2- 172.21.15.5:0/1726267461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824109030 0x7fc824102c70 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 --2- 172.21.15.5:0/1726267461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824108b90 0x7fc824102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 -- 172.21.15.5:0/1726267461 >> 172.21.15.5:0/1726267461 conn(0x7fc8240fe390 msgr2=0x7fc8241007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 -- 172.21.15.5:0/1726267461 shutdown_connections 2024-05-14T15:36:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 -- 172.21.15.5:0/1726267461 wait complete. 2024-05-14T15:36:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.519+0000 7fc829205640 1 Processor -- start 2024-05-14T15:36:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 -- start start 2024-05-14T15:36:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 0x7fc82419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 0x7fc82419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824109030 0x7fc82419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc824112d80 con 0x7fc824108b90 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc824112c00 con 0x7fc8241032e0 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc829205640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc824112f00 con 0x7fc824109030 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc82259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 0x7fc82419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc82259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 0x7fc82419bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51578/0 (socket says 172.21.15.5:51578) 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc82259c640 1 -- 172.21.15.5:0/1090178099 learned_addr learned my addr 172.21.15.5:0/1090178099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc822d9d640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 0x7fc82419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.520+0000 7fc82359e640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824109030 0x7fc82419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82259c640 1 -- 172.21.15.5:0/1090178099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824109030 msgr2=0x7fc82419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82259c640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824109030 0x7fc82419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82259c640 1 -- 172.21.15.5:0/1090178099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 msgr2=0x7fc82419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82259c640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 0x7fc82419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82259c640 1 -- 172.21.15.5:0/1090178099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8241a6a70 con 0x7fc824108b90 2024-05-14T15:36:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82359e640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824109030 0x7fc82419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc822d9d640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 0x7fc82419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc82259c640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 0x7fc82419bca0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc81802a930 tx=0x7fc818031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc80bfff640 1 -- 172.21.15.5:0/1090178099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81803f030 con 0x7fc824108b90 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8241a6d00 con 0x7fc824108b90 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc80bfff640 1 -- 172.21.15.5:0/1090178099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818004490 con 0x7fc824108b90 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc80bfff640 1 -- 172.21.15.5:0/1090178099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818035070 con 0x7fc824108b90 2024-05-14T15:36:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.521+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8241a71e0 con 0x7fc824108b90 2024-05-14T15:36:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.524+0000 7fc80bfff640 1 -- 172.21.15.5:0/1090178099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc818042030 con 0x7fc824108b90 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.524+0000 7fc830191740 1 Processor -- start 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 -- start start 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e04d1b0 0x561a0e3edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e3ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 0x561a0e3ee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a0e3535d0 con 0x561a0e02c4d0 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a0e30c890 con 0x561a0e04d1b0 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc830191740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561a0e34f590 con 0x561a0e0242f0 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc82259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 0x561a0e3ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc82259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 0x561a0e3ee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51586/0 (socket says 172.21.15.5:51586) 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc82259c640 1 -- 172.21.15.5:0/3085194542 learned_addr learned my addr 172.21.15.5:0/3085194542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc82359e640 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e04d1b0 0x561a0e3edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e3ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc80bfff640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc80c0b2950 0x7fc80c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc82359e640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc80c0b2950 0x7fc80c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc822d9d640 1 -- 172.21.15.5:0/3085194542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e04d1b0 msgr2=0x561a0e3edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e04d1b0 0x561a0e3edbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc822d9d640 1 -- 172.21.15.5:0/3085194542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 msgr2=0x561a0e3ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 0x561a0e3ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc822d9d640 1 -- 172.21.15.5:0/3085194542 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0e065210 con 0x561a0e0242f0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.525+0000 7fc82259c640 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 0x561a0e3ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.526+0000 7fc80bfff640 1 -- 172.21.15.5:0/1090178099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fc818102dc0 con 0x7fc824108b90 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.526+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e3ee120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc814009f20 tx=0x7fc8140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2985136a19678887 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.526+0000 7fc82359e640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc80c0b2950 0x7fc80c0b4e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x561a0e3ead00 tx=0x7fc81c004ab0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.526+0000 7fc8097fa640 1 -- 172.21.15.5:0/3085194542 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814004030 con 0x561a0e0242f0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.526+0000 7fc8097fa640 1 -- 172.21.15.5:0/3085194542 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc814005020 con 0x561a0e0242f0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.526+0000 7fc8097fa640 1 -- 172.21.15.5:0/3085194542 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814004ce0 con 0x561a0e0242f0 2024-05-14T15:36:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 -- 172.21.15.5:0/3085194542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 msgr2=0x561a0e3ee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e3ee120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc814009f20 tx=0x7fc8140118d0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 -- 172.21.15.5:0/3085194542 shutdown_connections 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e3ee120 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e02c4d0 0x561a0e3ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 --2- 172.21.15.5:0/3085194542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e04d1b0 0x561a0e3edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 -- 172.21.15.5:0/3085194542 >> 172.21.15.5:0/3085194542 conn(0x561a0e041300 msgr2=0x561a0e3ea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 -- 172.21.15.5:0/3085194542 shutdown_connections 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 -- 172.21.15.5:0/3085194542 wait complete. 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.527+0000 7fc830191740 1 -- start start 2024-05-14T15:36:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 Processor -- start 2024-05-14T15:36:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 0x561a0e3f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3f8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e04d1b0 0x561a0e404220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a0e3fad10 con 0x561a0e0242f0 2024-05-14T15:36:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a0e310370 con 0x561a0e02c4d0 2024-05-14T15:36:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561a0e3fae90 con 0x561a0e04d1b0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc830191740 10 client.0 _ll_register_callbacks cb 0x561a0e064a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 0x561a0e3f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 0x561a0e3f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51598/0 (socket says 172.21.15.5:51598) 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 -- 172.21.15.5:0/4198133417 learned_addr learned my addr 172.21.15.5:0/4198133417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc82259c640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc82359e640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e04d1b0 0x561a0e404220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 -- 172.21.15.5:0/4198133417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e04d1b0 msgr2=0x561a0e404220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e04d1b0 0x561a0e404220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 -- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 msgr2=0x561a0e3f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.528+0000 7fc822d9d640 1 -- 172.21.15.5:0/4198133417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0e131620 con 0x561a0e0242f0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc822d9d640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 0x561a0e3f81f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc814000c00 tx=0x7fc8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4199f04abcb1152 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc806ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc806ff5640 1 -- 172.21.15.5:0/4198133417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814028020 con 0x561a0e0242f0 2024-05-14T15:36:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc806ff5640 1 -- 172.21.15.5:0/4198133417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fc814016040 con 0x561a0e0242f0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc814005020 con 0x561a0e0242f0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc806ff5640 1 -- 172.21.15.5:0/4198133417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc814033050 con 0x561a0e0242f0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc830191740 10 client.18330 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a0e4070e0 con 0x561a0e0242f0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc806ff5640 1 -- 172.21.15.5:0/4198133417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc814005020 con 0x561a0e0242f0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc806ff5640 1 -- 172.21.15.5:0/4198133417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc8140037f0 con 0x561a0e0242f0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc830191740 10 client.18330 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc830191740 10 client.18330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.529+0000 7fc830191740 20 client.18330 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.530+0000 7fc830191740 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0e00bd80 0x561a0e40a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.530+0000 7fc830191740 4 client.18330 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.530+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0e3611e0 con 0x561a0e00bd80 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.530+0000 7fc82259c640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0e00bd80 0x561a0e40a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.530+0000 7fc806ff5640 10 client.18330 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.530+0000 7fc82259c640 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0e00bd80 0x561a0e40a550 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3ef667164db54a server_cookie=1522d46f6ec6ce10 in_seq=0 out_seq=0 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc806ff5640 1 -- 172.21.15.5:0/4198133417 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a0e3611e0 con 0x561a0e00bd80 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc806ff5640 10 client.18330 handle_command_reply: tid=0 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 client.18330 shutdown 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 msgr2=0x561a0e3f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 0x561a0e3f81f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc814000c00 tx=0x7fc8140046c0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 shutdown_connections 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0e00bd80 0x561a0e40a550 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e04d1b0 0x561a0e404220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 --2- 172.21.15.5:0/4198133417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e0242f0 0x561a0e3f81f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 >> 172.21.15.5:0/4198133417 conn(0x561a0e041300 msgr2=0x561a0e3eda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc806ff5640 0 client.18330 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 shutdown_connections 2024-05-14T15:36:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 1 -- 172.21.15.5:0/4198133417 wait complete. 2024-05-14T15:36:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.532+0000 7fc830191740 20 client.18330 trim_cache size 0 max 16384 2024-05-14T15:36:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.551+0000 7fc830191740 1 Processor -- start 2024-05-14T15:36:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 -- start start 2024-05-14T15:36:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e40fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e410170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241b3d80 con 0x561a0e04d1b0 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241b1650 con 0x561a0e02c4d0 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc830191740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241af4c0 con 0x561a0e0242f0 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3e9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51600/0 (socket says 172.21.15.5:51600) 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 -- 172.21.15.5:0/2434021429 learned_addr learned my addr 172.21.15.5:0/2434021429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82259c640 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e410170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc822d9d640 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e40fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 -- 172.21.15.5:0/2434021429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 msgr2=0x561a0e40fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e40fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 -- 172.21.15.5:0/2434021429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 msgr2=0x561a0e410170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e410170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82359e640 1 -- 172.21.15.5:0/2434021429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0e00bfa0 con 0x561a0e04d1b0 2024-05-14T15:36:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.552+0000 7fc82259c640 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e410170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.553+0000 7fc82359e640 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3e9980 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009b90 tx=0x7fc81c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ec46249e32425f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.553+0000 7fc808ff9640 1 -- 172.21.15.5:0/2434021429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c004d20 con 0x561a0e04d1b0 2024-05-14T15:36:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.553+0000 7fc808ff9640 1 -- 172.21.15.5:0/2434021429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c025070 con 0x561a0e04d1b0 2024-05-14T15:36:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.553+0000 7fc808ff9640 1 -- 172.21.15.5:0/2434021429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c005660 con 0x561a0e04d1b0 2024-05-14T15:36:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 -- 172.21.15.5:0/2434021429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 msgr2=0x561a0e3e9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3e9980 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009b90 tx=0x7fc81c009b20 comp rx=0 tx=0).stop 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 -- 172.21.15.5:0/2434021429 shutdown_connections 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3e9980 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e410170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 --2- 172.21.15.5:0/2434021429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e40fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 -- 172.21.15.5:0/2434021429 >> 172.21.15.5:0/2434021429 conn(0x561a0e041300 msgr2=0x561a0e3fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 -- 172.21.15.5:0/2434021429 shutdown_connections 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 -- 172.21.15.5:0/2434021429 wait complete. 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 -- start start 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.554+0000 7fc830191740 1 Processor -- start 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e400840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3fb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241afb10 con 0x561a0e04d1b0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241b0f80 con 0x561a0e02c4d0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8241b01e0 con 0x561a0e0242f0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc830191740 10 client.0 _ll_register_callbacks cb 0x7fc8181067e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3fb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51608/0 (socket says 172.21.15.5:51608) 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82359e640 1 -- 172.21.15.5:0/3769006374 learned_addr learned my addr 172.21.15.5:0/3769006374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e400840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 -- 172.21.15.5:0/3769006374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 msgr2=0x561a0e400840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e400840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 -- 172.21.15.5:0/3769006374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 msgr2=0x561a0e3fb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3fb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 -- 172.21.15.5:0/3769006374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0e3fbe20 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82359e640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc82259c640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3fb140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc818100700 tx=0x7fc8180ec040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e3e597485e5261 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc806ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.555+0000 7fc806ff5640 1 -- 172.21.15.5:0/3769006374 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818049040 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc806ff5640 1 -- 172.21.15.5:0/3769006374 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fc8180c5f20 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc81c025070 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc806ff5640 1 -- 172.21.15.5:0/3769006374 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8180c9400 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 10 client.27911 fetch_fsmap learned FSMap version 34 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a0e471790 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc806ff5640 1 -- 172.21.15.5:0/3769006374 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c025070 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc806ff5640 1 -- 172.21.15.5:0/3769006374 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc8180d2660 con 0x561a0e02c4d0 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 10 client.27911 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:36:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 10 client.27911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 20 client.27911 populate_metadata read hostname 'smithi005' 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0dfe11c0 0x561a0e473ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 4 client.27911 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a0e35b050 con 0x561a0dfe11c0 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.556+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0dfe11c0 0x561a0e473ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.557+0000 7fc806ff5640 10 client.27911 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.557+0000 7fc822d9d640 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0dfe11c0 0x561a0e473ed0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f366f3ac0a974da server_cookie=bd49c694f7f4756c in_seq=0 out_seq=0 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc806ff5640 1 -- 172.21.15.5:0/3769006374 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a0e35b050 con 0x561a0dfe11c0 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc806ff5640 10 client.27911 handle_command_reply: tid=0 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 client.27911 shutdown 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 msgr2=0x561a0e3fb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3fb140 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc818100700 tx=0x7fc8180ec040 comp rx=0 tx=0).stop 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 shutdown_connections 2024-05-14T15:36:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x561a0dfe11c0 0x561a0e473ed0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc806ff5640 0 client.27911 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0e04d1b0 0x561a0e3fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0e02c4d0 0x561a0e3fb140 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 --2- 172.21.15.5:0/3769006374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561a0e0242f0 0x561a0e400840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.921+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 >> 172.21.15.5:0/3769006374 conn(0x561a0e041300 msgr2=0x561a0e3f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 shutdown_connections 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc830191740 1 -- 172.21.15.5:0/3769006374 wait complete. 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc830191740 20 client.27911 trim_cache size 0 max 16384 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-05-14T15:36:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-05-14T15:36:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-05-14T15:36:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-05-14T15:36:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc80c0b2950 msgr2=0x7fc80c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc829205640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc80c0b2950 0x7fc80c0b4e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x561a0e3ead00 tx=0x7fc81c004ab0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 msgr2=0x7fc82419bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.922+0000 7fc829205640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 0x7fc82419bca0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc81802a930 tx=0x7fc818031a20 comp rx=0 tx=0).stop 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 shutdown_connections 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc80c0b2950 0x7fc80c0b4e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc824109030 0x7fc82419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc824108b90 0x7fc82419bca0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 --2- 172.21.15.5:0/1090178099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8241032e0 0x7fc82419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 >> 172.21.15.5:0/1090178099 conn(0x7fc8240fe390 msgr2=0x7fc82410e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 shutdown_connections 2024-05-14T15:36:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-05-14T15:36:58.923+0000 7fc829205640 1 -- 172.21.15.5:0/1090178099 wait complete. 2024-05-14T15:36:59.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1114714 9.12 MB/sec execute 500 sec latency 188.042 ms 2024-05-14T15:37:00.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1115909 9.12 MB/sec execute 501 sec latency 213.855 ms 2024-05-14T15:37:00.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:36:59 smithi144 ceph-mon[35650]: pgmap v383: 129 pgs: 129 active+clean; 82 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 259 op/s 2024-05-14T15:37:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:36:59 smithi193 ceph-mon[36001]: pgmap v383: 129 pgs: 129 active+clean; 82 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 259 op/s 2024-05-14T15:37:00.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:36:59 smithi005 ceph-mon[31559]: pgmap v383: 129 pgs: 129 active+clean; 82 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 259 op/s 2024-05-14T15:37:01.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1117469 9.11 MB/sec execute 502 sec latency 45.037 ms 2024-05-14T15:37:02.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1117614 9.09 MB/sec execute 503 sec latency 200.516 ms 2024-05-14T15:37:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:01 smithi144 ceph-mon[35650]: pgmap v384: 129 pgs: 129 active+clean; 82 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-05-14T15:37:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:01 smithi193 ceph-mon[36001]: pgmap v384: 129 pgs: 129 active+clean; 82 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-05-14T15:37:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:01 smithi005 ceph-mon[31559]: pgmap v384: 129 pgs: 129 active+clean; 82 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-05-14T15:37:03.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1118521 9.09 MB/sec execute 504 sec latency 127.056 ms 2024-05-14T15:37:04.161 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1119611 9.08 MB/sec execute 505 sec latency 21.090 ms 2024-05-14T15:37:04.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:03 smithi144 ceph-mon[35650]: pgmap v385: 129 pgs: 129 active+clean; 71 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-14T15:37:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:03 smithi193 ceph-mon[36001]: pgmap v385: 129 pgs: 129 active+clean; 71 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-14T15:37:04.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:03 smithi005 ceph-mon[31559]: pgmap v385: 129 pgs: 129 active+clean; 71 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-14T15:37:04.615 INFO:tasks.workunit.client.0.smithi005.stdout:[1120399] write failed on handle 12093 (Resource temporarily unavailable) 2024-05-14T15:37:04.615 INFO:tasks.workunit.client.0.smithi005.stdout:Child failed with status 1 2024-05-14T15:37:04.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:37:04.616 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-14T15:37:04.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-14T15:37:04.903 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e2b755a64f0d72176b57299d02e880cf0620d780 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-14T15:37:05.136 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d35da2cbaf73443eb60ada0bf16c922a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e2b755a64f0d72176b57299d02e880cf0620d780 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-14T15:37:05.139 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-14T15:37:05.151 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:37:05.164 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-14T15:37:05.174 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-14T15:37:05.174 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T15:37:05.174 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:37:05.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-14T15:37:05.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-14T15:37:05.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T15:37:05.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 -- 172.21.15.5:0/3313519326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c10a150 msgr2=0x7fbd4c103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 --2- 172.21.15.5:0/3313519326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c10a150 0x7fbd4c103d90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002a410 comp rx=0 tx=0).stop 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 -- 172.21.15.5:0/3313519326 shutdown_connections 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 --2- 172.21.15.5:0/3313519326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd4c104400 0x7fbd4c1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 --2- 172.21.15.5:0/3313519326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c10a150 0x7fbd4c103d90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 --2- 172.21.15.5:0/3313519326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c109cb0 0x7fbd4c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 -- 172.21.15.5:0/3313519326 >> 172.21.15.5:0/3313519326 conn(0x7fbd4c0793a0 msgr2=0x7fbd4c0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 -- 172.21.15.5:0/3313519326 shutdown_connections 2024-05-14T15:37:05.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.457+0000 7fbd5358d640 1 -- 172.21.15.5:0/3313519326 wait complete. 2024-05-14T15:37:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 Processor -- start 2024-05-14T15:37:05.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 -- start start 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 0x7fbd4c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c109cb0 0x7fbd4c1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd4c10a150 0x7fbd4c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c113ea0 con 0x7fbd4c104400 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c113d20 con 0x7fbd4c10a150 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd5358d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c114020 con 0x7fbd4c109cb0 2024-05-14T15:37:05.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd51329640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 0x7fbd4c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd51329640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 0x7fbd4c1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51632/0 (socket says 172.21.15.5:51632) 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.458+0000 7fbd51329640 1 -- 172.21.15.5:0/2496911551 learned_addr learned my addr 172.21.15.5:0/2496911551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd50b28640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c109cb0 0x7fbd4c1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51b2a640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd4c10a150 0x7fbd4c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51329640 1 -- 172.21.15.5:0/2496911551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c109cb0 msgr2=0x7fbd4c1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51329640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c109cb0 0x7fbd4c1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51329640 1 -- 172.21.15.5:0/2496911551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd4c10a150 msgr2=0x7fbd4c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51329640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd4c10a150 0x7fbd4c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51329640 1 -- 172.21.15.5:0/2496911551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c19ae10 con 0x7fbd4c104400 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd51329640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 0x7fbd4c1a0570 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400e9f0 tx=0x7fbd4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400ccc0 con 0x7fbd4c104400 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd440054d0 con 0x7fbd4c104400 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c19b100 con 0x7fbd4c104400 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44010430 con 0x7fbd4c104400 2024-05-14T15:37:05.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.459+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c19b5d0 con 0x7fbd4c104400 2024-05-14T15:37:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.460+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c104f90 con 0x7fbd4c104400 2024-05-14T15:37:05.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.463+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbd440046c0 con 0x7fbd4c104400 2024-05-14T15:37:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.463+0000 7fbd4a7fc640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd340b2950 0x7fbd340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.463+0000 7fbd50b28640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd340b2950 0x7fbd340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.463+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7fbd440e3060 con 0x7fbd4c104400 2024-05-14T15:37:05.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.464+0000 7fbd50b28640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd340b2950 0x7fbd340b4e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009230 tx=0x7fbd40031400 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:05.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.467+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd44054df0 con 0x7fbd4c104400 2024-05-14T15:37:05.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.681+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fbd4c102ab0 con 0x7fbd340b2950 2024-05-14T15:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.690+0000 7fbd4a7fc640 1 -- 172.21.15.5:0/2496911551 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fbd4c102ab0 con 0x7fbd340b2950 2024-05-14T15:37:05.691 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T15:25:05", "first": "2024-05-14T15:26:00", "last": "2024-05-14T15:37:01", "last_pruned": "2024-05-14T15:37:01", "created_count": 12, "pruned_count": 6, "active": true} 2024-05-14T15:37:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd340b2950 msgr2=0x7fbd340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd340b2950 0x7fbd340b4e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009230 tx=0x7fbd40031400 comp rx=0 tx=0).stop 2024-05-14T15:37:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 msgr2=0x7fbd4c1a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 0x7fbd4c1a0570 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400e9f0 tx=0x7fbd4400eeb0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 shutdown_connections 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbd340b2950 0x7fbd340b4e60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd4c109cb0 0x7fbd4c1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd4c10a150 0x7fbd4c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 --2- 172.21.15.5:0/2496911551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd4c104400 0x7fbd4c1a0570 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 >> 172.21.15.5:0/2496911551 conn(0x7fbd4c0793a0 msgr2=0x7fbd4c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 shutdown_connections 2024-05-14T15:37:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.692+0000 7fbd5358d640 1 -- 172.21.15.5:0/2496911551 wait complete. 2024-05-14T15:37:05.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T15:37:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 -- 172.21.15.5:0/2570245899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6940108ba0 msgr2=0x7f6940102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 --2- 172.21.15.5:0/2570245899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6940108ba0 0x7f6940102740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f693c009900 tx=0x7f693c02a410 comp rx=0 tx=0).stop 2024-05-14T15:37:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 -- 172.21.15.5:0/2570245899 shutdown_connections 2024-05-14T15:37:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 --2- 172.21.15.5:0/2570245899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 0x7f6940102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 --2- 172.21.15.5:0/2570245899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69401032f0 0x7f694010f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 --2- 172.21.15.5:0/2570245899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6940108ba0 0x7f6940102740 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 -- 172.21.15.5:0/2570245899 >> 172.21.15.5:0/2570245899 conn(0x7f69400782e0 msgr2=0x7f6940078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 -- 172.21.15.5:0/2570245899 shutdown_connections 2024-05-14T15:37:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.820+0000 7f6948557640 1 -- 172.21.15.5:0/2570245899 wait complete. 2024-05-14T15:37:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 Processor -- start 2024-05-14T15:37:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 -- start start 2024-05-14T15:37:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 0x7f69401a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6940108ba0 0x7f69401a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 0x7f694019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6940112df0 con 0x7f69401032f0 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6940112c70 con 0x7f6940108ba0 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f6948557640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6940112f70 con 0x7f6940109040 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f69462f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 0x7f69401a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f69462f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 0x7f69401a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57480/0 (socket says 172.21.15.5:57480) 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.821+0000 7f69462f3640 1 -- 172.21.15.5:0/216307222 learned_addr learned my addr 172.21.15.5:0/216307222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f6945af2640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6940108ba0 0x7f69401a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f6946af4640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 0x7f694019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69462f3640 1 -- 172.21.15.5:0/216307222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 msgr2=0x7f694019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69462f3640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 0x7f694019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69462f3640 1 -- 172.21.15.5:0/216307222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6940108ba0 msgr2=0x7f69401a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69462f3640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6940108ba0 0x7f69401a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69462f3640 1 -- 172.21.15.5:0/216307222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694019aeb0 con 0x7f69401032f0 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f6945af2640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6940108ba0 0x7f69401a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:37:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f6946af4640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 0x7f694019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69462f3640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 0x7f69401a0600 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f693c031a90 tx=0x7f693c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c03f030 con 0x7f69401032f0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694019b0e0 con 0x7f69401032f0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693c003cb0 con 0x7f69401032f0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c035070 con 0x7f69401032f0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.822+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69401a73a0 con 0x7f69401032f0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.823+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6940103e80 con 0x7f69401032f0 2024-05-14T15:37:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.825+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f693c042030 con 0x7f69401032f0 2024-05-14T15:37:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.825+0000 7f69377fe640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f692c0b2950 0x7f692c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.825+0000 7f6945af2640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f692c0b2950 0x7f692c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.825+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9562+0+0 (secure 0 0 0) 0x7f693c103540 con 0x7f69401032f0 2024-05-14T15:37:05.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.826+0000 7f6945af2640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f692c0b2950 0x7f692c0b4e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f694019bf20 tx=0x7f6930005d30 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:05.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:05.829+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f693c0caab0 con 0x7f69401032f0 2024-05-14T15:37:06.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:05 smithi005 ceph-mon[31559]: pgmap v386: 129 pgs: 129 active+clean; 71 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-05-14T15:37:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.169+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 --> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f694006ac00 con 0x7f692c0b2950 2024-05-14T15:37:06.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.201+0000 7f69377fe640 1 -- 172.21.15.5:0/216307222 <== mgr.15246 v2:172.21.15.5:6800/2405629464 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f694006ac00 con 0x7f692c0b2950 2024-05-14T15:37:06.202 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-05-14T15:37:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f692c0b2950 msgr2=0x7f692c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f692c0b2950 0x7f692c0b4e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f694019bf20 tx=0x7f6930005d30 comp rx=0 tx=0).stop 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 msgr2=0x7f69401a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 0x7f69401a0600 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f693c031a90 tx=0x7f693c004270 comp rx=0 tx=0).stop 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 shutdown_connections 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f692c0b2950 0x7f692c0b4e60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6940109040 0x7f694019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6940108ba0 0x7f69401a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 --2- 172.21.15.5:0/216307222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69401032f0 0x7f69401a0600 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 >> 172.21.15.5:0/216307222 conn(0x7f69400782e0 msgr2=0x7f694006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 shutdown_connections 2024-05-14T15:37:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:06.204+0000 7f6948557640 1 -- 172.21.15.5:0/216307222 wait complete. 2024-05-14T15:37:06.224 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-14T15:37:06.235 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-14T15:37:06.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:05 smithi144 ceph-mon[35650]: pgmap v386: 129 pgs: 129 active+clean; 71 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-05-14T15:37:06.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:05 smithi193 ceph-mon[36001]: pgmap v386: 129 pgs: 129 active+clean; 71 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 272 op/s 2024-05-14T15:37:07.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:06 smithi144 ceph-mon[35650]: from='client.18342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:37:07.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:06 smithi144 ceph-mon[35650]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T15:37:07.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:06 smithi193 ceph-mon[36001]: from='client.18342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:37:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:06 smithi193 ceph-mon[36001]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T15:37:07.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:06 smithi005 ceph-mon[31559]: from='client.18342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:37:07.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:06 smithi005 ceph-mon[31559]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T15:37:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:07 smithi144 ceph-mon[35650]: from='client.18348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:37:08.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:07 smithi144 ceph-mon[35650]: pgmap v388: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 73 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 353 op/s 2024-05-14T15:37:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:07 smithi193 ceph-mon[36001]: from='client.18348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:37:08.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:07 smithi193 ceph-mon[36001]: pgmap v388: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 73 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 353 op/s 2024-05-14T15:37:08.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:07 smithi005 ceph-mon[31559]: from='client.18348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T15:37:08.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:07 smithi005 ceph-mon[31559]: pgmap v388: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 73 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 353 op/s 2024-05-14T15:37:10.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:09 smithi144 ceph-mon[35650]: pgmap v389: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-05-14T15:37:10.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:09 smithi193 ceph-mon[36001]: pgmap v389: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-05-14T15:37:10.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:09 smithi005 ceph-mon[31559]: pgmap v389: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-05-14T15:37:12.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:11 smithi144 ceph-mon[35650]: pgmap v390: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-05-14T15:37:12.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:11 smithi193 ceph-mon[36001]: pgmap v390: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-05-14T15:37:12.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:11 smithi005 ceph-mon[31559]: pgmap v390: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 69 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-05-14T15:37:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:13 smithi144 ceph-mon[35650]: pgmap v391: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-05-14T15:37:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:13 smithi144 ceph-mon[35650]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T15:37:14.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:13 smithi193 ceph-mon[36001]: pgmap v391: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-05-14T15:37:14.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:13 smithi193 ceph-mon[36001]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T15:37:14.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:13 smithi005 ceph-mon[31559]: pgmap v391: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-05-14T15:37:14.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:13 smithi005 ceph-mon[31559]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T15:37:16.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:15 smithi144 ceph-mon[35650]: pgmap v393: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-05-14T15:37:16.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:15 smithi193 ceph-mon[36001]: pgmap v393: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-05-14T15:37:16.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:15 smithi005 ceph-mon[31559]: pgmap v393: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-05-14T15:37:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:17 smithi144 ceph-mon[35650]: pgmap v394: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-05-14T15:37:18.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:17 smithi193 ceph-mon[36001]: pgmap v394: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-05-14T15:37:18.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:17 smithi005 ceph-mon[31559]: pgmap v394: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 67 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-05-14T15:37:19.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-14T15:37:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:20.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:19 smithi005 ceph-mon[31559]: pgmap v395: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T15:37:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:19 smithi144 ceph-mon[35650]: pgmap v395: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T15:37:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:19 smithi193 ceph-mon[36001]: pgmap v395: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T15:37:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 -- 172.21.15.5:0/2955512505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c010e8d0 msgr2=0x7f91c0110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 --2- 172.21.15.5:0/2955512505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c010e8d0 0x7f91c0110db0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f91b800b0a0 tx=0x7f91b800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 -- 172.21.15.5:0/2955512505 shutdown_connections 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 --2- 172.21.15.5:0/2955512505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c010e8d0 0x7f91c0110db0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 --2- 172.21.15.5:0/2955512505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c0105460 0x7f91c0105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 --2- 172.21.15.5:0/2955512505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0105db0 0x7f91c010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 -- 172.21.15.5:0/2955512505 >> 172.21.15.5:0/2955512505 conn(0x7f91c00fedc0 msgr2=0x7f91c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 -- 172.21.15.5:0/2955512505 shutdown_connections 2024-05-14T15:37:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 -- 172.21.15.5:0/2955512505 wait complete. 2024-05-14T15:37:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.232+0000 7f91c5f89700 1 Processor -- start 2024-05-14T15:37:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 -- start start 2024-05-14T15:37:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 0x7f91c0103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0105db0 0x7f91c0103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c010e8d0 0x7f91c0103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91c0114430 con 0x7f91c0105460 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01142b0 con 0x7f91c0105db0 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91c5f89700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01145b0 con 0x7f91c010e8d0 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 0x7f91c0103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 0x7f91c0103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50718/0 (socket says 172.21.15.5:50718) 2024-05-14T15:37:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91bf7fe700 1 -- 172.21.15.5:0/1174957400 learned_addr learned my addr 172.21.15.5:0/1174957400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91beffd700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0105db0 0x7f91c0103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.233+0000 7f91bffff700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c010e8d0 0x7f91c0103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bf7fe700 1 -- 172.21.15.5:0/1174957400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c010e8d0 msgr2=0x7f91c0103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c010e8d0 0x7f91c0103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bf7fe700 1 -- 172.21.15.5:0/1174957400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0105db0 msgr2=0x7f91c0103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0105db0 0x7f91c0103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bf7fe700 1 -- 172.21.15.5:0/1174957400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c01a9120 con 0x7f91c0105460 2024-05-14T15:37:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 0x7f91c0103150 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e9a0 tx=0x7f91a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bcff9700 1 -- 172.21.15.5:0/1174957400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800ca10 con 0x7f91c0105460 2024-05-14T15:37:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c01a9410 con 0x7f91c0105460 2024-05-14T15:37:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bcff9700 1 -- 172.21.15.5:0/1174957400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a800cbb0 con 0x7f91c0105460 2024-05-14T15:37:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91bcff9700 1 -- 172.21.15.5:0/1174957400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8018680 con 0x7f91c0105460 2024-05-14T15:37:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.234+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c01a99a0 con 0x7f91c0105460 2024-05-14T15:37:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.238+0000 7f91bcff9700 1 -- 172.21.15.5:0/1174957400 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f91a8010430 con 0x7f91c0105460 2024-05-14T15:37:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91bcff9700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f91ac0b2a70 0x7f91ac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 Processor -- start 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91beffd700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f91ac0b2a70 0x7f91ac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 -- start start 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91bcff9700 1 -- 172.21.15.5:0/1174957400 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f91a80e2b80 con 0x7f91c0105460 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47959f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4795e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 0x55fa4795f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa47873860 con 0x55fa4795ec60 2024-05-14T15:37:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa478127c0 con 0x55fa4795a450 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.239+0000 7f91d499db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fa4786f820 con 0x55fa477d3e40 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91beffd700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f91ac0b2a70 0x7f91ac0b4f80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91c0102420 tx=0x7f91b000b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 0x55fa4795f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 0x55fa4795f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50724/0 (socket says 172.21.15.5:50724) 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 -- 172.21.15.5:0/271997031 learned_addr learned my addr 172.21.15.5:0/271997031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47959f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91beffd700 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4795e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 -- 172.21.15.5:0/271997031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 msgr2=0x55fa47959f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47959f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 -- 172.21.15.5:0/271997031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 msgr2=0x55fa4795e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4795e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 -- 172.21.15.5:0/271997031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa477c08d0 con 0x55fa4795ec60 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47959f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91bffff700 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 0x55fa4795f070 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f91b8015000 tx=0x7f91b80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200c72ca22573090 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.240+0000 7f91b5ffb700 1 -- 172.21.15.5:0/271997031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b800bcb0 con 0x55fa4795ec60 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91b5ffb700 1 -- 172.21.15.5:0/271997031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b8006020 con 0x55fa4795ec60 2024-05-14T15:37:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91b5ffb700 1 -- 172.21.15.5:0/271997031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8005040 con 0x55fa4795ec60 2024-05-14T15:37:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 -- 172.21.15.5:0/271997031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 msgr2=0x55fa4795f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 0x55fa4795f070 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f91b8015000 tx=0x7f91b80092e0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 -- 172.21.15.5:0/271997031 shutdown_connections 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795ec60 0x55fa4795f070 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47959f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 --2- 172.21.15.5:0/271997031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4795e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.241+0000 7f91d499db80 1 -- 172.21.15.5:0/271997031 >> 172.21.15.5:0/271997031 conn(0x55fa4781b600 msgr2=0x55fa4794c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.242+0000 7f91d499db80 1 -- 172.21.15.5:0/271997031 shutdown_connections 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.242+0000 7f91d499db80 1 -- 172.21.15.5:0/271997031 wait complete. 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.242+0000 7f91d499db80 1 -- start start 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.242+0000 7f91d499db80 1 Processor -- start 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47963920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47963e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795ec60 0x55fa479643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa47966b50 con 0x55fa4795a450 2024-05-14T15:37:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa477d9c30 con 0x55fa4795ec60 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fa47966cd0 con 0x55fa477d3e40 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91d499db80 10 client.0 _ll_register_callbacks cb 0x55fa4780b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47963e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47963e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50738/0 (socket says 172.21.15.5:50738) 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 -- 172.21.15.5:0/3101824405 learned_addr learned my addr 172.21.15.5:0/3101824405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47963920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91bffff700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795ec60 0x55fa479643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 -- 172.21.15.5:0/3101824405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 msgr2=0x55fa47963920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47963920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 -- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795ec60 msgr2=0x55fa479643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795ec60 0x55fa479643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 -- 172.21.15.5:0/3101824405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4780f690 con 0x55fa4795a450 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f91beffd700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47963e60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f91b00116f0 tx=0x7f91b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da393e0c5f74f48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.243+0000 7f919effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f919effd700 1 -- 172.21.15.5:0/3101824405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b001e830 con 0x55fa4795a450 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f919effd700 1 -- 172.21.15.5:0/3101824405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f91b001e9d0 con 0x55fa4795a450 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b8006020 con 0x55fa4795a450 2024-05-14T15:37:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f919effd700 1 -- 172.21.15.5:0/3101824405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f91b0031050 con 0x55fa4795a450 2024-05-14T15:37:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 10 client.18360 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fa47971760 con 0x55fa4795a450 2024-05-14T15:37:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f919effd700 1 -- 172.21.15.5:0/3101824405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b8006020 con 0x55fa4795a450 2024-05-14T15:37:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f919effd700 1 -- 172.21.15.5:0/3101824405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f91b0026020 con 0x55fa4795a450 2024-05-14T15:37:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 10 client.18360 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 20 client.18360 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa47973da0 0x55fa479761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 4 client.18360 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.244+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa47881240 con 0x55fa47973da0 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.245+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa47973da0 0x55fa479761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.245+0000 7f919effd700 10 client.18360 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.245+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa47973da0 0x55fa479761e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af748c2d5b93bde server_cookie=6b5e17b7af5777eb in_seq=0 out_seq=0 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.246+0000 7f919effd700 1 -- 172.21.15.5:0/3101824405 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa47881240 con 0x55fa47973da0 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.246+0000 7f919effd700 10 client.18360 handle_command_reply: tid=0 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.246+0000 7f91d499db80 1 client.18360 shutdown 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.246+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 msgr2=0x55fa47963e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.246+0000 7f91d499db80 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47963e60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f91b00116f0 tx=0x7f91b00047c0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 shutdown_connections 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa47973da0 0x55fa479761e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795ec60 0x55fa479643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa477d3e40 0x55fa47963920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f919effd700 0 client.18360 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 --2- 172.21.15.5:0/3101824405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47963e60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 >> 172.21.15.5:0/3101824405 conn(0x55fa4781b600 msgr2=0x55fa47950450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 shutdown_connections 2024-05-14T15:37:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 1 -- 172.21.15.5:0/3101824405 wait complete. 2024-05-14T15:37:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.247+0000 7f91d499db80 20 client.18360 trim_cache size 0 max 16384 2024-05-14T15:37:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 Processor -- start 2024-05-14T15:37:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 -- start start 2024-05-14T15:37:21.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 0x55fa47962c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4794c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa4796fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01b0db0 con 0x55fa477d3e40 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01ae640 con 0x55fa4795a450 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91d499db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01ac4b0 con 0x55fa4795ec60 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 0x55fa47962c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 0x55fa47962c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50746/0 (socket says 172.21.15.5:50746) 2024-05-14T15:37:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91bf7fe700 1 -- 172.21.15.5:0/1333122831 learned_addr learned my addr 172.21.15.5:0/1333122831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91beffd700 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4794c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.288+0000 7f91bffff700 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa4796fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91beffd700 1 -- 172.21.15.5:0/1333122831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 msgr2=0x55fa4796fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91beffd700 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa4796fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91beffd700 1 -- 172.21.15.5:0/1333122831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 msgr2=0x55fa47962c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91beffd700 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 0x55fa47962c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91beffd700 1 -- 172.21.15.5:0/1333122831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa479641e0 con 0x55fa4795a450 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 0x55fa47962c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91beffd700 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4794c2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f91b001ca10 tx=0x7f91b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ea3f892c3a1eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91b57fa700 1 -- 172.21.15.5:0/1333122831 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b001cc80 con 0x55fa4795a450 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91b57fa700 1 -- 172.21.15.5:0/1333122831 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b003a070 con 0x55fa4795a450 2024-05-14T15:37:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.289+0000 7f91b57fa700 1 -- 172.21.15.5:0/1333122831 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b001edf0 con 0x55fa4795a450 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 -- 172.21.15.5:0/1333122831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 msgr2=0x55fa4794c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4794c2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f91b001ca10 tx=0x7f91b0004750 comp rx=0 tx=0).stop 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 -- 172.21.15.5:0/1333122831 shutdown_connections 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa4796fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa477d3e40 0x55fa47962c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 --2- 172.21.15.5:0/1333122831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa4795a450 0x55fa4794c2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 -- 172.21.15.5:0/1333122831 >> 172.21.15.5:0/1333122831 conn(0x55fa4781b600 msgr2=0x55fa4794bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 -- 172.21.15.5:0/1333122831 shutdown_connections 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.290+0000 7f91d499db80 1 -- 172.21.15.5:0/1333122831 wait complete. 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 -- start start 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 Processor -- start 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa477d3e40 0x55fa4796c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47969930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa47969f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01acb00 con 0x55fa4795a450 2024-05-14T15:37:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01adf70 con 0x55fa477d3e40 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91d499db80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91c01ad1d0 con 0x55fa4795ec60 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.291+0000 7f91beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47969930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91d499db80 10 client.0 _ll_register_callbacks cb 0x55fa4780b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47969930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50750/0 (socket says 172.21.15.5:50750) 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 -- 172.21.15.5:0/951038760 learned_addr learned my addr 172.21.15.5:0/951038760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa477d3e40 0x55fa4796c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91bffff700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa47969f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 -- 172.21.15.5:0/951038760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 msgr2=0x55fa47969f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa47969f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 -- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa477d3e40 msgr2=0x55fa4796c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa477d3e40 0x55fa4796c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 -- 172.21.15.5:0/951038760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4796a720 con 0x55fa4795a450 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa477d3e40 0x55fa4796c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f91beffd700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47969930 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f91b000ba00 tx=0x7f91b001ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c636a639445e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f919effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f919effd700 1 -- 172.21.15.5:0/951038760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b001c3a0 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.292+0000 7f919effd700 1 -- 172.21.15.5:0/951038760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9562+0+0 (secure 0 0 0) 0x7f91b001c540 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b003a070 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f919effd700 1 -- 172.21.15.5:0/951038760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f91b0050050 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 10 client.18366 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fa479de550 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f919effd700 1 -- 172.21.15.5:0/951038760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b003a070 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f919effd700 1 -- 172.21.15.5:0/951038760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f91b0034070 con 0x55fa4795a450 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 10 client.18366 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 20 client.18366 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa479deb40 0x55fa479e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 4 client.18366 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.293+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa4787b2e0 con 0x55fa479deb40 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa479deb40 0x55fa479e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f919effd700 10 client.18366 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91bf7fe700 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa479deb40 0x55fa479e0f60 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43bb7ec486447fe server_cookie=c1c9b7f23d735938 in_seq=0 out_seq=0 2024-05-14T15:37:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f919effd700 1 -- 172.21.15.5:0/951038760 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa4787b2e0 con 0x55fa479deb40 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f919effd700 10 client.18366 handle_command_reply: tid=0 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 client.18366 shutdown 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 msgr2=0x55fa47969930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47969930 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f91b000ba00 tx=0x7f91b001ce30 comp rx=0 tx=0).stop 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 shutdown_connections 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x55fa479deb40 0x55fa479e0f60 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa4795ec60 0x55fa47969f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa477d3e40 0x55fa4796c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.294+0000 7f91d499db80 1 --2- 172.21.15.5:0/951038760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa4795a450 0x55fa47969930 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f919effd700 0 client.18366 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 >> 172.21.15.5:0/951038760 conn(0x55fa4781b600 msgr2=0x55fa4794c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 shutdown_connections 2024-05-14T15:37:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91d499db80 1 -- 172.21.15.5:0/951038760 wait complete. 2024-05-14T15:37:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91d499db80 20 client.18366 trim_cache size 0 max 16384 2024-05-14T15:37:21.307 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:37:21.307 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-14T15:37:21.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-14T15:37:21.308 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:37:21.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f91ac0b2a70 msgr2=0x7f91ac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f91ac0b2a70 0x7f91ac0b4f80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91c0102420 tx=0x7f91b000b410 comp rx=0 tx=0).stop 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 msgr2=0x7f91c0103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 0x7f91c0103150 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e9a0 tx=0x7f91a800ecb0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 shutdown_connections 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f91ac0b2a70 0x7f91ac0b4f80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91c010e8d0 0x7f91c0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c0105460 0x7f91c0103150 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 --2- 172.21.15.5:0/1174957400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c0105db0 0x7f91c0103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.295+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 >> 172.21.15.5:0/1174957400 conn(0x7f91c00fedc0 msgr2=0x7f91c00ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.296+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 shutdown_connections 2024-05-14T15:37:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:21.296+0000 7f91c5f89700 1 -- 172.21.15.5:0/1174957400 wait complete. 2024-05-14T15:37:22.103 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-14T15:37:22.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:37:22.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:21 smithi144 ceph-mon[35650]: pgmap v396: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T15:37:22.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:21 smithi193 ceph-mon[36001]: pgmap v396: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T15:37:22.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:21 smithi005 ceph-mon[31559]: pgmap v396: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-05-14T15:37:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:24.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:23 smithi144 ceph-mon[35650]: pgmap v397: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-05-14T15:37:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:23 smithi144 ceph-mon[35650]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T15:37:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:23 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:37:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:23 smithi193 ceph-mon[36001]: pgmap v397: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-05-14T15:37:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:23 smithi193 ceph-mon[36001]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T15:37:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:23 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:37:24.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:23 smithi005 ceph-mon[31559]: pgmap v397: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-05-14T15:37:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:23 smithi005 ceph-mon[31559]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T15:37:24.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:23 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 -- 172.21.15.5:0/1607587795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8010e920 msgr2=0x7efc80110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 --2- 172.21.15.5:0/1607587795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8010e920 0x7efc80110e00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b0a0 tx=0x7efc7800b3b0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 -- 172.21.15.5:0/1607587795 shutdown_connections 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 --2- 172.21.15.5:0/1607587795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8010e920 0x7efc80110e00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 --2- 172.21.15.5:0/1607587795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc801054f0 0x7efc80105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 --2- 172.21.15.5:0/1607587795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc80105e40 0x7efc8010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 -- 172.21.15.5:0/1607587795 >> 172.21.15.5:0/1607587795 conn(0x7efc80078e00 msgr2=0x7efc80079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 -- 172.21.15.5:0/1607587795 shutdown_connections 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.598+0000 7efc855ad700 1 -- 172.21.15.5:0/1607587795 wait complete. 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 Processor -- start 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 -- start start 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 0x7efc801031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc80105e40 0x7efc80103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8010e920 0x7efc80103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc801144f0 con 0x7efc801054f0 2024-05-14T15:37:24.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc80114370 con 0x7efc8010e920 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.599+0000 7efc855ad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc80114670 con 0x7efc80105e40 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 0x7efc801031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 0x7efc801031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50766/0 (socket says 172.21.15.5:50766) 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 -- 172.21.15.5:0/42415849 learned_addr learned my addr 172.21.15.5:0/42415849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc767fc700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc80105e40 0x7efc80103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7f7fe700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8010e920 0x7efc80103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 -- 172.21.15.5:0/42415849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc80105e40 msgr2=0x7efc80103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc80105e40 0x7efc80103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 -- 172.21.15.5:0/42415849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8010e920 msgr2=0x7efc80103c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8010e920 0x7efc80103c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 -- 172.21.15.5:0/42415849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc801a91a0 con 0x7efc801054f0 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc767fc700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc80105e40 0x7efc80103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7f7fe700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8010e920 0x7efc80103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7effd700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 0x7efc801031e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e9c0 tx=0x7efc7000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7000cbd0 con 0x7efc801054f0 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc801a9490 con 0x7efc801054f0 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc7000cd70 con 0x7efc801054f0 2024-05-14T15:37:24.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70010430 con 0x7efc801054f0 2024-05-14T15:37:24.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.600+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a9980 con 0x7efc801054f0 2024-05-14T15:37:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.601+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc800512c0 con 0x7efc801054f0 2024-05-14T15:37:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.604+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7efc70004750 con 0x7efc801054f0 2024-05-14T15:37:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.605+0000 7efc7cff9700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efc6c0b29f0 0x7efc6c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.605+0000 7efc767fc700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efc6c0b29f0 0x7efc6c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.605+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7efc700e2ff0 con 0x7efc801054f0 2024-05-14T15:37:24.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.606+0000 7efc767fc700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efc6c0b29f0 0x7efc6c0b4f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc80079370 tx=0x7efc68009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:24.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.609+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc70054df0 con 0x7efc801054f0 2024-05-14T15:37:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.877+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc8002e5f0 con 0x7efc801054f0 2024-05-14T15:37:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.878+0000 7efc7cff9700 1 -- 172.21.15.5:0/42415849 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11864 (secure 0 0 0) 0x7efc700aa5f0 con 0x7efc801054f0 2024-05-14T15:37:24.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:37:24.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:35:24.885756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27547":{"gid":27547,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3068356254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3068356254},{"type":"v1","addr":"172.21.15.144:6835","nonce":3068356254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:37:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.880+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efc6c0b29f0 msgr2=0x7efc6c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.880+0000 7efc855ad700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efc6c0b29f0 0x7efc6c0b4f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc80079370 tx=0x7efc68009260 comp rx=0 tx=0).stop 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.880+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 msgr2=0x7efc801031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.880+0000 7efc855ad700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 0x7efc801031e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e9c0 tx=0x7efc7000ed80 comp rx=0 tx=0).stop 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.880+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 shutdown_connections 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7efc6c0b29f0 0x7efc6c0b4f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8010e920 0x7efc80103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc801054f0 0x7efc801031e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 --2- 172.21.15.5:0/42415849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc80105e40 0x7efc80103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 >> 172.21.15.5:0/42415849 conn(0x7efc80078e00 msgr2=0x7efc80100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 shutdown_connections 2024-05-14T15:37:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:24.881+0000 7efc855ad700 1 -- 172.21.15.5:0/42415849 wait complete. 2024-05-14T15:37:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-14T15:37:25.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:24 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:37:25.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:24 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:37:25.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:24 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:37:25.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:24 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/42415849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:37:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:24 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:37:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:24 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:37:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:24 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:37:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:24 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/42415849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:37:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:24 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:37:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:24 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:37:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:24 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:37:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:24 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/42415849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:37:25.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-14T15:37:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:26.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:25 smithi005 ceph-mon[31559]: pgmap v399: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-05-14T15:37:26.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:25 smithi144 ceph-mon[35650]: pgmap v399: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-05-14T15:37:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:25 smithi193 ceph-mon[36001]: pgmap v399: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-05-14T15:37:27.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 -- 172.21.15.5:0/1422089353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 msgr2=0x7f26841106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 --2- 172.21.15.5:0/1422089353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f26841106f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f266c009990 tx=0x7f266c009ca0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 -- 172.21.15.5:0/1422089353 shutdown_connections 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 --2- 172.21.15.5:0/1422089353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2684110c30 0x7f2684113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 --2- 172.21.15.5:0/1422089353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684103080 0x7f2684103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 --2- 172.21.15.5:0/1422089353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f26841106f0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 -- 172.21.15.5:0/1422089353 >> 172.21.15.5:0/1422089353 conn(0x7f26840fedf0 msgr2=0x7f2684101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.728+0000 7f268a7fd700 1 -- 172.21.15.5:0/1422089353 shutdown_connections 2024-05-14T15:37:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.729+0000 7f268a7fd700 1 -- 172.21.15.5:0/1422089353 wait complete. 2024-05-14T15:37:27.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.729+0000 7f268a7fd700 1 Processor -- start 2024-05-14T15:37:27.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.729+0000 7f268a7fd700 1 -- start start 2024-05-14T15:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f268a7fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2684103080 0x7f268419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f268a7fd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f268419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f268a7fd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684110c30 0x7f26841a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f268a7fd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2684116660 con 0x7f26841039d0 2024-05-14T15:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f268a7fd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26841164e0 con 0x7f2684103080 2024-05-14T15:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f268a7fd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26841167e0 con 0x7f2684110c30 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f268419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f268419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43588/0 (socket says 172.21.15.5:43588) 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 -- 172.21.15.5:0/2545233770 learned_addr learned my addr 172.21.15.5:0/2545233770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f2688d9a700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684110c30 0x7f26841a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f2683fff700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2684103080 0x7f268419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 -- 172.21.15.5:0/2545233770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684110c30 msgr2=0x7f26841a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684110c30 0x7f26841a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 -- 172.21.15.5:0/2545233770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2684103080 msgr2=0x7f268419d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2684103080 0x7f268419d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.730+0000 7f267bfff700 1 -- 172.21.15.5:0/2545233770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26841a1f80 con 0x7f26841039d0 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.731+0000 7f267bfff700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f268419d650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f266c006010 tx=0x7f266c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.731+0000 7f2681ffb700 1 -- 172.21.15.5:0/2545233770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c027070 con 0x7f26841039d0 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.731+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26841a2210 con 0x7f26841039d0 2024-05-14T15:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.731+0000 7f2681ffb700 1 -- 172.21.15.5:0/2545233770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c003f20 con 0x7f26841039d0 2024-05-14T15:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.731+0000 7f2681ffb700 1 -- 172.21.15.5:0/2545233770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c022a20 con 0x7f26841039d0 2024-05-14T15:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.731+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841a9b90 con 0x7f26841039d0 2024-05-14T15:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.734+0000 7f2681ffb700 1 -- 172.21.15.5:0/2545233770 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f266c02a070 con 0x7f26841039d0 2024-05-14T15:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.734+0000 7f2681ffb700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f26700b29f0 0x7f26700b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 Processor -- start 2024-05-14T15:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2681ffb700 1 -- 172.21.15.5:0/2545233770 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f266c0e3d00 con 0x7f26841039d0 2024-05-14T15:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2683fff700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f26700b29f0 0x7f26700b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 -- start start 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df65f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df66470 0x55870df66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55870de83860 con 0x55870dde3e40 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55870de227c0 con 0x55870df61c60 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2699211b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55870de7f820 con 0x55870df66470 2024-05-14T15:37:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2683fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2683fff700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f26700b29f0 0x7f26700b4f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2674005950 tx=0x7f267400a300 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2683fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df61720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43602/0 (socket says 172.21.15.5:43602) 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2683fff700 1 -- 172.21.15.5:0/1368970439 learned_addr learned my addr 172.21.15.5:0/1368970439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f267bfff700 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df65f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.735+0000 7f2688d9a700 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df66470 0x55870df66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f2683fff700 1 -- 172.21.15.5:0/1368970439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df66470 msgr2=0x55870df66880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f2683fff700 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df66470 0x55870df66880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f2683fff700 1 -- 172.21.15.5:0/1368970439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 msgr2=0x55870df65f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f2683fff700 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df65f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f2683fff700 1 -- 172.21.15.5:0/1368970439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870ddd08d0 con 0x55870dde3e40 2024-05-14T15:37:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f2683fff700 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df61720 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f267400fe20 tx=0x7f2674012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8c5244b8ac38a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f267a7fc700 1 -- 172.21.15.5:0/1368970439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674016070 con 0x55870dde3e40 2024-05-14T15:37:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f267a7fc700 1 -- 172.21.15.5:0/1368970439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267401c420 con 0x55870dde3e40 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.736+0000 7f267a7fc700 1 -- 172.21.15.5:0/1368970439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267401b660 con 0x55870dde3e40 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 -- 172.21.15.5:0/1368970439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 msgr2=0x55870df61720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df61720 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f267400fe20 tx=0x7f2674012000 comp rx=0 tx=0).stop 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 -- 172.21.15.5:0/1368970439 shutdown_connections 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df66470 0x55870df66880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df61720 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 --2- 172.21.15.5:0/1368970439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df65f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 -- 172.21.15.5:0/1368970439 >> 172.21.15.5:0/1368970439 conn(0x55870de2b600 msgr2=0x55870df5c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 -- 172.21.15.5:0/1368970439 shutdown_connections 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 -- 172.21.15.5:0/1368970439 wait complete. 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.737+0000 7f2699211b80 1 -- start start 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 Processor -- start 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df6b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df61c60 0x55870df6b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df66470 0x55870df6bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55870df6e420 con 0x55870dde3e40 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55870dde9c30 con 0x55870df66470 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55870df6e5a0 con 0x55870df61c60 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df6b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df6b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43608/0 (socket says 172.21.15.5:43608) 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 -- 172.21.15.5:0/756316443 learned_addr learned my addr 172.21.15.5:0/756316443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2699211b80 10 client.0 _ll_register_callbacks cb 0x55870de1b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2688d9a700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df66470 0x55870df6bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f267bfff700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df61c60 0x55870df6b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 -- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df61c60 msgr2=0x55870df6b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df61c60 0x55870df6b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 -- 172.21.15.5:0/756316443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df66470 msgr2=0x55870df6bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.738+0000 7f2683fff700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df66470 0x55870df6bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f2683fff700 1 -- 172.21.15.5:0/756316443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870de1f690 con 0x55870dde3e40 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f2683fff700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df6b120 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2674010710 tx=0x7f267401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86d944ee2941634 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f26637fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f26637fe700 1 -- 172.21.15.5:0/756316443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674033020 con 0x55870dde3e40 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267401c420 con 0x55870dde3e40 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f26637fe700 1 -- 172.21.15.5:0/756316443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f2674010ad0 con 0x55870dde3e40 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f26637fe700 1 -- 172.21.15.5:0/756316443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f267403b050 con 0x55870dde3e40 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f2699211b80 10 client.18384 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55870df78fc0 con 0x55870dde3e40 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.739+0000 7f26637fe700 1 -- 172.21.15.5:0/756316443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267401c420 con 0x55870dde3e40 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f26637fe700 1 -- 172.21.15.5:0/756316443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f267402f420 con 0x55870dde3e40 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f2699211b80 10 client.18384 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f2699211b80 10 client.18384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f2699211b80 20 client.18384 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f2699211b80 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870df7d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f2699211b80 4 client.18384 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55870de91240 con 0x55870df7b540 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f267bfff700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870df7d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f267bfff700 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870df7d960 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61e008b5ef39119a server_cookie=fe8e96180c09c3d5 in_seq=0 out_seq=0 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.740+0000 7f26637fe700 10 client.18384 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f26637fe700 1 -- 172.21.15.5:0/756316443 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55870de91240 con 0x55870df7b540 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f26637fe700 10 client.18384 handle_command_reply: tid=0 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f2699211b80 1 client.18384 shutdown 2024-05-14T15:37:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 msgr2=0x55870df6b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f2699211b80 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df6b120 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2674010710 tx=0x7f267401c8b0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 shutdown_connections 2024-05-14T15:37:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.741+0000 7f2699211b80 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870df7d960 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f26637fe700 0 client.18384 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:37:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df66470 0x55870df6bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870dde3e40 0x55870df6b120 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 1 --2- 172.21.15.5:0/756316443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870df61c60 0x55870df6b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 >> 172.21.15.5:0/756316443 conn(0x55870de2b600 msgr2=0x55870df60450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 shutdown_connections 2024-05-14T15:37:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 1 -- 172.21.15.5:0/756316443 wait complete. 2024-05-14T15:37:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.742+0000 7f2699211b80 20 client.18384 trim_cache size 0 max 16384 2024-05-14T15:37:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 Processor -- start 2024-05-14T15:37:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 -- start start 2024-05-14T15:37:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df6a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df61700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870df77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b50f0 con 0x55870df66470 2024-05-14T15:37:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b2980 con 0x55870df61c60 2024-05-14T15:37:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2699211b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b07f0 con 0x55870dde3e40 2024-05-14T15:37:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2688d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870df77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2688d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870df77840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43616/0 (socket says 172.21.15.5:43616) 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.757+0000 7f2688d9a700 1 -- 172.21.15.5:0/1705034562 learned_addr learned my addr 172.21.15.5:0/1705034562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2688d9a700 1 -- 172.21.15.5:0/1705034562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 msgr2=0x55870df6a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2688d9a700 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df6a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2688d9a700 1 -- 172.21.15.5:0/1705034562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 msgr2=0x55870df61700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2688d9a700 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df61700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2688d9a700 1 -- 172.21.15.5:0/1705034562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870df69080 con 0x55870df66470 2024-05-14T15:37:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2688d9a700 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870df77840 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fa0 tx=0x7f267c009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2c61e71af091a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2679ffb700 1 -- 172.21.15.5:0/1705034562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01b030 con 0x55870df66470 2024-05-14T15:37:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2679ffb700 1 -- 172.21.15.5:0/1705034562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c0178f0 con 0x55870df66470 2024-05-14T15:37:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.758+0000 7f2679ffb700 1 -- 172.21.15.5:0/1705034562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01fa00 con 0x55870df66470 2024-05-14T15:37:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 -- 172.21.15.5:0/1705034562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 msgr2=0x55870df77840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870df77840 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fa0 tx=0x7f267c009100 comp rx=0 tx=0).stop 2024-05-14T15:37:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 -- 172.21.15.5:0/1705034562 shutdown_connections 2024-05-14T15:37:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870df77840 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df6a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 --2- 172.21.15.5:0/1705034562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df61700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 -- 172.21.15.5:0/1705034562 >> 172.21.15.5:0/1705034562 conn(0x55870de2b600 msgr2=0x55870df5bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 -- 172.21.15.5:0/1705034562 shutdown_connections 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 -- 172.21.15.5:0/1705034562 wait complete. 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.759+0000 7f2699211b80 1 -- start start 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 Processor -- start 2024-05-14T15:37:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df7e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870dfdda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b0e40 con 0x55870df66470 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b22b0 con 0x55870df61c60 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b1510 con 0x55870dde3e40 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2699211b80 10 client.0 _ll_register_callbacks cb 0x55870de1b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f267bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df7e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2688d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870dfdda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2683fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f267bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df7e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.5:51982/0 (socket says 172.21.15.5:51982) 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f267bfff700 1 -- 172.21.15.5:0/2137181848 learned_addr learned my addr 172.21.15.5:0/2137181848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.760+0000 7f2683fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df7dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54166/0 (socket says 172.21.15.5:54166) 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2688d9a700 1 -- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 msgr2=0x55870df7dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2688d9a700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df7dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2688d9a700 1 -- 172.21.15.5:0/2137181848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 msgr2=0x55870df7e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:27.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2688d9a700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df7e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2688d9a700 1 -- 172.21.15.5:0/2137181848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870dfde090 con 0x55870df66470 2024-05-14T15:37:27.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2683fff700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:37:27.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f267bfff700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df7e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:37:27.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2688d9a700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870dfdda50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f267c0099e0 tx=0x7f267c017cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e277068a618530 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:27.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f26637fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:27.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f26637fe700 1 -- 172.21.15.5:0/2137181848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01b050 con 0x55870df66470 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f26637fe700 1 -- 172.21.15.5:0/2137181848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f267c01f600 con 0x55870df66470 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c0178f0 con 0x55870df66470 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f26637fe700 1 -- 172.21.15.5:0/2137181848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f267c034050 con 0x55870df66470 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2699211b80 10 client.18390 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55870dfe5360 con 0x55870df66470 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.761+0000 7f26637fe700 1 -- 172.21.15.5:0/2137181848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c0178f0 con 0x55870df66470 2024-05-14T15:37:27.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f26637fe700 1 -- 172.21.15.5:0/2137181848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f267c02b800 con 0x55870df66470 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2699211b80 10 client.18390 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2699211b80 10 client.18390 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2699211b80 20 client.18390 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2699211b80 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870dfe79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2699211b80 4 client.18390 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 --> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55870de8b2e0 con 0x55870df7b540 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2683fff700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870dfe79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:27.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f2683fff700 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870dfe79c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2462b77288694738 server_cookie=e3839f989ebae717 in_seq=0 out_seq=0 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.762+0000 7f26637fe700 10 client.18390 ms_handle_connect on v2:172.21.15.193:6838/570860802 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f26637fe700 1 -- 172.21.15.5:0/2137181848 <== mds.2 v2:172.21.15.193:6838/570860802 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55870de8b2e0 con 0x55870df7b540 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f26637fe700 10 client.18390 handle_command_reply: tid=0 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 client.18390 shutdown 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 msgr2=0x55870dfdda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870dfdda50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f267c0099e0 tx=0x7f267c017cd0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 shutdown_connections 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55870df66470 0x55870dfdda50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55870dde3e40 0x55870df7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.193:6838/570860802,v1:172.21.15.193:6839/570860802] conn(0x55870df7b540 0x55870dfe79c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 --2- 172.21.15.5:0/2137181848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55870df61c60 0x55870df7e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f26637fe700 0 client.18390 ms_handle_reset on v2:172.21.15.193:6838/570860802 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 >> 172.21.15.5:0/2137181848 conn(0x55870de2b600 msgr2=0x55870df6a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 shutdown_connections 2024-05-14T15:37:27.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 1 -- 172.21.15.5:0/2137181848 wait complete. 2024-05-14T15:37:27.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.763+0000 7f2699211b80 20 client.18390 trim_cache size 0 max 16384 2024-05-14T15:37:27.775 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:37:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f26700b29f0 msgr2=0x7f26700b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f26700b29f0 0x7f26700b4f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2674005950 tx=0x7f267400a300 comp rx=0 tx=0).stop 2024-05-14T15:37:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 msgr2=0x7f268419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f268419d650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f266c006010 tx=0x7f266c0047b0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 shutdown_connections 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f26700b29f0 0x7f26700b4f00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2684110c30 0x7f26841a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2684103080 0x7f268419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 --2- 172.21.15.5:0/2545233770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26841039d0 0x7f268419d650 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 >> 172.21.15.5:0/2545233770 conn(0x7f26840fedf0 msgr2=0x7f26841128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 shutdown_connections 2024-05-14T15:37:27.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:27.764+0000 7f268a7fd700 1 -- 172.21.15.5:0/2545233770 wait complete. 2024-05-14T15:37:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:27 smithi144 ceph-mon[35650]: pgmap v400: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-05-14T15:37:28.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:27 smithi193 ceph-mon[36001]: pgmap v400: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-05-14T15:37:28.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:27 smithi005 ceph-mon[31559]: pgmap v400: 129 pgs: 129 active+clean; 65 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-05-14T15:37:28.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-14T15:37:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:30.295 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:29 smithi005 ceph-mon[31559]: pgmap v401: 129 pgs: 129 active+clean; 64 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-05-14T15:37:30.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:29 smithi144 ceph-mon[35650]: pgmap v401: 129 pgs: 129 active+clean; 64 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-05-14T15:37:30.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:29 smithi193 ceph-mon[36001]: pgmap v401: 129 pgs: 129 active+clean; 64 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-05-14T15:37:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 -- 172.21.15.5:0/995660928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84108f10 msgr2=0x7fdb84109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 --2- 172.21.15.5:0/995660928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84108f10 0x7fdb84109320 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdb700099c0 tx=0x7fdb70009cd0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 -- 172.21.15.5:0/995660928 shutdown_connections 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 --2- 172.21.15.5:0/995660928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb84109860 0x7fdb841111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 --2- 172.21.15.5:0/995660928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb841045e0 0x7fdb841049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 --2- 172.21.15.5:0/995660928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84108f10 0x7fdb84109320 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 -- 172.21.15.5:0/995660928 >> 172.21.15.5:0/995660928 conn(0x7fdb840ffed0 msgr2=0x7fdb84102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 -- 172.21.15.5:0/995660928 shutdown_connections 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.718+0000 7fdb8b065700 1 -- 172.21.15.5:0/995660928 wait complete. 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 Processor -- start 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 -- start start 2024-05-14T15:37:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb841045e0 0x7fdb84076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84108f10 0x7fdb84077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 0x7fdb84077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84114470 con 0x7fdb84109860 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841142f0 con 0x7fdb841045e0 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb8b065700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841145f0 con 0x7fdb84108f10 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb89602700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 0x7fdb84077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb89602700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 0x7fdb84077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43646/0 (socket says 172.21.15.5:43646) 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb89602700 1 -- 172.21.15.5:0/3838058235 learned_addr learned my addr 172.21.15.5:0/3838058235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.719+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb841045e0 0x7fdb84076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84108f10 0x7fdb84077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb89602700 1 -- 172.21.15.5:0/3838058235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84108f10 msgr2=0x7fdb84077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb89602700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84108f10 0x7fdb84077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb89602700 1 -- 172.21.15.5:0/3838058235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb841045e0 msgr2=0x7fdb84076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb89602700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb841045e0 0x7fdb84076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb89602700 1 -- 172.21.15.5:0/3838058235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb841a9200 con 0x7fdb84109860 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84108f10 0x7fdb84077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb89602700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 0x7fdb84077650 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00ada0 tx=0x7fdb7c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb81ffb700 1 -- 172.21.15.5:0/3838058235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c017020 con 0x7fdb84109860 2024-05-14T15:37:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb81ffb700 1 -- 172.21.15.5:0/3838058235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c00ce30 con 0x7fdb84109860 2024-05-14T15:37:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb81ffb700 1 -- 172.21.15.5:0/3838058235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c005350 con 0x7fdb84109860 2024-05-14T15:37:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb841a94f0 con 0x7fdb84109860 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.720+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841a99b0 con 0x7fdb84109860 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.723+0000 7fdb81ffb700 1 -- 172.21.15.5:0/3838058235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fdb7c00c980 con 0x7fdb84109860 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.724+0000 7fdb81ffb700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb740b29f0 0x7fdb740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.724+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb740b29f0 0x7fdb740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.724+0000 7fdb81ffb700 1 -- 172.21.15.5:0/3838058235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fdb7c0e2170 con 0x7fdb84109860 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.724+0000 7fdb99a79b80 1 Processor -- start 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.724+0000 7fdb99a79b80 1 -- start start 2024-05-14T15:37:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.724+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb740b29f0 0x7fdb740b4f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdb78005950 tx=0x7fdb7800a300 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x560566336720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x56056633af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560566258860 con 0x5605661b8e40 2024-05-14T15:37:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5605661f77c0 con 0x560566336c60 2024-05-14T15:37:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560566254820 con 0x56056633b470 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x560566336720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x560566336720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43650/0 (socket says 172.21.15.5:43650) 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 -- 172.21.15.5:0/3475723255 learned_addr learned my addr 172.21.15.5:0/3475723255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x56056633af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb89602700 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 -- 172.21.15.5:0/3475723255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 msgr2=0x56056633b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 -- 172.21.15.5:0/3475723255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 msgr2=0x56056633af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x56056633af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 -- 172.21.15.5:0/3475723255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605661a58d0 con 0x5605661b8e40 2024-05-14T15:37:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x560566336720 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800fde0 tx=0x7fdb78012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb0862d75de673 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb6effd700 1 -- 172.21.15.5:0/3475723255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78016070 con 0x5605661b8e40 2024-05-14T15:37:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.725+0000 7fdb6effd700 1 -- 172.21.15.5:0/3475723255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb78025470 con 0x5605661b8e40 2024-05-14T15:37:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb6effd700 1 -- 172.21.15.5:0/3475723255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7801b680 con 0x5605661b8e40 2024-05-14T15:37:30.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3475723255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 msgr2=0x560566336720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x560566336720 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800fde0 tx=0x7fdb78012000 comp rx=0 tx=0).stop 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3475723255 shutdown_connections 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x560566336720 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3475723255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x56056633af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3475723255 >> 172.21.15.5:0/3475723255 conn(0x560566200600 msgr2=0x5605663314b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.726+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3475723255 shutdown_connections 2024-05-14T15:37:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.727+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3475723255 wait complete. 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.727+0000 7fdb99a79b80 1 -- start start 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.727+0000 7fdb99a79b80 1 Processor -- start 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605661b8e40 0x560566340220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566340760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 0x560566340ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560566343420 con 0x56056633b470 2024-05-14T15:37:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5605661bec30 con 0x560566336c60 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605663435a0 con 0x5605661b8e40 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb99a79b80 10 client.0 _ll_register_callbacks cb 0x5605661f0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 0x560566340ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 0x560566340ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43664/0 (socket says 172.21.15.5:43664) 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 -- 172.21.15.5:0/1333511761 learned_addr learned my addr 172.21.15.5:0/1333511761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb88e01700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605661b8e40 0x560566340220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb83fff700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566340760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 -- 172.21.15.5:0/1333511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605661b8e40 msgr2=0x560566340220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605661b8e40 0x560566340220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 -- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 msgr2=0x560566340760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566340760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 -- 172.21.15.5:0/1333511761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605661f4690 con 0x56056633b470 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb88e01700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605661b8e40 0x560566340220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb83fff700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566340760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:37:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb89602700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 0x560566340ca0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00c730 tx=0x7fdb7c0a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131c16d1e5cc63f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb63fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb63fff700 1 -- 172.21.15.5:0/1333511761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0a90a0 con 0x56056633b470 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.728+0000 7fdb63fff700 1 -- 172.21.15.5:0/1333511761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fdb7c0a9300 con 0x56056633b470 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb78025470 con 0x56056633b470 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb63fff700 1 -- 172.21.15.5:0/1333511761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb7c0ad450 con 0x56056633b470 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 10 client.18402 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56056634e090 con 0x56056633b470 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb63fff700 1 -- 172.21.15.5:0/1333511761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb78025470 con 0x56056633b470 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb63fff700 1 -- 172.21.15.5:0/1333511761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb7c012070 con 0x56056633b470 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 10 client.18402 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 10 client.18402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 20 client.18402 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663506d0 0x560566352af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 4 client.18402 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560566266240 con 0x5605663506d0 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.729+0000 7fdb88e01700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663506d0 0x560566352af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.730+0000 7fdb88e01700 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663506d0 0x560566352af0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb5056c8a1a572f7 server_cookie=ccef24aa9ce73b7c in_seq=0 out_seq=0 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.730+0000 7fdb63fff700 10 client.18402 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb63fff700 1 -- 172.21.15.5:0/1333511761 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560566266240 con 0x5605663506d0 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb63fff700 10 client.18402 handle_command_reply: tid=0 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 client.18402 shutdown 2024-05-14T15:37:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 msgr2=0x560566340ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 0x560566340ca0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00c730 tx=0x7fdb7c0a8c60 comp rx=0 tx=0).stop 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 shutdown_connections 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663506d0 0x560566352af0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56056633b470 0x560566340ca0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605661b8e40 0x560566340220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/1333511761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566340760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 >> 172.21.15.5:0/1333511761 conn(0x560566200600 msgr2=0x560566335450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:30.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb63fff700 0 client.18402 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:37:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 shutdown_connections 2024-05-14T15:37:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 1 -- 172.21.15.5:0/1333511761 wait complete. 2024-05-14T15:37:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.731+0000 7fdb99a79b80 20 client.18402 trim_cache size 0 max 16384 2024-05-14T15:37:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 Processor -- start 2024-05-14T15:37:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 -- start start 2024-05-14T15:37:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x56056632b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566331610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841b5270 con 0x5605661b8e40 2024-05-14T15:37:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.746+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841b2b00 con 0x560566336c60 2024-05-14T15:37:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841b0970 con 0x56056633b470 2024-05-14T15:37:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x56056632b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x56056632b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43676/0 (socket says 172.21.15.5:43676) 2024-05-14T15:37:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 -- 172.21.15.5:0/3290017737 learned_addr learned my addr 172.21.15.5:0/3290017737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb89602700 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566331610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 -- 172.21.15.5:0/3290017737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 msgr2=0x56056633fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 -- 172.21.15.5:0/3290017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 msgr2=0x560566331610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566331610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 -- 172.21.15.5:0/3290017737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560566337740 con 0x5605661b8e40 2024-05-14T15:37:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x56056632b7f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800fde0 tx=0x7fdb780046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c868cb03918812 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb6e7fc700 1 -- 172.21.15.5:0/3290017737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78016040 con 0x5605661b8e40 2024-05-14T15:37:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb6e7fc700 1 -- 172.21.15.5:0/3290017737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb78033e50 con 0x5605661b8e40 2024-05-14T15:37:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.747+0000 7fdb6e7fc700 1 -- 172.21.15.5:0/3290017737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780054d0 con 0x5605661b8e40 2024-05-14T15:37:30.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3290017737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 msgr2=0x56056632b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x56056632b7f0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800fde0 tx=0x7fdb780046f0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3290017737 shutdown_connections 2024-05-14T15:37:30.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x56056633fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x56056632b7f0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3290017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x560566331610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3290017737 >> 172.21.15.5:0/3290017737 conn(0x560566200600 msgr2=0x5605663401e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:30.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3290017737 shutdown_connections 2024-05-14T15:37:30.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3290017737 wait complete. 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.748+0000 7fdb99a79b80 1 -- start start 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 Processor -- start 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x5605663b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x5605663b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x560566343dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841b0fc0 con 0x5605661b8e40 2024-05-14T15:37:30.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841b2430 con 0x560566336c60 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb841b1690 con 0x56056633b470 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb99a79b80 10 client.0 _ll_register_callbacks cb 0x5605661f0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb88e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x5605663b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb88e01700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x5605663b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43686/0 (socket says 172.21.15.5:43686) 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb88e01700 1 -- 172.21.15.5:0/3467723482 learned_addr learned my addr 172.21.15.5:0/3467723482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x5605663b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.749+0000 7fdb89602700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x560566343dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb88e01700 1 -- 172.21.15.5:0/3467723482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 msgr2=0x560566343dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x560566343dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb88e01700 1 -- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 msgr2=0x5605663b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x5605663b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb88e01700 1 -- 172.21.15.5:0/3467723482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560566344440 con 0x5605661b8e40 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb88e01700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x5605663b31e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdb78025650 tx=0x7fdb78025930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6397e8fd865b1db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb63fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb63fff700 1 -- 172.21.15.5:0/3467723482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78040060 con 0x5605661b8e40 2024-05-14T15:37:30.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb63fff700 1 -- 172.21.15.5:0/3467723482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fdb780056a0 con 0x5605661b8e40 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb78033e50 con 0x5605661b8e40 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb63fff700 1 -- 172.21.15.5:0/3467723482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb78016040 con 0x5605661b8e40 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb99a79b80 10 client.18408 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.750+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605663ba830 con 0x5605661b8e40 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb63fff700 1 -- 172.21.15.5:0/3467723482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb78033e50 con 0x5605661b8e40 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb63fff700 1 -- 172.21.15.5:0/3467723482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb78030030 con 0x5605661b8e40 2024-05-14T15:37:30.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb99a79b80 10 client.18408 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb99a79b80 10 client.18408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb99a79b80 20 client.18408 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663bacc0 0x5605663bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb99a79b80 4 client.18408 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 --> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605662602e0 con 0x5605663bacc0 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.751+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663bacc0 0x5605663bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb63fff700 10 client.18408 ms_handle_connect on v2:172.21.15.144:6836/214149479 2024-05-14T15:37:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb83fff700 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663bacc0 0x5605663bd0e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bab288f353c9d05 server_cookie=47a9f0e92d682e75 in_seq=0 out_seq=0 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb63fff700 1 -- 172.21.15.5:0/3467723482 <== mds.1 v2:172.21.15.144:6836/214149479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605662602e0 con 0x5605663bacc0 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb63fff700 10 client.18408 handle_command_reply: tid=0 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 client.18408 shutdown 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 msgr2=0x5605663b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x5605663b31e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdb78025650 tx=0x7fdb78025930 comp rx=0 tx=0).stop 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 shutdown_connections 2024-05-14T15:37:30.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:6836/214149479,v1:172.21.15.144:6837/214149479] conn(0x5605663bacc0 0x5605663bd0e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56056633b470 0x560566343dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605661b8e40 0x5605663b31e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb63fff700 0 client.18408 ms_handle_reset on v2:172.21.15.144:6836/214149479 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 --2- 172.21.15.5:0/3467723482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560566336c60 0x5605663b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 >> 172.21.15.5:0/3467723482 conn(0x560566200600 msgr2=0x560566331370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 shutdown_connections 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 1 -- 172.21.15.5:0/3467723482 wait complete. 2024-05-14T15:37:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.752+0000 7fdb99a79b80 20 client.18408 trim_cache size 0 max 16384 2024-05-14T15:37:30.766 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:37:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb740b29f0 msgr2=0x7fdb740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb740b29f0 0x7fdb740b4f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdb78005950 tx=0x7fdb7800a300 comp rx=0 tx=0).stop 2024-05-14T15:37:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 msgr2=0x7fdb84077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 0x7fdb84077650 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00ada0 tx=0x7fdb7c00c330 comp rx=0 tx=0).stop 2024-05-14T15:37:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 shutdown_connections 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fdb740b29f0 0x7fdb740b4f00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb84109860 0x7fdb84077650 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb841045e0 0x7fdb84076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 --2- 172.21.15.5:0/3838058235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84108f10 0x7fdb84077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 >> 172.21.15.5:0/3838058235 conn(0x7fdb840ffed0 msgr2=0x7fdb8410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 shutdown_connections 2024-05-14T15:37:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:30.753+0000 7fdb8b065700 1 -- 172.21.15.5:0/3838058235 wait complete. 2024-05-14T15:37:31.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-14T15:37:31.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:32.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:31 smithi005 ceph-mon[31559]: pgmap v402: 129 pgs: 129 active+clean; 64 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-05-14T15:37:32.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:31 smithi144 ceph-mon[35650]: pgmap v402: 129 pgs: 129 active+clean; 64 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-05-14T15:37:32.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:31 smithi193 ceph-mon[36001]: pgmap v402: 129 pgs: 129 active+clean; 64 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-05-14T15:37:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 -- 172.21.15.5:0/2342535357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98076680 msgr2=0x7f9d98110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/2342535357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98076680 0x7f9d98110660 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009ca0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 -- 172.21.15.5:0/2342535357 shutdown_connections 2024-05-14T15:37:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/2342535357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d98110ba0 0x7f9d98113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/2342535357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d98075d30 0x7f9d98076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/2342535357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98076680 0x7f9d98110660 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.682+0000 7f9d9ff57700 1 -- 172.21.15.5:0/2342535357 >> 172.21.15.5:0/2342535357 conn(0x7f9d980fffe0 msgr2=0x7f9d98102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.683+0000 7f9d9ff57700 1 -- 172.21.15.5:0/2342535357 shutdown_connections 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.683+0000 7f9d9ff57700 1 -- 172.21.15.5:0/2342535357 wait complete. 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.683+0000 7f9d9ff57700 1 Processor -- start 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.683+0000 7f9d9ff57700 1 -- start start 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9ff57700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d98075d30 0x7f9d9819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9ff57700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d98076680 0x7f9d9819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9ff57700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 0x7f9d981a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9ff57700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981166a0 con 0x7f9d98110ba0 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9ff57700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98116520 con 0x7f9d98075d30 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9ff57700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98116820 con 0x7f9d98076680 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 0x7f9d981a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 0x7f9d981a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43704/0 (socket says 172.21.15.5:43704) 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1434586810 learned_addr learned my addr 172.21.15.5:0/1434586810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d98076680 0x7f9d9819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d98075d30 0x7f9d9819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1434586810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d98076680 msgr2=0x7f9d9819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d98076680 0x7f9d9819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1434586810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d98075d30 msgr2=0x7f9d9819d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d98075d30 0x7f9d9819d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1434586810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981a1f00 con 0x7f9d98110ba0 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.684+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 0x7f9d981a1860 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9d900094f0 tx=0x7f9d9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.685+0000 7f9d8effd700 1 -- 172.21.15.5:0/1434586810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004db0 con 0x7f9d98110ba0 2024-05-14T15:37:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.685+0000 7f9d8effd700 1 -- 172.21.15.5:0/1434586810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d90009e20 con 0x7f9d98110ba0 2024-05-14T15:37:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.685+0000 7f9d8effd700 1 -- 172.21.15.5:0/1434586810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90015400 con 0x7f9d98110ba0 2024-05-14T15:37:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.685+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a21f0 con 0x7f9d98110ba0 2024-05-14T15:37:33.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.685+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a9b10 con 0x7f9d98110ba0 2024-05-14T15:37:33.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.688+0000 7f9d8effd700 1 -- 172.21.15.5:0/1434586810 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9d90005260 con 0x7f9d98110ba0 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.689+0000 7f9d8effd700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d840b29f0 0x7f9d840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.689+0000 7f9dae96bb80 1 Processor -- start 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.689+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d840b29f0 0x7f9d840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.689+0000 7f9d8effd700 1 -- 172.21.15.5:0/1434586810 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f9d900e23f0 con 0x7f9d98110ba0 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.689+0000 7f9dae96bb80 1 -- start start 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e649e40 0x559f9e7cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f9e6e9940 con 0x559f9e7cdd80 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f9e64eda0 con 0x559f9e7c7e10 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f9e6e5900 con 0x559f9e649e40 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43706/0 (socket says 172.21.15.5:43706) 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d840b29f0 0x7f9d840b4f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005950 tx=0x7f9d9400b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 -- 172.21.15.5:0/114000019 learned_addr learned my addr 172.21.15.5:0/114000019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e649e40 0x559f9e7cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 -- 172.21.15.5:0/114000019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e649e40 msgr2=0x559f9e7cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e649e40 0x559f9e7cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 -- 172.21.15.5:0/114000019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 msgr2=0x559f9e7c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 -- 172.21.15.5:0/114000019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9e6368d0 con 0x559f9e7cdd80 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.690+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7c78d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1c9a0763170d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.691+0000 7f9d83fff700 1 -- 172.21.15.5:0/114000019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88003a60 con 0x559f9e7cdd80 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.691+0000 7f9d83fff700 1 -- 172.21.15.5:0/114000019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d88005520 con 0x559f9e7cdd80 2024-05-14T15:37:33.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.691+0000 7f9d83fff700 1 -- 172.21.15.5:0/114000019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800bd30 con 0x559f9e7cdd80 2024-05-14T15:37:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.691+0000 7f9dae96bb80 1 -- 172.21.15.5:0/114000019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 msgr2=0x559f9e7c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.691+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7c78d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009f90 comp rx=0 tx=0).stop 2024-05-14T15:37:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 -- 172.21.15.5:0/114000019 shutdown_connections 2024-05-14T15:37:33.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e649e40 0x559f9e7cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/114000019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7c78d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 -- 172.21.15.5:0/114000019 >> 172.21.15.5:0/114000019 conn(0x559f9e691630 msgr2=0x559f9e7c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 -- 172.21.15.5:0/114000019 shutdown_connections 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 -- 172.21.15.5:0/114000019 wait complete. 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 -- start start 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.692+0000 7f9dae96bb80 1 Processor -- start 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e649e40 0x559f9e7d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f9e7d4760 con 0x559f9e7cdd80 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f9e6a2ed0 con 0x559f9e649e40 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f9e7d48e0 con 0x559f9e7c7e10 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9dae96bb80 10 client.0 _ll_register_callbacks cb 0x559f9e681950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9d4f2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9d4f2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.5:54264/0 (socket says 172.21.15.5:54264) 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43708/0 (socket says 172.21.15.5:43708) 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9d4f2700 1 -- 172.21.15.5:0/1846604937 learned_addr learned my addr 172.21.15.5:0/1846604937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e649e40 0x559f9e7d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1846604937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 msgr2=0x559f9e7d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e649e40 msgr2=0x559f9e7d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e649e40 0x559f9e7d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9e4f4700 1 -- 172.21.15.5:0/1846604937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9e6a0760 con 0x559f9e7cdd80 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.693+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7d1fc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9d900df700 tx=0x7f9d900a8a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3cb3c1e38a987f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d817fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d817fa700 1 -- 172.21.15.5:0/1846604937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d900a9990 con 0x559f9e7cdd80 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d817fa700 1 -- 172.21.15.5:0/1846604937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f9d900b1c60 con 0x559f9e7cdd80 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d88005520 con 0x559f9e7cdd80 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d817fa700 1 -- 172.21.15.5:0/1846604937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d900aa380 con 0x559f9e7cdd80 2024-05-14T15:37:33.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9dae96bb80 10 client.18420 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559f9e7df3e0 con 0x559f9e7cdd80 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d817fa700 1 -- 172.21.15.5:0/1846604937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d88005520 con 0x559f9e7cdd80 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9d817fa700 1 -- 172.21.15.5:0/1846604937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9d90012070 con 0x559f9e7cdd80 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9dae96bb80 10 client.18420 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9dae96bb80 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.694+0000 7f9dae96bb80 20 client.18420 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:33.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.695+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e7e18b0 0x559f9e7e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.695+0000 7f9dae96bb80 4 client.18420 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.695+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f9e6f7320 con 0x559f9e7e18b0 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.695+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e7e18b0 0x559f9e7e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.695+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e7e18b0 0x559f9e7e3cd0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa05fb5b555e4d8c server_cookie=94766b31031772fe in_seq=0 out_seq=0 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.695+0000 7f9d817fa700 10 client.18420 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9d817fa700 1 -- 172.21.15.5:0/1846604937 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f9e6f7320 con 0x559f9e7e18b0 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9d817fa700 10 client.18420 handle_command_reply: tid=0 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 client.18420 shutdown 2024-05-14T15:37:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 msgr2=0x559f9e7d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7d1fc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9d900df700 tx=0x7f9d900a8a00 comp rx=0 tx=0).stop 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 shutdown_connections 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e7e18b0 0x559f9e7e3cd0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e7cdd80 0x559f9e7d1fc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e649e40 0x559f9e7d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9d817fa700 0 client.18420 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.696+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/1846604937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.697+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 >> 172.21.15.5:0/1846604937 conn(0x559f9e691630 msgr2=0x559f9e7c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.697+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 shutdown_connections 2024-05-14T15:37:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.697+0000 7f9dae96bb80 1 -- 172.21.15.5:0/1846604937 wait complete. 2024-05-14T15:37:33.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.697+0000 7f9dae96bb80 20 client.18420 trim_cache size 0 max 16384 2024-05-14T15:37:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 Processor -- start 2024-05-14T15:37:33.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 -- start start 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7bcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7cdd80 0x559f9e7dd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b50c0 con 0x559f9e649e40 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b2950 con 0x559f9e7cdd80 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b07c0 con 0x559f9e7c7e10 2024-05-14T15:37:33.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9d9dcf3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9d9dcf3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43718/0 (socket says 172.21.15.5:43718) 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/3935845355 learned_addr learned my addr 172.21.15.5:0/3935845355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.712+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7bcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7cdd80 0x559f9e7dd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/3935845355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 msgr2=0x559f9e7bcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7bcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/3935845355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7cdd80 msgr2=0x559f9e7dd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7cdd80 0x559f9e7dd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/3935845355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9e7c2f40 con 0x559f9e649e40 2024-05-14T15:37:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7c78d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400ba80 tx=0x7f9d94013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c1b060f3415434 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d837fe700 1 -- 172.21.15.5:0/3935845355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9402a670 con 0x559f9e649e40 2024-05-14T15:37:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d837fe700 1 -- 172.21.15.5:0/3935845355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d94030070 con 0x559f9e649e40 2024-05-14T15:37:33.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.713+0000 7f9d837fe700 1 -- 172.21.15.5:0/3935845355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401ee30 con 0x559f9e649e40 2024-05-14T15:37:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 -- 172.21.15.5:0/3935845355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 msgr2=0x559f9e7c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7c78d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400ba80 tx=0x7f9d94013000 comp rx=0 tx=0).stop 2024-05-14T15:37:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 -- 172.21.15.5:0/3935845355 shutdown_connections 2024-05-14T15:37:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7cdd80 0x559f9e7dd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7c78d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/3935845355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7c7e10 0x559f9e7bcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 -- 172.21.15.5:0/3935845355 >> 172.21.15.5:0/3935845355 conn(0x559f9e691630 msgr2=0x559f9e7cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 -- 172.21.15.5:0/3935845355 shutdown_connections 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 -- 172.21.15.5:0/3935845355 wait complete. 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 -- start start 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.714+0000 7f9dae96bb80 1 Processor -- start 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7cdd80 0x559f9e7d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b0e10 con 0x559f9e649e40 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b2280 con 0x559f9e7c7e10 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b14e0 con 0x559f9e7cdd80 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9dae96bb80 10 client.0 _ll_register_callbacks cb 0x559f9e681950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7d9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43732/0 (socket says 172.21.15.5:43732) 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/612811690 learned_addr learned my addr 172.21.15.5:0/612811690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9e4f4700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7cdd80 0x559f9e7d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/612811690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7cdd80 msgr2=0x559f9e7d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7cdd80 0x559f9e7d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 msgr2=0x559f9e7d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.715+0000 7f9d9dcf3700 1 -- 172.21.15.5:0/612811690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9e7d1920 con 0x559f9e649e40 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d9dcf3700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7d9f40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9d9402d040 tx=0x7f9d9401e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c0f4771ee045bb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d817fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-14T15:37:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d817fa700 1 -- 172.21.15.5:0/612811690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94010db0 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d817fa700 1 -- 172.21.15.5:0/612811690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f9d9402a470 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d94030070 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d817fa700 1 -- 172.21.15.5:0/612811690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d94045050 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 10 client.18426 fetch_fsmap learned FSMap version 34 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559f9e84bc60 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d817fa700 1 -- 172.21.15.5:0/612811690 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d94030070 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9d817fa700 1 -- 172.21.15.5:0/612811690 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9d9403cb80 con 0x559f9e649e40 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 10 client.18426 fetch_fsmap finished waiting for FSMap version 34 2024-05-14T15:37:33.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 10 client.18426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 20 client.18426 populate_metadata read hostname 'smithi005' 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e84c310 0x559f9e84e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 4 client.18426 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.716+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 --> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f9e6f13c0 con 0x559f9e84c310 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.717+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e84c310 0x559f9e84e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.717+0000 7f9d9d4f2700 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e84c310 0x559f9e84e730 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55f320b8424173d server_cookie=6b1d7fad8ede66c5 in_seq=0 out_seq=0 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.717+0000 7f9d817fa700 10 client.18426 ms_handle_connect on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.717+0000 7f9d817fa700 1 -- 172.21.15.5:0/612811690 <== mds.0 v2:172.21.15.144:6838/4183436774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f9e6f13c0 con 0x559f9e84c310 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.717+0000 7f9d817fa700 10 client.18426 handle_command_reply: tid=0 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 client.18426 shutdown 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 msgr2=0x559f9e7d9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7d9f40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9d9402d040 tx=0x7f9d9401e630 comp rx=0 tx=0).stop 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 shutdown_connections 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:6838/4183436774,v1:172.21.15.144:6839/4183436774] conn(0x559f9e84c310 0x559f9e84e730 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f9e7cdd80 0x559f9e7d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9d817fa700 0 client.18426 ms_handle_reset on v2:172.21.15.144:6838/4183436774 2024-05-14T15:37:33.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f9e649e40 0x559f9e7d9f40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 --2- 172.21.15.5:0/612811690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f9e7c7e10 0x559f9e7d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 >> 172.21.15.5:0/612811690 conn(0x559f9e691630 msgr2=0x559f9e7cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 shutdown_connections 2024-05-14T15:37:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 1 -- 172.21.15.5:0/612811690 wait complete. 2024-05-14T15:37:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9dae96bb80 20 client.18426 trim_cache size 0 max 16384 2024-05-14T15:37:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-14T15:37:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d840b29f0 msgr2=0x7f9d840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d840b29f0 0x7f9d840b4f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005950 tx=0x7f9d9400b410 comp rx=0 tx=0).stop 2024-05-14T15:37:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.718+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 msgr2=0x7f9d981a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 0x7f9d981a1860 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9d900094f0 tx=0x7f9d9000d3b0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 shutdown_connections 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f9d840b29f0 0x7f9d840b4f00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d98110ba0 0x7f9d981a1860 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d98075d30 0x7f9d9819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 --2- 172.21.15.5:0/1434586810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d98076680 0x7f9d9819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 >> 172.21.15.5:0/1434586810 conn(0x7f9d980fffe0 msgr2=0x7f9d98100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 shutdown_connections 2024-05-14T15:37:33.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:33.719+0000 7f9d9ff57700 1 -- 172.21.15.5:0/1434586810 wait complete. 2024-05-14T15:37:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:33 smithi144 ceph-mon[35650]: pgmap v403: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-05-14T15:37:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:33 smithi193 ceph-mon[36001]: pgmap v403: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-05-14T15:37:34.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:33 smithi005 ceph-mon[31559]: pgmap v403: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-05-14T15:37:34.487 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-14T15:37:34.493 INFO:tasks.fwd_scrub:done joining 2024-05-14T15:37:34.493 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-14T15:37:34.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T15:37:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/3986334111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf010b7f0 msgr2=0x7f6bf010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/3986334111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf010b7f0 0x7f6bf010dc10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009900 tx=0x7f6be402a510 comp rx=0 tx=0).stop 2024-05-14T15:37:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/3986334111 shutdown_connections 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/3986334111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010e1a0 0x7f6bf0110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/3986334111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf010b7f0 0x7f6bf010dc10 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/3986334111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf0104d90 0x7f6bf01051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/3986334111 >> 172.21.15.5:0/3986334111 conn(0x7f6bf0078280 msgr2=0x7f6bf00786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/3986334111 shutdown_connections 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.625+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/3986334111 wait complete. 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.626+0000 7f6bf5a8b640 1 Processor -- start 2024-05-14T15:37:34.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.626+0000 7f6bf5a8b640 1 -- start start 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 0x7f6bf01a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010b7f0 0x7f6bf01a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf010e1a0 0x7f6bf019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0113e00 con 0x7f6bf0104d90 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0113c80 con 0x7f6bf010e1a0 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0113f80 con 0x7f6bf010b7f0 2024-05-14T15:37:34.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 0x7f6bf01a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 0x7f6bf01a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43742/0 (socket says 172.21.15.5:43742) 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 -- 172.21.15.5:0/16208795 learned_addr learned my addr 172.21.15.5:0/16208795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6beeffd640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010b7f0 0x7f6bf01a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 -- 172.21.15.5:0/16208795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010b7f0 msgr2=0x7f6bf01a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010b7f0 0x7f6bf01a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 -- 172.21.15.5:0/16208795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf010e1a0 msgr2=0x7f6bf019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf010e1a0 0x7f6bf019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 -- 172.21.15.5:0/16208795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf019adb0 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6beeffd640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010b7f0 0x7f6bf01a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bef7fe640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 0x7f6bf01a0500 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9f0 tx=0x7f6be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000cd80 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be00108e0 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0018980 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf019b040 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.627+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf01a72a0 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.628+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf006a840 con 0x7f6bf0104d90 2024-05-14T15:37:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.630+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6be0010430 con 0x7f6bf0104d90 2024-05-14T15:37:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.631+0000 7f6becff9640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6bd80b29e0 0x7f6bd80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.631+0000 7f6beeffd640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6bd80b29e0 0x7f6bd80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.631+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f6be00e32d0 con 0x7f6bf0104d90 2024-05-14T15:37:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.631+0000 7f6beeffd640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6bd80b29e0 0x7f6bd80b4ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6be40096c0 tx=0x7f6be4031400 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.634+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6be00aa8d0 con 0x7f6bf0104d90 2024-05-14T15:37:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.828+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bf01a74a0 con 0x7f6bf0104d90 2024-05-14T15:37:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.830+0000 7f6becff9640 1 -- 172.21.15.5:0/16208795 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25248 (secure 0 0 0) 0x7f6be00aa1d0 con 0x7f6bf0104d90 2024-05-14T15:37:34.832 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:37:34.832 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":110,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:37:23.281832+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2413445605":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/365942426":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/4280832550":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/3326284548":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:37:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.832+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6bd80b29e0 msgr2=0x7f6bd80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.832+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6bd80b29e0 0x7f6bd80b4ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6be40096c0 tx=0x7f6be4031400 comp rx=0 tx=0).stop 2024-05-14T15:37:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.832+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 msgr2=0x7f6bf01a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.832+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 0x7f6bf01a0500 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6be000e9f0 tx=0x7f6be000eeb0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 shutdown_connections 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f6bd80b29e0 0x7f6bd80b4ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bf010b7f0 0x7f6bf01a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf010e1a0 0x7f6bf019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 --2- 172.21.15.5:0/16208795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bf0104d90 0x7f6bf01a0500 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 >> 172.21.15.5:0/16208795 conn(0x7f6bf0078280 msgr2=0x7f6bf0101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 shutdown_connections 2024-05-14T15:37:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.833+0000 7f6bf5a8b640 1 -- 172.21.15.5:0/16208795 wait complete. 2024-05-14T15:37:34.846 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T15:16:05.463462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T15:20:51.199481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T15:20:54.665379+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-14T15:21:04.523784+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-14T15:37:34.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T15:37:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 -- 172.21.15.5:0/757773445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554104390 msgr2=0x7fe554110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 --2- 172.21.15.5:0/757773445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554104390 0x7fe554110760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009900 tx=0x7fe54c02a410 comp rx=0 tx=0).stop 2024-05-14T15:37:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 -- 172.21.15.5:0/757773445 shutdown_connections 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 --2- 172.21.15.5:0/757773445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe55410a0e0 0x7fe554103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 --2- 172.21.15.5:0/757773445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554104390 0x7fe554110760 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 --2- 172.21.15.5:0/757773445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe554109c40 0x7fe5541037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 -- 172.21.15.5:0/757773445 >> 172.21.15.5:0/757773445 conn(0x7fe5540ff300 msgr2=0x7fe554101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 -- 172.21.15.5:0/757773445 shutdown_connections 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.966+0000 7fe55b338640 1 -- 172.21.15.5:0/757773445 wait complete. 2024-05-14T15:37:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 Processor -- start 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 -- start start 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe554104390 0x7fe5541a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe554109c40 0x7fe5541a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 0x7fe55419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe554113e30 con 0x7fe55410a0e0 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe554113cb0 con 0x7fe554109c40 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe55b338640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe554113fb0 con 0x7fe554104390 2024-05-14T15:37:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe5598d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 0x7fe55419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe5598d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 0x7fe55419c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43760/0 (socket says 172.21.15.5:43760) 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe5598d5640 1 -- 172.21.15.5:0/1130507037 learned_addr learned my addr 172.21.15.5:0/1130507037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe5590d4640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe554104390 0x7fe5541a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.967+0000 7fe5588d3640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe554109c40 0x7fe5541a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5598d5640 1 -- 172.21.15.5:0/1130507037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe554104390 msgr2=0x7fe5541a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5598d5640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe554104390 0x7fe5541a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5598d5640 1 -- 172.21.15.5:0/1130507037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe554109c40 msgr2=0x7fe5541a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5598d5640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe554109c40 0x7fe5541a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:34.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5598d5640 1 -- 172.21.15.5:0/1130507037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55419cf30 con 0x7fe55410a0e0 2024-05-14T15:37:34.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5598d5640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 0x7fe55419c850 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe54c004770 tx=0x7fe54c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:34.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c03f030 con 0x7fe55410a0e0 2024-05-14T15:37:34.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55419d1c0 con 0x7fe55410a0e0 2024-05-14T15:37:34.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe54c003aa0 con 0x7fe55410a0e0 2024-05-14T15:37:34.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c035070 con 0x7fe55410a0e0 2024-05-14T15:37:34.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.968+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5541a9470 con 0x7fe55410a0e0 2024-05-14T15:37:34.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.969+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe554104f20 con 0x7fe55410a0e0 2024-05-14T15:37:34.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.971+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe54c042030 con 0x7fe55410a0e0 2024-05-14T15:37:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.971+0000 7fe5527fc640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe53c0b2960 0x7fe53c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.971+0000 7fe5590d4640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe53c0b2960 0x7fe53c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.971+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fe54c103350 con 0x7fe55410a0e0 2024-05-14T15:37:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.972+0000 7fe5590d4640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe53c0b2960 0x7fe53c0b4e70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe55419db80 tx=0x7fe544009680 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:34.975+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe54c0ca300 con 0x7fe55410a0e0 2024-05-14T15:37:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.169+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe55406a290 con 0x7fe55410a0e0 2024-05-14T15:37:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.169+0000 7fe5527fc640 1 -- 172.21.15.5:0/1130507037 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fe54c0ce600 con 0x7fe55410a0e0 2024-05-14T15:37:35.171 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-14T15:37:35.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe53c0b2960 msgr2=0x7fe53c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe53c0b2960 0x7fe53c0b4e70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe55419db80 tx=0x7fe544009680 comp rx=0 tx=0).stop 2024-05-14T15:37:35.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 msgr2=0x7fe55419c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 0x7fe55419c850 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe54c004770 tx=0x7fe54c004270 comp rx=0 tx=0).stop 2024-05-14T15:37:35.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 shutdown_connections 2024-05-14T15:37:35.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fe53c0b2960 0x7fe53c0b4e70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55410a0e0 0x7fe55419c850 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe554109c40 0x7fe5541a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 --2- 172.21.15.5:0/1130507037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe554104390 0x7fe5541a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 >> 172.21.15.5:0/1130507037 conn(0x7fe5540ff300 msgr2=0x7fe554106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 shutdown_connections 2024-05-14T15:37:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.171+0000 7fe55b338640 1 -- 172.21.15.5:0/1130507037 wait complete. 2024-05-14T15:37:35.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:34 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/16208795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:37:35.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T15:37:35.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:34 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/16208795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:37:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 -- 172.21.15.5:0/1489794860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c108b60 msgr2=0x7f185c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 --2- 172.21.15.5:0/1489794860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c108b60 0x7f185c102750 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1854009ca0 tx=0x7f185402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 -- 172.21.15.5:0/1489794860 shutdown_connections 2024-05-14T15:37:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 --2- 172.21.15.5:0/1489794860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c103300 0x7f185c10f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 --2- 172.21.15.5:0/1489794860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c109000 0x7f185c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 --2- 172.21.15.5:0/1489794860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c108b60 0x7f185c102750 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 -- 172.21.15.5:0/1489794860 >> 172.21.15.5:0/1489794860 conn(0x7f185c0782b0 msgr2=0x7f185c0786f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 -- 172.21.15.5:0/1489794860 shutdown_connections 2024-05-14T15:37:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 -- 172.21.15.5:0/1489794860 wait complete. 2024-05-14T15:37:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:34 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/16208795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.302+0000 7f186144c640 1 Processor -- start 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 -- start start 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 0x7f185c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c108b60 0x7f185c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c109000 0x7f185c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f185c112d50 con 0x7f185c103300 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f185c112bd0 con 0x7f185c109000 2024-05-14T15:37:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f186144c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f185c112ed0 con 0x7f185c108b60 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 0x7f185c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 0x7f185c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43764/0 (socket says 172.21.15.5:43764) 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 -- 172.21.15.5:0/1896541340 learned_addr learned my addr 172.21.15.5:0/1896541340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185a7fc640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c108b60 0x7f185c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185b7fe640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c109000 0x7f185c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 -- 172.21.15.5:0/1896541340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c108b60 msgr2=0x7f185c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c108b60 0x7f185c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 -- 172.21.15.5:0/1896541340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c109000 msgr2=0x7f185c19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c109000 0x7f185c19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185affd640 1 -- 172.21.15.5:0/1896541340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c19e5f0 con 0x7f185c103300 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.303+0000 7f185a7fc640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c108b60 0x7f185c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.304+0000 7f185affd640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 0x7f185c199790 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1854006010 tx=0x7f18540319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.304+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854003bf0 con 0x7f185c103300 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.304+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c19e880 con 0x7f185c103300 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.304+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1854003d90 con 0x7f185c103300 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.304+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185402fb60 con 0x7f185c103300 2024-05-14T15:37:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.304+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c19edd0 con 0x7f185c103300 2024-05-14T15:37:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.305+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c06a290 con 0x7f185c103300 2024-05-14T15:37:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.307+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f185402fd00 con 0x7f185c103300 2024-05-14T15:37:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.307+0000 7f1853fff640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f18480b2960 0x7f18480b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.308+0000 7f185a7fc640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f18480b2960 0x7f18480b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.308+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7f18541040b0 con 0x7f185c103300 2024-05-14T15:37:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.308+0000 7f185a7fc640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f18480b2960 0x7f18480b4e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1844005db0 tx=0x7f1844005d40 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.311+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18540cb7c0 con 0x7f185c103300 2024-05-14T15:37:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.503+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f185c107030 con 0x7f185c103300 2024-05-14T15:37:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.503+0000 7f1853fff640 1 -- 172.21.15.5:0/1896541340 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f18540cb0c0 con 0x7f185c103300 2024-05-14T15:37:35.505 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-14T15:37:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f18480b2960 msgr2=0x7f18480b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f18480b2960 0x7f18480b4e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1844005db0 tx=0x7f1844005d40 comp rx=0 tx=0).stop 2024-05-14T15:37:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 msgr2=0x7f185c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 0x7f185c199790 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1854006010 tx=0x7f18540319f0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 shutdown_connections 2024-05-14T15:37:35.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f18480b2960 0x7f18480b4e70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f185c108b60 0x7f185c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.505+0000 7f186144c640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c109000 0x7f185c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.506+0000 7f186144c640 1 --2- 172.21.15.5:0/1896541340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f185c103300 0x7f185c199790 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.506+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 >> 172.21.15.5:0/1896541340 conn(0x7f185c0782b0 msgr2=0x7f185c104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.506+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 shutdown_connections 2024-05-14T15:37:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.506+0000 7f186144c640 1 -- 172.21.15.5:0/1896541340 wait complete. 2024-05-14T15:37:35.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T15:37:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 -- 172.21.15.5:0/1954110821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624103270 msgr2=0x7fb62410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1954110821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624103270 0x7fb62410f640 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009ca0 tx=0x7fb61c02a8c0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 -- 172.21.15.5:0/1954110821 shutdown_connections 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1954110821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb624108fc0 0x7fb624102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1954110821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624103270 0x7fb62410f640 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1954110821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb624108b20 0x7fb6241026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 -- 172.21.15.5:0/1954110821 >> 172.21.15.5:0/1954110821 conn(0x7fb624078230 msgr2=0x7fb624078670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 -- 172.21.15.5:0/1954110821 shutdown_connections 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.635+0000 7fb62bb72640 1 -- 172.21.15.5:0/1954110821 wait complete. 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 Processor -- start 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 -- start start 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb624103270 0x7fb6241a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb624108b20 0x7fb6241a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 0x7fb62419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb624113eb0 con 0x7fb624108fc0 2024-05-14T15:37:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb624113d30 con 0x7fb624103270 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62bb72640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb624114030 con 0x7fb624108b20 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62a10f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 0x7fb62419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62a10f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 0x7fb62419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43776/0 (socket says 172.21.15.5:43776) 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62a10f640 1 -- 172.21.15.5:0/1992932238 learned_addr learned my addr 172.21.15.5:0/1992932238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62990e640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb624103270 0x7fb6241a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.636+0000 7fb62910d640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb624108b20 0x7fb6241a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62a10f640 1 -- 172.21.15.5:0/1992932238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb624108b20 msgr2=0x7fb6241a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62a10f640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb624108b20 0x7fb6241a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62a10f640 1 -- 172.21.15.5:0/1992932238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb624103270 msgr2=0x7fb6241a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62a10f640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb624103270 0x7fb6241a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62a10f640 1 -- 172.21.15.5:0/1992932238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62419ae20 con 0x7fb624108fc0 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62a10f640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 0x7fb62419a6f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005cc0 tx=0x7fb61c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003bf0 con 0x7fb624108fc0 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c003d90 con 0x7fb624108fc0 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62419b050 con 0x7fb624108fc0 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c02fb20 con 0x7fb624108fc0 2024-05-14T15:37:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.637+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6241a7310 con 0x7fb624108fc0 2024-05-14T15:37:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.638+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb624103e00 con 0x7fb624108fc0 2024-05-14T15:37:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.640+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb61c004580 con 0x7fb624108fc0 2024-05-14T15:37:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.641+0000 7fb61affd640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb6100b2960 0x7fb6100b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.641+0000 7fb62990e640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb6100b2960 0x7fb6100b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.641+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fb61c104050 con 0x7fb624108fc0 2024-05-14T15:37:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.642+0000 7fb62990e640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb6100b2960 0x7fb6100b4e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb624103800 tx=0x7fb620009110 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.645+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb61c0cb760 con 0x7fb624108fc0 2024-05-14T15:37:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.837+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb62402dc30 con 0x7fb624108fc0 2024-05-14T15:37:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.838+0000 7fb61affd640 1 -- 172.21.15.5:0/1992932238 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7fb61c0098b0 con 0x7fb624108fc0 2024-05-14T15:37:35.839 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-14T15:37:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb6100b2960 msgr2=0x7fb6100b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb6100b2960 0x7fb6100b4e70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb624103800 tx=0x7fb620009110 comp rx=0 tx=0).stop 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 msgr2=0x7fb62419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 0x7fb62419a6f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005cc0 tx=0x7fb61c02f380 comp rx=0 tx=0).stop 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 shutdown_connections 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fb6100b2960 0x7fb6100b4e70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb624108b20 0x7fb6241a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb624108fc0 0x7fb62419a6f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 --2- 172.21.15.5:0/1992932238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb624103270 0x7fb6241a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 >> 172.21.15.5:0/1992932238 conn(0x7fb624078230 msgr2=0x7fb6240fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 shutdown_connections 2024-05-14T15:37:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.840+0000 7fb62bb72640 1 -- 172.21.15.5:0/1992932238 wait complete. 2024-05-14T15:37:35.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-14T15:37:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.970+0000 7fd2621c0640 1 -- 172.21.15.5:0/2166678144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c102180 msgr2=0x7fd25c102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2166678144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c102180 0x7fd25c102590 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd244009900 tx=0x7fd24402a410 comp rx=0 tx=0).stop 2024-05-14T15:37:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 -- 172.21.15.5:0/2166678144 shutdown_connections 2024-05-14T15:37:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2166678144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25c10e680 0x7fd25c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2166678144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c102180 0x7fd25c102590 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2166678144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd25c109cd0 0x7fd25c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 -- 172.21.15.5:0/2166678144 >> 172.21.15.5:0/2166678144 conn(0x7fd25c0779b0 msgr2=0x7fd25c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 -- 172.21.15.5:0/2166678144 shutdown_connections 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 -- 172.21.15.5:0/2166678144 wait complete. 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 Processor -- start 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.971+0000 7fd2621c0640 1 -- start start 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd2621c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd25c102180 0x7fd25c1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd2621c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25c109cd0 0x7fd25c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd2621c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 0x7fd25c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd2621c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c113e90 con 0x7fd25c10e680 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd2621c0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c113d10 con 0x7fd25c102180 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd2621c0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c114010 con 0x7fd25c109cd0 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 0x7fd25c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 0x7fd25c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45974/0 (socket says 172.21.15.5:45974) 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 -- 172.21.15.5:0/2026371467 learned_addr learned my addr 172.21.15.5:0/2026371467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd253fff640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25c109cd0 0x7fd25c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25b7fe640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd25c102180 0x7fd25c1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 -- 172.21.15.5:0/2026371467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25c109cd0 msgr2=0x7fd25c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25c109cd0 0x7fd25c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 -- 172.21.15.5:0/2026371467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd25c102180 msgr2=0x7fd25c1a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd25c102180 0x7fd25c1a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 -- 172.21.15.5:0/2026371467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c19adc0 con 0x7fd25c10e680 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.972+0000 7fd25bfff640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 0x7fd25c19a690 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd25400eb40 tx=0x7fd25400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.973+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254004b20 con 0x7fd25c10e680 2024-05-14T15:37:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.973+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd254004cc0 con 0x7fd25c10e680 2024-05-14T15:37:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.973+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254010720 con 0x7fd25c10e680 2024-05-14T15:37:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.973+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c19b050 con 0x7fd25c10e680 2024-05-14T15:37:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.973+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c102d50 con 0x7fd25c10e680 2024-05-14T15:37:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.974+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c069410 con 0x7fd25c10e680 2024-05-14T15:37:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.976+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fd254005560 con 0x7fd25c10e680 2024-05-14T15:37:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.977+0000 7fd2597fa640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd2480b2960 0x7fd2480b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.977+0000 7fd25b7fe640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd2480b2960 0x7fd2480b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.977+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fd2540e31a0 con 0x7fd25c10e680 2024-05-14T15:37:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.978+0000 7fd25b7fe640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd2480b2960 0x7fd2480b4e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd25c075650 tx=0x7fd24c012040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:35.981+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2540aa740 con 0x7fd25c10e680 2024-05-14T15:37:36.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.220+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd25c107bc0 con 0x7fd25c10e680 2024-05-14T15:37:36.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.221+0000 7fd2597fa640 1 -- 172.21.15.5:0/2026371467 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v110) ==== 103+0+11 (secure 0 0 0) 0x7fd2540aa040 con 0x7fd25c10e680 2024-05-14T15:37:36.222 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-14T15:37:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd2480b2960 msgr2=0x7fd2480b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd2480b2960 0x7fd2480b4e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd25c075650 tx=0x7fd24c012040 comp rx=0 tx=0).stop 2024-05-14T15:37:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 msgr2=0x7fd25c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 0x7fd25c19a690 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd25400eb40 tx=0x7fd25400c2d0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 shutdown_connections 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd2480b2960 0x7fd2480b4e70 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd25c075650 tx=0x7fd24c012040 comp rx=0 tx=0).stop 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25c109cd0 0x7fd25c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25c10e680 0x7fd25c19a690 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 --2- 172.21.15.5:0/2026371467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd25c102180 0x7fd25c1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 >> 172.21.15.5:0/2026371467 conn(0x7fd25c0779b0 msgr2=0x7fd25c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 shutdown_connections 2024-05-14T15:37:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.226+0000 7fd2621c0640 1 -- 172.21.15.5:0/2026371467 wait complete. 2024-05-14T15:37:36.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:35 smithi005 ceph-mon[31559]: pgmap v404: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 951 KiB/s wr, 64 op/s 2024-05-14T15:37:36.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1130507037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1896541340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:35 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1992932238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-14T15:37:36.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:35 smithi144 ceph-mon[35650]: pgmap v404: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 951 KiB/s wr, 64 op/s 2024-05-14T15:37:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:35 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1130507037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:35 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1896541340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:35 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1992932238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:35 smithi193 ceph-mon[36001]: pgmap v404: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 951 KiB/s wr, 64 op/s 2024-05-14T15:37:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:35 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1130507037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:35 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1896541340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:35 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1992932238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T15:37:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.363+0000 7fc59349a640 1 -- 172.21.15.5:0/2423757541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108ee0 msgr2=0x7fc58c110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.363+0000 7fc59349a640 1 --2- 172.21.15.5:0/2423757541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108ee0 0x7fc58c110680 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc584009ca0 tx=0x7fc58402a8c0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 -- 172.21.15.5:0/2423757541 shutdown_connections 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 --2- 172.21.15.5:0/2423757541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108ee0 0x7fc58c110680 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 --2- 172.21.15.5:0/2423757541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108420 0x7fc58c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 --2- 172.21.15.5:0/2423757541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c1023a0 0x7fc58c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 -- 172.21.15.5:0/2423757541 >> 172.21.15.5:0/2423757541 conn(0x7fc58c0fd970 msgr2=0x7fc58c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 -- 172.21.15.5:0/2423757541 shutdown_connections 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 -- 172.21.15.5:0/2423757541 wait complete. 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 Processor -- start 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.364+0000 7fc59349a640 1 -- start start 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc59349a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c1023a0 0x7fc58c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc59349a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 0x7fc58c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc59349a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108ee0 0x7fc58c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc59349a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c113df0 con 0x7fc58c108420 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc59349a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c113c70 con 0x7fc58c108ee0 2024-05-14T15:37:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc59349a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c113f70 con 0x7fc58c1023a0 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 0x7fc58c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 0x7fc58c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45994/0 (socket says 172.21.15.5:45994) 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 -- 172.21.15.5:0/764217762 learned_addr learned my addr 172.21.15.5:0/764217762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc591236640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c1023a0 0x7fc58c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc591a37640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108ee0 0x7fc58c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 -- 172.21.15.5:0/764217762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c1023a0 msgr2=0x7fc58c19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c1023a0 0x7fc58c19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 -- 172.21.15.5:0/764217762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108ee0 msgr2=0x7fc58c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108ee0 0x7fc58c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 -- 172.21.15.5:0/764217762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c19f650 con 0x7fc58c108420 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc591a37640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108ee0 0x7fc58c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.365+0000 7fc590a35640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 0x7fc58c19ad50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00e9c0 tx=0x7fc57c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.366+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00cd80 con 0x7fc58c108420 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.366+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c0054d0 con 0x7fc58c108420 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.366+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c19f940 con 0x7fc58c108420 2024-05-14T15:37:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.366+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c010430 con 0x7fc58c108420 2024-05-14T15:37:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.366+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c1a72c0 con 0x7fc58c108420 2024-05-14T15:37:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.367+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c06a2c0 con 0x7fc58c108420 2024-05-14T15:37:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.368+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc57c004fb0 con 0x7fc58c108420 2024-05-14T15:37:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.370+0000 7fc5827fc640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5780b2960 0x7fc5780b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:37:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.370+0000 7fc591236640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5780b2960 0x7fc5780b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:37:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.370+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9518+0+0 (secure 0 0 0) 0x7fc57c0e3040 con 0x7fc58c108420 2024-05-14T15:37:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.370+0000 7fc591236640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5780b2960 0x7fc5780b4e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc588009bd0 tx=0x7fc588005f90 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:37:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.373+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc57c054df0 con 0x7fc58c108420 2024-05-14T15:37:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.615+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc58c10b770 con 0x7fc58c108420 2024-05-14T15:37:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.625+0000 7fc5827fc640 1 -- 172.21.15.5:0/764217762 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816165 (secure 0 0 0) 0x7fc57c0aa640 con 0x7fc58c108420 2024-05-14T15:37:36.638 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:37:36.638 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:37:36.638 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2024-05-14T15:37:36.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.638 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 15246, 2024-05-14T15:37:36.639 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-05-14T15:37:36.639 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-05-14T15:37:36.639 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:36.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:36.639 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2405629464 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2405629464 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:36.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/2405629464", 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-05-14T15:24:50.611408+0000", 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24965, 2024-05-14T15:37:36.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.642 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T15:37:36.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.646 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-05-14T15:37:36.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-05-14T15:37:36.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-05-14T15:37:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-05-14T15:37:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-05-14T15:37:36.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-05-14T15:37:36.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-05-14T15:37:36.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T15:37:36.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-14T15:37:36.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-14T15:37:36.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:36.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-05-14T15:37:36.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-05-14T15:37:36.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T15:37:36.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.683 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.688 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-05-14T15:37:36.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-05-14T15:37:36.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-05-14T15:37:36.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:36.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-14T15:37:36.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-05-14T15:37:36.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-05-14T15:37:36.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-05-14T15:37:36.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-05-14T15:37:36.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.719 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.721 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-05-14T15:37:36.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.726 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T15:37:36.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-05-14T15:37:36.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T15:37:36.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.738 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-14T15:37:36.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.743 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-05-14T15:37:36.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-05-14T15:37:36.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T15:37:36.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-05-14T15:37:36.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:36.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-05-14T15:37:36.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T15:37:36.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.759 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-05-14T15:37:36.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-05-14T15:37:36.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.762 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-05-14T15:37:36.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.767 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T15:37:36.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.772 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.778 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.782 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:36.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-05-14T15:37:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-05-14T15:37:36.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T15:37:36.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T15:37:36.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.794 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.799 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.804 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-05-14T15:37:36.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-05-14T15:37:36.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T15:37:36.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.817 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.819 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.819 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.819 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.819 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.823 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-05-14T15:37:36.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T15:37:36.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T15:37:36.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.844 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T15:37:36.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.849 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T15:37:36.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T15:37:36.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-05-14T15:37:36.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.867 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T15:37:36.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-05-14T15:37:36.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T15:37:36.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T15:37:36.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.884 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T15:37:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T15:37:36.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T15:37:36.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.905 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:36.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-05-14T15:37:36.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-05-14T15:37:36.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-05-14T15:37:36.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-05-14T15:37:36.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-14T15:37:36.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.920 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.921 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.924 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-05-14T15:37:36.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-05-14T15:37:36.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-14T15:37:36.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-05-14T15:37:36.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-14T15:37:36.945 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.949 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-05-14T15:37:36.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-05-14T15:37:36.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-05-14T15:37:36.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T15:37:36.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:36.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T15:37:36.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-14T15:37:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-05-14T15:37:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.984 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:36.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-05-14T15:37:36.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-05-14T15:37:36.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:36.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:36.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:36.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:36.999 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.003 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.015 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.020 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.026 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.030 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.036 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T15:37:37.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.043 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.048 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-05-14T15:37:37.050 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-05-14T15:37:37.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.052 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.053 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-05-14T15:37:37.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.064 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-05-14T15:37:37.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-05-14T15:37:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-05-14T15:37:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.082 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.088 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.089 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.097 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.102 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.107 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.111 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.119 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.123 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-05-14T15:37:37.128 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.130 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.131 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.134 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.140 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.141 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-05-14T15:37:37.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T15:37:37.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.153 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-14T15:37:37.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.158 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.159 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.162 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-05-14T15:37:37.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.175 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.180 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-05-14T15:37:37.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-05-14T15:37:37.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:37.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-05-14T15:37:37.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.195 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-14T15:37:37.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-14T15:37:37.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-14T15:37:37.199 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.203 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.204 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-05-14T15:37:37.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.210 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.219 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.220 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.223 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.228 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.232 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.234 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.238 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.248 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-05-14T15:37:37.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.258 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.261 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-05-14T15:37:37.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-05-14T15:37:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:37.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.285 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smb", 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.289 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.290 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.293 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: "update_orchestration": { 2024-05-14T15:37:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "update_orchestration", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T15:37:37.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.304 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.305 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.316 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.320 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.327 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.332 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-05-14T15:37:37.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T15:37:37.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-05-14T15:37:37.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-14T15:37:37.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.371 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.374 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.389 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.391 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.391 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.391 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.391 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.391 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.391 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.395 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.400 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.406 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.407 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-05-14T15:37:37.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.415 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.417 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.423 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.428 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T15:37:37.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24979, 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-05-14T15:37:37.439 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-05-14T15:37:37.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.456 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-05-14T15:37:37.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-05-14T15:37:37.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.469 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-14T15:37:37.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-05-14T15:37:37.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-05-14T15:37:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T15:37:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T15:37:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T15:37:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-05-14T15:37:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-05-14T15:37:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-14T15:37:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T15:37:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-05-14T15:37:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T15:37:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-05-14T15:37:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T15:37:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-05-14T15:37:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-05-14T15:37:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-05-14T15:37:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-05-14T15:37:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-05-14T15:37:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-05-14T15:37:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T15:37:37.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.577 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T15:37:37.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T15:37:37.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-05-14T15:37:37.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-05-14T15:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-05-14T15:37:37.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-05-14T15:37:37.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T15:37:37.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.598 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-05-14T15:37:37.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-05-14T15:37:37.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.604 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-05-14T15:37:37.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-05-14T15:37:37.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-05-14T15:37:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-05-14T15:37:37.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-05-14T15:37:37.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.614 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-05-14T15:37:37.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-05-14T15:37:37.627 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T15:37:37.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.633 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-05-14T15:37:37.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T15:37:37.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.636 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T15:37:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.641 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T15:37:37.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T15:37:37.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T15:37:37.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.655 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.657 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.663 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T15:37:37.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T15:37:37.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.665 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T15:37:37.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.669 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T15:37:37.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T15:37:37.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T15:37:37.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T15:37:37.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T15:37:37.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-05-14T15:37:37.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.696 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.698 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.700 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T15:37:37.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T15:37:37.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T15:37:37.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T15:37:37.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-05-14T15:37:37.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-05-14T15:37:37.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.718 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-05-14T15:37:37.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.723 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.724 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.724 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.728 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.729 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-05-14T15:37:37.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-05-14T15:37:37.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-14T15:37:37.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.745 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-14T15:37:37.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-05-14T15:37:37.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-14T15:37:37.750 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.750 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.750 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-05-14T15:37:37.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-05-14T15:37:37.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-05-14T15:37:37.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.752 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.753 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T15:37:37.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-05-14T15:37:37.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.776 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.780 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-05-14T15:37:37.790 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-05-14T15:37:37.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T15:37:37.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.799 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.800 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-05-14T15:37:37.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-05-14T15:37:37.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.812 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-05-14T15:37:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-05-14T15:37:37.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-05-14T15:37:37.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.818 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.824 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.831 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.832 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.834 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.836 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-05-14T15:37:37.842 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.842 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.842 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.842 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-05-14T15:37:37.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.847 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-05-14T15:37:37.859 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.859 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.859 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.860 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-05-14T15:37:37.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-05-14T15:37:37.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-05-14T15:37:37.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-05-14T15:37:37.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T15:37:37.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.863 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.865 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.866 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.866 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.866 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.867 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.867 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.867 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.869 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-05-14T15:37:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-05-14T15:37:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-05-14T15:37:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T15:37:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-05-14T15:37:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-05-14T15:37:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-05-14T15:37:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-05-14T15:37:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-05-14T15:37:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-05-14T15:37:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-05-14T15:37:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-05-14T15:37:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-05-14T15:37:37.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-05-14T15:37:37.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.897 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.899 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.903 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.903 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.910 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.913 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.935 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.936 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.939 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.953 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-05-14T15:37:37.958 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.970 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.974 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:37.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.987 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.990 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:37.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:37.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:37.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:37.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:37.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-05-14T15:37:37.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:37.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-05-14T15:37:38.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.006 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.010 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.020 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.025 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.028 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-14T15:37:38.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-14T15:37:38.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.030 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.036 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-05-14T15:37:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.048 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-05-14T15:37:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T15:37:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-05-14T15:37:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-05-14T15:37:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-05-14T15:37:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-05-14T15:37:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-05-14T15:37:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-05-14T15:37:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-05-14T15:37:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-05-14T15:37:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-05-14T15:37:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-05-14T15:37:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smb", 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "update_orchestration": { 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "update_orchestration", 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-05-14T15:37:38.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-05-14T15:37:38.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.148 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.149 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.151 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.155 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.159 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.166 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.170 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-05-14T15:37:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T15:37:38.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:38.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.180 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.184 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-05-14T15:37:38.189 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.190 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.190 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.190 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-05-14T15:37:38.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-05-14T15:37:38.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-05-14T15:37:38.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T15:37:38.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-05-14T15:37:38.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.217 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.223 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-05-14T15:37:38.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.229 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-05-14T15:37:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.249 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-05-14T15:37:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T15:37:38.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.257 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-05-14T15:37:38.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-05-14T15:37:38.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:38.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-05-14T15:37:38.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.261 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.264 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.269 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.270 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.272 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-05-14T15:37:38.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-05-14T15:37:38.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-05-14T15:37:38.280 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.281 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:38.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.284 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-05-14T15:37:38.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-05-14T15:37:38.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-05-14T15:37:38.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-05-14T15:37:38.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-05-14T15:37:38.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-05-14T15:37:38.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-05-14T15:37:38.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.303 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-05-14T15:37:38.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-05-14T15:37:38.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T15:37:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.306 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-05-14T15:37:38.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-05-14T15:37:38.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-05-14T15:37:38.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.309 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-05-14T15:37:38.314 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.314 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.314 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.315 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-05-14T15:37:38.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-05-14T15:37:38.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T15:37:38.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.319 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-05-14T15:37:38.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-05-14T15:37:38.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-05-14T15:37:38.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T15:37:38.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T15:37:38.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.322 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-05-14T15:37:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-05-14T15:37:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-14T15:37:38.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-14T15:37:38.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T15:37:38.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T15:37:38.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.326 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-05-14T15:37:38.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-05-14T15:37:38.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:38.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-05-14T15:37:38.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:38.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-05-14T15:37:38.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T15:37:38.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.329 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-05-14T15:37:38.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-05-14T15:37:38.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T15:37:38.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T15:37:38.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T15:37:38.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.333 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-05-14T15:37:38.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-05-14T15:37:38.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-05-14T15:37:38.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-05-14T15:37:38.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-05-14T15:37:38.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-05-14T15:37:38.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T15:37:38.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-05-14T15:37:38.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.338 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-05-14T15:37:38.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.341 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-14T15:37:38.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.351 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.359 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T15:37:38.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.361 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-14T15:37:38.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.365 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-05-14T15:37:38.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-05-14T15:37:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-05-14T15:37:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T15:37:38.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.372 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.373 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-05-14T15:37:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-05-14T15:37:38.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T15:37:38.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T15:37:38.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-05-14T15:37:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-05-14T15:37:38.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-05-14T15:37:38.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-05-14T15:37:38.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-05-14T15:37:38.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T15:37:38.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.402 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-05-14T15:37:38.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:38.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-05-14T15:37:38.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T15:37:38.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-05-14T15:37:38.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-05-14T15:37:38.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T15:37:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-05-14T15:37:38.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.426 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T15:37:38.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T15:37:38.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-05-14T15:37:38.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T15:37:38.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.444 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.446 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.451 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.456 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-05-14T15:37:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-05-14T15:37:38.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.470 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T15:37:38.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.472 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.477 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-05-14T15:37:38.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.483 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T15:37:38.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T15:37:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T15:37:38.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:38.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:38.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.517 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.522 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.527 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T15:37:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T15:37:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T15:37:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T15:37:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T15:37:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-05-14T15:37:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T15:37:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T15:37:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T15:37:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T15:37:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-05-14T15:37:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T15:37:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T15:37:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T15:37:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T15:37:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T15:37:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-05-14T15:37:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-05-14T15:37:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-05-14T15:37:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-14T15:37:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-14T15:37:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-05-14T15:37:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-05-14T15:37:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-05-14T15:37:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-05-14T15:37:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-05-14T15:37:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T15:37:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.636 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.637 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.640 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-05-14T15:37:38.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-05-14T15:37:38.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-05-14T15:37:38.651 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-05-14T15:37:38.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T15:37:38.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:38.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.659 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-05-14T15:37:38.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-05-14T15:37:38.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.670 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T15:37:38.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-05-14T15:37:38.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T15:37:38.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.677 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.679 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.689 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.694 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.695 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-05-14T15:37:38.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-05-14T15:37:38.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-05-14T15:37:38.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-05-14T15:37:38.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-05-14T15:37:38.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-05-14T15:37:38.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T15:37:38.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.705 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.705 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.706 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.706 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.706 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.706 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.707 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.708 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.710 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.711 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.716 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.719 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.723 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-05-14T15:37:38.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-05-14T15:37:38.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-05-14T15:37:38.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-05-14T15:37:38.737 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.737 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.737 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.739 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.743 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.750 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.755 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.760 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.762 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.766 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T15:37:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-05-14T15:37:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-05-14T15:37:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-05-14T15:37:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T15:37:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T15:37:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T15:37:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-05-14T15:37:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-05-14T15:37:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-05-14T15:37:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-05-14T15:37:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-14T15:37:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-05-14T15:37:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.899 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-05-14T15:37:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-05-14T15:37:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smb", 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "update_orchestration": { 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "update_orchestration", 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.988 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:38.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:38.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:38.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:38.999 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:39.004 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.006 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.006 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.006 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.006 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.007 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.007 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.007 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.008 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:39.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:39.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.010 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:39.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.011 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-05-14T15:37:39.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:39.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.022 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:39.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:39.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.027 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:39.032 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-05-14T15:37:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-05-14T15:37:39.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.042 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-05-14T15:37:39.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-05-14T15:37:39.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-05-14T15:37:39.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-05-14T15:37:39.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.050 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-05-14T15:37:39.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-05-14T15:37:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-05-14T15:37:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T15:37:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.077 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.081 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-14T15:37:39.083 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.089 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-05-14T15:37:39.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-05-14T15:37:39.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.096 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T15:37:39.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-05-14T15:37:39.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.102 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-05-14T15:37:39.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-05-14T15:37:39.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-05-14T15:37:39.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.106 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-05-14T15:37:39.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-05-14T15:37:39.110 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "sqlite3_killpoint": { 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sqlite3_killpoint", 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "dev", 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-05-14T15:37:39.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-05-14T15:37:39.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T15:37:39.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-14T15:37:39.120 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-14T15:37:39.121 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-14T15:37:39.122 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-14T15:37:39.123 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-14T15:37:39.124 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-05-14T15:37:39.125 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-05-14T15:37:39.126 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 91, 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-05-14T15:37:39.127 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2956379788 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1568876391 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3466381949 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1122501888 2024-05-14T15:37:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-14T15:37:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2262942382 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-14T15:37:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-05-14T15:37:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1107051232 2024-05-14T15:37:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-14T15:37:39.134 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:37:39.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:37 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2026371467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T15:37:39.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:37 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/764217762' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T15:37:39.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:38 smithi193 ceph-mon[36001]: pgmap v405: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.0 MiB/s wr, 68 op/s 2024-05-14T15:37:39.136 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:37 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2026371467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T15:37:39.136 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:37 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/764217762' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T15:37:39.137 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:38 smithi144 ceph-mon[35650]: pgmap v405: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.0 MiB/s wr, 68 op/s 2024-05-14T15:37:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5780b2960 msgr2=0x7fc5780b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5780b2960 0x7fc5780b4e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc588009bd0 tx=0x7fc588005f90 comp rx=0 tx=0).stop 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 msgr2=0x7fc58c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 0x7fc58c19ad50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00e9c0 tx=0x7fc57c00ee80 comp rx=0 tx=0).stop 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 shutdown_connections 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fc5780b2960 0x7fc5780b4e70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc58c108ee0 0x7fc58c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc58c108420 0x7fc58c19ad50 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 --2- 172.21.15.5:0/764217762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c1023a0 0x7fc58c19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:37:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 >> 172.21.15.5:0/764217762 conn(0x7fc58c0fd970 msgr2=0x7fc58c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:37:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 shutdown_connections 2024-05-14T15:37:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:37:36.630+0000 7fc59349a640 1 -- 172.21.15.5:0/764217762 wait complete. 2024-05-14T15:37:39.147 DEBUG:tasks.check_counter:Getting stats from a 2024-05-14T15:37:39.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-14T15:37:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:37 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2026371467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-14T15:37:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:37 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/764217762' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T15:37:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:38 smithi005 ceph-mon[31559]: pgmap v405: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.0 MiB/s wr, 68 op/s 2024-05-14T15:37:39.410 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:40.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:40 smithi005 ceph-mon[31559]: pgmap v406: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.0 MiB/s wr, 67 op/s 2024-05-14T15:37:40.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:40 smithi144 ceph-mon[35650]: pgmap v406: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.0 MiB/s wr, 67 op/s 2024-05-14T15:37:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:40 smithi193 ceph-mon[36001]: pgmap v406: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.0 MiB/s wr, 67 op/s 2024-05-14T15:37:41.133 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:37:41.133 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:41.133 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 285, 2024-05-14T15:37:41.133 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 274, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1728960, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 168544, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.056478030, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.016884253, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.045022583, 2024-05-14T15:37:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.005324081, 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 273, 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.014380631, 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000052676 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:41.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1728960, 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 168544 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 6, 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 211, 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3360, 2024-05-14T15:37:41.136 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 194688, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.015203099, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.010204373, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002568774, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 209, 2024-05-14T15:37:41.137 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.012598246, 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000060278 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.138 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 194688 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-05-14T15:37:41.139 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-05-14T15:37:41.140 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-14T15:37:41.140 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000268355, 2024-05-14T15:37:41.140 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T15:37:41.140 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000196068, 2024-05-14T15:37:41.140 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:41.140 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:41.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-14T15:37:41.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:41.143 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:41.144 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:41.145 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:41.146 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:37:41.147 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-05-14T15:37:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-14T15:37:41.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 287, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1688718, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 287, 2024-05-14T15:37:41.150 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 287, 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1688718, 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:41.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:41.152 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:37:42.041 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-14T15:37:42.042 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-14T15:37:42.042 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-14T15:37:42.042 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-14T15:37:42.042 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-14T15:37:42.042 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-14T15:37:42.042 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-14T15:37:42.042 DEBUG:tasks.check_counter:Getting stats from d 2024-05-14T15:37:42.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-14T15:37:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:42 smithi144 ceph-mon[35650]: pgmap v407: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 576 KiB/s wr, 45 op/s 2024-05-14T15:37:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:42 smithi193 ceph-mon[36001]: pgmap v407: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 576 KiB/s wr, 45 op/s 2024-05-14T15:37:42.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:42 smithi005 ceph-mon[31559]: pgmap v407: 129 pgs: 129 active+clean; 60 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 576 KiB/s wr, 45 op/s 2024-05-14T15:37:43.948 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:37:43.948 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 292, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 281, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1730864, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 173024, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.057249415, 2024-05-14T15:37:43.949 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.016389789, 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.046486620, 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.005454007, 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 280, 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.010715029, 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000038267 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.950 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1730864, 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 173024 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:43.951 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-05-14T15:37:43.952 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 214, 2024-05-14T15:37:43.952 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-05-14T15:37:43.952 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 195120, 2024-05-14T15:37:43.952 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-14T15:37:43.952 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-05-14T15:37:43.952 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.016045918, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.011028711, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002386005, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 211, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.012591449, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000059675 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.953 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 195120 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.954 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3040, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000729071, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000041525, 2024-05-14T15:37:43.955 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000657807, 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000811181, 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000270393 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:43.956 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-05-14T15:37:43.957 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:43.958 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:43.959 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:43.960 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148754, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-14T15:37:43.962 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:43.963 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-14T15:37:43.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 294, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1689628, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:43.965 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 294, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 294, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1689628, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:43.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:43.967 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:43.967 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:37:44.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:44 smithi005 ceph-mon[31559]: pgmap v408: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 762 KiB/s wr, 53 op/s 2024-05-14T15:37:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:44 smithi144 ceph-mon[35650]: pgmap v408: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 762 KiB/s wr, 53 op/s 2024-05-14T15:37:44.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:44 smithi193 ceph-mon[36001]: pgmap v408: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 762 KiB/s wr, 53 op/s 2024-05-14T15:37:44.681 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-14T15:37:44.681 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-14T15:37:44.681 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-14T15:37:44.682 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-14T15:37:44.682 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-14T15:37:44.682 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-14T15:37:44.682 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-14T15:37:44.682 DEBUG:tasks.check_counter:Getting stats from g 2024-05-14T15:37:44.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-14T15:37:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:46.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:46 smithi005 ceph-mon[31559]: pgmap v409: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 517 KiB/s wr, 25 op/s 2024-05-14T15:37:46.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:46 smithi144 ceph-mon[35650]: pgmap v409: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 517 KiB/s wr, 25 op/s 2024-05-14T15:37:46.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:46 smithi193 ceph-mon[36001]: pgmap v409: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 517 KiB/s wr, 25 op/s 2024-05-14T15:37:46.712 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:37:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001386485, 2024-05-14T15:37:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000192607, 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T15:37:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 288, 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 277, 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1729776, 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 170448, 2024-05-14T15:37:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.052956992, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.015707974, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.043156962, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.005683975, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 276, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.010456842, 2024-05-14T15:37:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000037887 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1729776, 2024-05-14T15:37:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 170448 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 6, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 213, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3360, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 195040, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-05-14T15:37:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.016313283, 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.010962476, 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002727806, 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 211, 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.018810199, 2024-05-14T15:37:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000089147 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-05-14T15:37:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 195040 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148754, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-14T15:37:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-14T15:37:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 290, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1689108, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 290, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 290, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1689108, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:46.732 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:37:47.408 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-14T15:37:47.408 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-14T15:37:47.408 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-14T15:37:47.408 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-14T15:37:47.408 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-14T15:37:47.409 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-14T15:37:47.409 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-14T15:37:47.409 DEBUG:tasks.check_counter:Getting stats from j 2024-05-14T15:37:47.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-14T15:37:47.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:37:48.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:48 smithi005 ceph-mon[31559]: pgmap v410: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 530 KiB/s wr, 26 op/s 2024-05-14T15:37:48.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:48 smithi144 ceph-mon[35650]: pgmap v410: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 530 KiB/s wr, 26 op/s 2024-05-14T15:37:48.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:48 smithi193 ceph-mon[36001]: pgmap v410: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 530 KiB/s wr, 26 op/s 2024-05-14T15:37:49.377 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-14T15:37:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-05-14T15:37:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-05-14T15:37:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-05-14T15:37:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001420603, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000280173, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 216, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 195472, 2024-05-14T15:37:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.016001093, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.011344155, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002162793, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 213, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.012541975, 2024-05-14T15:37:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000058882 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T15:37:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 195472 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 298, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 285, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1734448, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 174752, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-05-14T15:37:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.059894781, 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.016830658, 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.050271305, 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.005452082, 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 285, 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.012271337, 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000043057 2024-05-14T15:37:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1734448, 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 174752 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-05-14T15:37:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 296, 2024-05-14T15:37:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1689888, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 296, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 296, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1689888, 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-05-14T15:37:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-05-14T15:37:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-05-14T15:37:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-05-14T15:37:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-14T15:37:49.395 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-14T15:37:50.079 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-14T15:37:50.079 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-14T15:37:50.079 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-14T15:37:50.079 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-14T15:37:50.079 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-14T15:37:50.080 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-14T15:37:50.080 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-14T15:37:50.080 DEBUG:tasks.check_counter:Getting stats from b 2024-05-14T15:37:50.080 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-14T15:37:50.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:50 smithi144 ceph-mon[35650]: pgmap v411: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 395 KiB/s wr, 21 op/s 2024-05-14T15:37:50.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:50 smithi193 ceph-mon[36001]: pgmap v411: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 395 KiB/s wr, 21 op/s 2024-05-14T15:37:50.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:50 smithi005 ceph-mon[31559]: pgmap v411: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 395 KiB/s wr, 21 op/s 2024-05-14T15:37:50.452 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:37:52.217 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:52 smithi144 ceph-mon[35650]: pgmap v412: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 198 KiB/s wr, 11 op/s 2024-05-14T15:37:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:52 smithi193 ceph-mon[36001]: pgmap v412: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 198 KiB/s wr, 11 op/s 2024-05-14T15:37:52.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:52 smithi005 ceph-mon[31559]: pgmap v412: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 198 KiB/s wr, 11 op/s 2024-05-14T15:37:52.522 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 194121, 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 317960, 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 38779401, 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 479290729, 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 31, 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-05-14T15:37:52.523 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 17.468952384, 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 8.354027159, 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 7.000231826, 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 2.284431277, 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 317962, 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 41.096123639, 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000129248 2024-05-14T15:37:52.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 373869, 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.065696430, 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000175 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 420192, 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 110576 2024-05-14T15:37:52.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 186254, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 311001, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 42421389, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 373418788, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 16, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-05-14T15:37:52.526 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 13.457884332, 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 5.251095974, 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 6.054164826, 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 2.080146437, 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 311003, 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 37.650690298, 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000121062 2024-05-14T15:37:52.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 371348, 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.066173797, 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000178 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1837696, 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 579424 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.528 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 446981, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 614550, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 133451048, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 652591448, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 239, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 9, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 22.957945258, 2024-05-14T15:37:52.529 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 10.881679774, 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 12.697531373, 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 2.352113897, 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 614541, 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 57.178989050, 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000093043 2024-05-14T15:37:52.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 516108, 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.103645606, 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000200 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 147936, 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 206416 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.531 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 128304, 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 124.687016290, 2024-05-14T15:37:52.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000971809 2024-05-14T15:37:52.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.533 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-05-14T15:37:52.533 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-14T15:37:52.533 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-14T15:37:52.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 67756, 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 9.912867766, 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000146302 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 303089, 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 303089, 2024-05-14T15:37:52.534 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 303089, 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 427.596112829, 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001410793 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-05-14T15:37:52.535 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 2610, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 18, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 26, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 2165, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 889, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 1276, 2024-05-14T15:37:52.536 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 895, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 51, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 51, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 2, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 451225, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 275687, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-05-14T15:37:52.537 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 3134, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-05-14T15:37:52.538 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 1663, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 67, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 21176629, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 6, 2024-05-14T15:37:52.539 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 1663, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 380, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 20, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 360, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 1283, 2024-05-14T15:37:52.540 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 8165, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 129360, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 67003, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 30861, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 14216, 2024-05-14T15:37:52.541 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 211222 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 26, 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-05-14T15:37:52.542 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 52, 2024-05-14T15:37:52.543 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-05-14T15:37:52.543 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-05-14T15:37:52.543 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-05-14T15:37:52.543 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 39202, 2024-05-14T15:37:52.543 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 39202, 2024-05-14T15:37:52.543 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-05-14T15:37:52.544 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-05-14T15:37:52.545 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-05-14T15:37:52.545 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 20, 2024-05-14T15:37:52.545 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-05-14T15:37:52.545 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 44, 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-05-14T15:37:52.546 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 183297, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 180796, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 180796, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 2501, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 3459, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 290, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 286, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 286, 2024-05-14T15:37:52.547 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 4, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 1, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 6, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 681410806, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 689590379, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-05-14T15:37:52.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 63191, 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 217.756672510, 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.003446007 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 743, 2024-05-14T15:37:52.549 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 39965, 2024-05-14T15:37:52.550 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 39222, 2024-05-14T15:37:52.550 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 46, 2024-05-14T15:37:52.550 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 202, 2024-05-14T15:37:52.550 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 156, 2024-05-14T15:37:52.550 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 2165, 2024-05-14T15:37:52.550 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 86579, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 84414, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 51, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 43565, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 43514, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 153816, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 231824 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.551 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 451585, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 1, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 303089, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 5379, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 2, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 39269, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 57.278637192, 2024-05-14T15:37:52.552 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001458622 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 398, 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.458527283, 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001152078 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-05-14T15:37:52.553 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.555 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-05-14T15:37:52.555 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 94508, 2024-05-14T15:37:52.555 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 49.025396931, 2024-05-14T15:37:52.555 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000518743 2024-05-14T15:37:52.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.555 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-05-14T15:37:52.556 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.556 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.556 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.556 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-05-14T15:37:52.556 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.557 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.558 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-05-14T15:37:52.558 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.558 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.558 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.559 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-05-14T15:37:52.559 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.559 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.560 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.560 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-05-14T15:37:52.560 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 12, 2024-05-14T15:37:52.560 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.004550163, 2024-05-14T15:37:52.560 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000379180 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 20, 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.027493934, 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001374696 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-05-14T15:37:52.561 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 12, 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.102302728, 2024-05-14T15:37:52.562 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.008525227 2024-05-14T15:37:52.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.563 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-05-14T15:37:52.563 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-05-14T15:37:52.563 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000261856, 2024-05-14T15:37:52.563 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000261856 2024-05-14T15:37:52.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 120186, 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 246.463086629, 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002050680 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-05-14T15:37:52.564 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 8204, 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 15.428396216, 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001880594 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-05-14T15:37:52.566 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 6, 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.145925987, 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.024320997 2024-05-14T15:37:52.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-05-14T15:37:52.568 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001256489, 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000628244 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1260, 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.840082478, 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000666732 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-05-14T15:37:52.570 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 9, 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.026526528, 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002947392 2024-05-14T15:37:52.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-05-14T15:37:52.572 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 39202, 2024-05-14T15:37:52.573 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 57.793668415, 2024-05-14T15:37:52.573 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001474253 2024-05-14T15:37:52.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.573 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-05-14T15:37:52.573 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.574 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 2, 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 7, 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 5, 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 42088, 2024-05-14T15:37:52.575 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 21044, 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 746, 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:52.576 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:52.577 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:52.578 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:52.578 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:52.578 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:52.578 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:52.578 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:52.579 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:52.580 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 311693, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 432, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 176, 2024-05-14T15:37:52.581 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 2, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:52.582 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 9592, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 184, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 3481027, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 32087, 2024-05-14T15:37:52.583 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-05-14T15:37:52.584 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 185647, 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 719225597, 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 32, 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 185647, 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 185647, 2024-05-14T15:37:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 6038.428678613, 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.032526400 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 185647, 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 245862 2024-05-14T15:37:52.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 185647, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 2790, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 182857, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 2408, 2024-05-14T15:37:52.587 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 35085, 2024-05-14T15:37:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1107, 2024-05-14T15:37:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 64278, 2024-05-14T15:37:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 729, 2024-05-14T15:37:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-05-14T15:37:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-05-14T15:37:52.588 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 18, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 79380, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1683, 2024-05-14T15:37:52.589 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 52897, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-05-14T15:37:52.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-05-14T15:37:52.591 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-05-14T15:37:52.592 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 110, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 31, 2024-05-14T15:37:52.593 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 12, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 12, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 6182, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 2095 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.594 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-05-14T15:37:52.595 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 16, 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 8, 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 78404, 2024-05-14T15:37:52.596 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 39202, 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:52.597 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 827352, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 146351159, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 827352, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 827352, 2024-05-14T15:37:52.598 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 146351159, 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.599 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:52.600 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 185647, 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 723503159, 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 185647, 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 723503159, 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.601 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:52.602 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 185647, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 185647, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 185647, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 185647, 2024-05-14T15:37:52.603 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-05-14T15:37:52.604 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-14T15:37:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 39202, 2024-05-14T15:37:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 4277562, 2024-05-14T15:37:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:52.605 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 39202, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1087, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 4277562, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.606 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55f4bc868680": { 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:52.607 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 183297, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 685396075, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 183297, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 63191, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 685396075, 2024-05-14T15:37:52.608 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.609 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:52.609 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-14T15:37:53.544 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-14T15:37:53.544 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-14T15:37:53.544 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=26 2024-05-14T15:37:53.545 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-14T15:37:53.545 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-14T15:37:53.545 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-05-14T15:37:53.545 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-05-14T15:37:53.545 DEBUG:tasks.check_counter:Getting stats from e 2024-05-14T15:37:53.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-14T15:37:53.794 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:37:54.234 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:54 smithi144 ceph-mon[35650]: pgmap v413: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 198 KiB/s wr, 14 op/s 2024-05-14T15:37:54.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:54 smithi193 ceph-mon[36001]: pgmap v413: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 198 KiB/s wr, 14 op/s 2024-05-14T15:37:54.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:54 smithi005 ceph-mon[31559]: pgmap v413: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 198 KiB/s wr, 14 op/s 2024-05-14T15:37:55.810 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-14T15:37:55.810 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:55.810 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 299, 2024-05-14T15:37:55.810 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 301, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 42035343, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 95092, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.042396473, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.007903257, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.024101438, 2024-05-14T15:37:55.811 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.007374897, 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 299, 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.015878639, 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000053105 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 306, 2024-05-14T15:37:55.813 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 303, 2024-05-14T15:37:55.814 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 35569858, 2024-05-14T15:37:55.814 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 106392, 2024-05-14T15:37:55.814 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-05-14T15:37:55.814 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-05-14T15:37:55.814 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.045927916, 2024-05-14T15:37:55.814 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.005845411, 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.033098612, 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.006752958, 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 303, 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012166202, 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000040152 2024-05-14T15:37:55.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 186672, 2024-05-14T15:37:55.816 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 28416 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 121, 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 154, 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 29170870, 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 182161, 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-05-14T15:37:55.817 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.025743251, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.003518925, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.018687218, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.002589134, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 152, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.005090436, 2024-05-14T15:37:55.818 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033489 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 144000 2024-05-14T15:37:55.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-14T15:37:55.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1068, 2024-05-14T15:37:55.821 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.050749794, 2024-05-14T15:37:55.821 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000047518 2024-05-14T15:37:55.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.821 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.822 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 5414, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 3015, 2024-05-14T15:37:55.824 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 2399, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 1107, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 2, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-05-14T15:37:55.825 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-05-14T15:37:55.826 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 67, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 21176629, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 6, 2024-05-14T15:37:55.827 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-05-14T15:37:55.828 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-05-14T15:37:55.829 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-05-14T15:37:55.830 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-05-14T15:37:55.831 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-05-14T15:37:55.832 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-05-14T15:37:55.833 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-05-14T15:37:55.833 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-05-14T15:37:55.833 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-05-14T15:37:55.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.833 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 23856, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-05-14T15:37:55.834 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 4, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 8, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-05-14T15:37:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 681410806, 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 689625572, 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 689590379, 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 23856 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 186, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 5304, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 5118, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 44, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 44, 2024-05-14T15:37:55.837 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 5414, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 5414, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 125912, 2024-05-14T15:37:55.838 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215416 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-05-14T15:37:55.839 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.841 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.842 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.843 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.844 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.844 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-05-14T15:37:55.844 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.844 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.844 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.845 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.846 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.847 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.849 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-05-14T15:37:55.850 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.851 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.852 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-05-14T15:37:55.852 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.852 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.852 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.853 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.854 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.855 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.856 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.856 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-05-14T15:37:55.856 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.856 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.856 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-05-14T15:37:55.857 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.859 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-05-14T15:37:55.859 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.859 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.859 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.859 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 2, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 0, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-05-14T15:37:55.861 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 2203, 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:55.862 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:55.864 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:55.864 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:55.864 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:55.864 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:55.864 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:55.864 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:55.865 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:55.866 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:55.866 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:55.866 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:55.866 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:55.866 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:55.866 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 232774, 2024-05-14T15:37:55.867 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 332, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:55.868 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 9592, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 184, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:55.869 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 4428562, 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 27912, 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-14T15:37:55.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 664, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 26, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 4, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 664, 2024-05-14T15:37:55.871 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 664, 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.877986876, 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001322269 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 664, 2024-05-14T15:37:55.872 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 665 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 664, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 664, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 371, 2024-05-14T15:37:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-05-14T15:37:55.874 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 290, 2024-05-14T15:37:55.874 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-05-14T15:37:55.874 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-05-14T15:37:55.874 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-05-14T15:37:55.874 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-05-14T15:37:55.874 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 2, 2024-05-14T15:37:55.875 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-05-14T15:37:55.876 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-05-14T15:37:55.876 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-05-14T15:37:55.876 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-05-14T15:37:55.876 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-05-14T15:37:55.876 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-05-14T15:37:55.876 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-05-14T15:37:55.877 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-05-14T15:37:55.878 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-05-14T15:37:55.878 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-05-14T15:37:55.878 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-05-14T15:37:55.878 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-05-14T15:37:55.878 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-05-14T15:37:55.878 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-05-14T15:37:55.879 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 110, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 5, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 9, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 9, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-05-14T15:37:55.880 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-05-14T15:37:55.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T15:37:55.882 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:55.883 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 722, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 106705148, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 722, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:55.884 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 722, 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 106705148, 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-05-14T15:37:55.885 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 664, 2024-05-14T15:37:55.886 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 106401228, 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 664, 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 106401228, 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:55.888 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 664, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 664, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 664, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 664, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.889 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:55.890 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:55.891 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5647ddfd4dd0": { 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-14T15:37:55.892 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-14T15:37:55.893 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:55.894 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-14T15:37:56.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:56 smithi144 ceph-mon[35650]: pgmap v414: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T15:37:56.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:56 smithi193 ceph-mon[36001]: pgmap v414: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T15:37:56.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:56 smithi005 ceph-mon[31559]: pgmap v414: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T15:37:56.592 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-14T15:37:56.592 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-14T15:37:56.592 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-14T15:37:56.592 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-14T15:37:56.593 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-14T15:37:56.593 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-14T15:37:56.593 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T15:37:56.593 DEBUG:tasks.check_counter:Getting stats from h 2024-05-14T15:37:56.594 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-14T15:37:56.831 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:37:58.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:37:58 smithi144 ceph-mon[35650]: pgmap v415: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-05-14T15:37:58.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:37:58 smithi193 ceph-mon[36001]: pgmap v415: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-05-14T15:37:58.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:37:58 smithi005 ceph-mon[31559]: pgmap v415: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-05-14T15:37:58.716 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-14T15:37:58.716 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 249408, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 124821, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 115641148, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 20260889, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 13, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 10.839283088, 2024-05-14T15:37:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 4.621644297, 2024-05-14T15:37:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 8.696114159, 2024-05-14T15:37:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.004366436, 2024-05-14T15:37:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 124819, 2024-05-14T15:37:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.676417489, 2024-05-14T15:37:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000013430 2024-05-14T15:37:58.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.719 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:58.719 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 373971, 2024-05-14T15:37:58.719 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.052815107, 2024-05-14T15:37:58.719 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000141 2024-05-14T15:37:58.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.720 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-05-14T15:37:58.720 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 6592 2024-05-14T15:37:58.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.720 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:37:58.720 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 26387, 2024-05-14T15:37:58.720 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 26463, 2024-05-14T15:37:58.721 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 6914797, 2024-05-14T15:37:58.721 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 2855879, 2024-05-14T15:37:58.721 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 56, 2024-05-14T15:37:58.721 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-05-14T15:37:58.721 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 1.197223939, 2024-05-14T15:37:58.722 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.501942473, 2024-05-14T15:37:58.722 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.964670875, 2024-05-14T15:37:58.722 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000235640, 2024-05-14T15:37:58.722 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:58.722 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 26460, 2024-05-14T15:37:58.722 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.374612521, 2024-05-14T15:37:58.723 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000014157 2024-05-14T15:37:58.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.723 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:58.723 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 26411, 2024-05-14T15:37:58.724 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012079928, 2024-05-14T15:37:58.724 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000457 2024-05-14T15:37:58.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.724 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 4784, 2024-05-14T15:37:58.725 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 159008 2024-05-14T15:37:58.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.725 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:37:58.725 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 379, 2024-05-14T15:37:58.725 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 509, 2024-05-14T15:37:58.726 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1824331, 2024-05-14T15:37:58.726 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 564484, 2024-05-14T15:37:58.726 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-05-14T15:37:58.726 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-05-14T15:37:58.726 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.063242853, 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.020668313, 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.050171771, 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.006315224, 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 509, 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.039352884, 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000077314 2024-05-14T15:37:58.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1821520, 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 554560 2024-05-14T15:37:58.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 45, 2024-05-14T15:37:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002014198, 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000044759 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-05-14T15:37:58.730 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000010156, 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000005078 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 26314, 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 4, 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4, 2024-05-14T15:37:58.731 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.003207258, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000801814 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 26310, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-05-14T15:37:58.732 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 29, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 7, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 22, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 19, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-05-14T15:37:58.733 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 2, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 26342, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 20, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 26310, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 12, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-05-14T15:37:58.734 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-05-14T15:37:58.735 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 89, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 21253413, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 6, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T15:37:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-05-14T15:37:58.737 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-05-14T15:37:58.738 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 38, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 25, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 26, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-05-14T15:37:58.739 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-05-14T15:37:58.740 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-05-14T15:37:58.741 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-05-14T15:37:58.741 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-05-14T15:37:58.741 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-05-14T15:37:58.741 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-05-14T15:37:58.741 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T15:37:58.741 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-05-14T15:37:58.742 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 10, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 10, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 2, 2024-05-14T15:37:58.743 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4197187, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-05-14T15:37:58.744 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 9, 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.018611744, 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002067971 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-05-14T15:37:58.745 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 32, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 32, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 45, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 49, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 4, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 29, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 29, 2024-05-14T15:37:58.746 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 4, 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 4, 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 123836, 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223632 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-05-14T15:37:58.747 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 26317, 2024-05-14T15:37:58.748 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-05-14T15:37:58.748 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 26314, 2024-05-14T15:37:58.748 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 32, 2024-05-14T15:37:58.748 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 1, 2024-05-14T15:37:58.748 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-05-14T15:37:58.748 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.749 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.749 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.749 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-05-14T15:37:58.749 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-05-14T15:37:58.749 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002478760, 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001239380 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-05-14T15:37:58.751 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000728498, 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000364249 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.753 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.754 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.755 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.756 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-05-14T15:37:58.756 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-05-14T15:37:58.758 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.759 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.760 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.761 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.762 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.762 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-05-14T15:37:58.762 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.762 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.762 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.763 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-05-14T15:37:58.763 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.763 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.763 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.764 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-05-14T15:37:58.765 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.766 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.767 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.768 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.769 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.769 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-05-14T15:37:58.769 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 2, 2024-05-14T15:37:58.769 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 5, 2024-05-14T15:37:58.769 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 3, 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 3, 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 1, 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 697, 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-05-14T15:37:58.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:37:58.771 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:37:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:37:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:37:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:37:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:37:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:37:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:37:58.773 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:37:58.774 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 259690, 2024-05-14T15:37:58.775 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 121, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:37:58.776 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 9592, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 184, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 156124, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 796, 2024-05-14T15:37:58.777 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-05-14T15:37:58.778 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 50, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 13600, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 33, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 50, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 50, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.268995722, 2024-05-14T15:37:58.779 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.005379914 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 50, 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 74 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.780 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 50, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 0, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 50, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-05-14T15:37:58.781 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 0, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 9, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 10, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-05-14T15:37:58.782 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 6, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 0, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-05-14T15:37:58.783 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-05-14T15:37:58.784 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-05-14T15:37:58.785 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-05-14T15:37:58.786 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 110, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 22, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 9, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 14, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 5, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 17, 2024-05-14T15:37:58.787 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-05-14T15:37:58.788 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 2 2024-05-14T15:37:58.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.788 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-05-14T15:37:58.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-05-14T15:37:58.790 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 276170, 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 99923426, 2024-05-14T15:37:58.791 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 276170, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 276170, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 99923426, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.792 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.793 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.793 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-05-14T15:37:58.793 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:58.793 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 50, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 13600, 2024-05-14T15:37:58.794 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 50, 2024-05-14T15:37:58.795 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 13600, 2024-05-14T15:37:58.795 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:58.795 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.795 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.795 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:58.796 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 50, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 50, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 50, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 50, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.797 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-05-14T15:37:58.798 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:58.799 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5636e94ac5b0": { 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:37:58.800 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-05-14T15:37:58.801 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:37:58.801 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 10, 2024-05-14T15:37:58.801 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 2883, 2024-05-14T15:37:58.801 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:37:58.801 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 10, 2024-05-14T15:37:58.801 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 9, 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2883, 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:37:58.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.803 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:37:58.803 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-14T15:37:59.531 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-14T15:37:59.531 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-14T15:37:59.531 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-14T15:37:59.531 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=38 2024-05-14T15:37:59.531 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-14T15:37:59.532 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-14T15:37:59.532 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T15:37:59.532 DEBUG:tasks.check_counter:Getting stats from k 2024-05-14T15:37:59.532 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-14T15:37:59.769 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T15:38:00.157 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:00 smithi144 ceph-mon[35650]: pgmap v416: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-14T15:38:00.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:00 smithi193 ceph-mon[36001]: pgmap v416: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-14T15:38:00.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:00 smithi005 ceph-mon[31559]: pgmap v416: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-14T15:38:01.613 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 0, 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2, 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 0, 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 0, 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-05-14T15:38:01.614 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 0, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.001043717, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.000246453, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:38:01.615 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T15:38:01.616 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 219, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 320, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 196000, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-05-14T15:38:01.617 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.012143516, 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008164804, 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.001759025, 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 216, 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.018741009, 2024-05-14T15:38:01.618 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000086763 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 196000 2024-05-14T15:38:01.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 302, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 289, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1735536, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 177312, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 2, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-05-14T15:38:01.620 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.051898920, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.014631568, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.042932778, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.006260122, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 289, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012093637, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000041846 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.621 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1735536, 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 177312 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.622 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 0, 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:38:01.623 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:38:01.624 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:38:01.625 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:38:01.626 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:38:01.627 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 148771, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 32, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:38:01.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 928, 2024-05-14T15:38:01.629 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 2, 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-05-14T15:38:01.630 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 300, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1690408, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 300, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-05-14T15:38:01.631 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 300, 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1690408, 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-05-14T15:38:01.632 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-05-14T15:38:02.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:02 smithi144 ceph-mon[35650]: pgmap v417: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-05-14T15:38:02.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:02 smithi193 ceph-mon[36001]: pgmap v417: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-05-14T15:38:02.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:02 smithi005 ceph-mon[31559]: pgmap v417: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-05-14T15:38:02.334 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-14T15:38:02.334 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-14T15:38:02.334 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-14T15:38:02.334 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-14T15:38:02.334 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-14T15:38:02.334 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-14T15:38:02.335 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-14T15:38:02.335 DEBUG:tasks.check_counter:Getting stats from c 2024-05-14T15:38:02.335 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-14T15:38:02.584 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:38:04.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:04 smithi144 ceph-mon[35650]: pgmap v418: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-05-14T15:38:04.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:04 smithi193 ceph-mon[36001]: pgmap v418: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-05-14T15:38:04.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:04 smithi005 ceph-mon[31559]: pgmap v418: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-05-14T15:38:04.468 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T15:38:04.468 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:38:04.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2800, 2024-05-14T15:38:04.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2802, 2024-05-14T15:38:04.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 794596, 2024-05-14T15:38:04.468 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 890407, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.209411048, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.066639945, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.051584137, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.075348223, 2024-05-14T15:38:04.469 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2800, 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.087429562, 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000031224 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.470 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 8, 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 220, 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3916, 2024-05-14T15:38:04.471 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 525450, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.011639818, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.007299912, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.002340987, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000048726, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 218, 2024-05-14T15:38:04.472 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.017694846, 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000081169 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-05-14T15:38:04.473 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 524752 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 333, 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 312, 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1802796, 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 179966, 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-05-14T15:38:04.474 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.055456623, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.015993988, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.045686694, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.006347921, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 311, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.014254682, 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045834 2024-05-14T15:38:04.475 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1802528, 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 179648 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.476 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4716, 2024-05-14T15:38:04.477 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.121908186, 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000025849 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.478 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.479 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-05-14T15:38:04.480 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-05-14T15:38:04.481 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-05-14T15:38:04.482 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T15:38:04.483 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T15:38:04.484 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T15:38:04.485 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T15:38:04.486 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T15:38:04.487 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T15:38:04.488 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-05-14T15:38:04.489 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 10, 2024-05-14T15:38:04.490 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-05-14T15:38:04.491 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4197187, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 4196879, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.492 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 10 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 3, 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 3, 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 1, 2024-05-14T15:38:04.493 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 1, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-05-14T15:38:04.494 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 124024, 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215440 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-05-14T15:38:04.495 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.496 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.497 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.498 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.499 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.500 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T15:38:04.502 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.503 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.503 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.503 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.506 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T15:38:04.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.508 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T15:38:04.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.511 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.512 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T15:38:04.513 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 5, 2024-05-14T15:38:04.514 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 3, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 5063, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T15:38:04.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:38:04.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:38:04.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T15:38:04.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:38:04.519 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 182081, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 48, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T15:38:04.520 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9592, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 184, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T15:38:04.521 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 8512, 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 13, 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T15:38:04.522 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 2803, 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 2803, 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T15:38:04.523 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2803, 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.027969071, 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000723499 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2803, 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2804 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.524 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 2803, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 2803, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1826, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 975, 2024-05-14T15:38:04.525 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-05-14T15:38:04.526 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T15:38:04.527 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T15:38:04.528 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T15:38:04.529 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 110, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T15:38:04.530 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 22, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 4, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 5, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.531 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-05-14T15:38:04.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-05-14T15:38:04.533 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3137, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2298997, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:04.534 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3137, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3137, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2298997, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.535 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:04.536 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2803, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2888, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2803, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2888, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.537 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:04.538 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2803, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2803, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2803, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2803, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.539 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:04.540 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x555e809b88f0": { 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:04.542 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:04.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:04.544 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:04.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:04.544 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.544 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:04.544 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T15:38:05.295 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-14T15:38:05.295 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-14T15:38:05.295 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-14T15:38:05.296 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-14T15:38:05.296 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-05-14T15:38:05.296 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-14T15:38:05.296 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T15:38:05.296 DEBUG:tasks.check_counter:Getting stats from f 2024-05-14T15:38:05.296 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-14T15:38:05.541 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:38:05.547 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:05 smithi144 ceph-mon[35650]: pgmap v419: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-14T15:38:05.553 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:05 smithi193 ceph-mon[36001]: pgmap v419: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-14T15:38:05.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:05 smithi005 ceph-mon[31559]: pgmap v419: 129 pgs: 129 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-14T15:38:07.270 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:07 smithi193 ceph-mon[36001]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T15:38:07.270 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:07 smithi193 ceph-mon[36001]: pgmap v421: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:07.502 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T15:38:07.502 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 980, 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 980, 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 261130, 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 310880, 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-05-14T15:38:07.503 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.080058147, 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.024020576, 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.021945572, 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.025592101, 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 979, 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.033212143, 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000033924 2024-05-14T15:38:07.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.505 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 798, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1014, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 196158, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 779405, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.078725520, 2024-05-14T15:38:07.506 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.025276485, 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.018072827, 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.027944536, 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1011, 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.041802058, 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000041347 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:07.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 526272 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:38:07.508 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1302, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1281, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2129360, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 488288, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 3, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.155799942, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.047403085, 2024-05-14T15:38:07.509 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.073589121, 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.038976416, 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1280, 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.057589825, 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000044992 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:07.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1804112, 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 180464 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T15:38:07.511 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4609, 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.127826247, 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000027734 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.512 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.513 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.514 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-05-14T15:38:07.515 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-05-14T15:38:07.516 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T15:38:07.517 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-05-14T15:38:07.518 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T15:38:07.519 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-05-14T15:38:07.520 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-05-14T15:38:07.521 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T15:38:07.522 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-05-14T15:38:07.523 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-05-14T15:38:07.524 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 6, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-05-14T15:38:07.525 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4196123, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 4195982, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.526 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 6 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 3, 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 3, 2024-05-14T15:38:07.527 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 1, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 1, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 0, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 0, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 0, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-05-14T15:38:07.528 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 118868, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215440 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-05-14T15:38:07.529 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T15:38:07.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.531 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.532 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T15:38:07.533 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.534 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.535 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.536 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T15:38:07.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.537 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.537 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.541 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.541 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T15:38:07.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.541 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.541 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T15:38:07.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.543 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.544 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.545 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T15:38:07.546 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.547 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.548 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.549 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 3, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-05-14T15:38:07.550 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2931, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:38:07.551 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:38:07.552 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:38:07.553 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:38:07.554 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:38:07.555 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 190234, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 55, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T15:38:07.556 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9952, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 193, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 8512, 2024-05-14T15:38:07.557 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 13, 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T15:38:07.558 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 2740, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 2740, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2740, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.249965707, 2024-05-14T15:38:07.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000821155 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2740, 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2741 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 2740, 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 2740, 2024-05-14T15:38:07.560 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1762, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 976, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T15:38:07.561 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-05-14T15:38:07.562 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T15:38:07.563 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T15:38:07.564 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T15:38:07.565 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 111, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 5, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 6, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-05-14T15:38:07.566 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.567 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-05-14T15:38:07.568 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3076, 2024-05-14T15:38:07.569 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2289725, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3076, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3076, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2289725, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.570 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:07.571 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2740, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1824, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2740, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1824, 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:07.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T15:38:07.573 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2740, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2740, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2740, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2740, 2024-05-14T15:38:07.574 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:07.575 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T15:38:07.576 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5634c18aa8f0": { 2024-05-14T15:38:07.577 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:07.578 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:07.579 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:07.579 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T15:38:07.579 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T15:38:07.579 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:07.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:07.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:07.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:07.580 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.580 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:07.580 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T15:38:07.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:07 smithi005 ceph-mon[31559]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T15:38:07.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:07 smithi005 ceph-mon[31559]: pgmap v421: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:07.581 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:07 smithi144 ceph-mon[35650]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T15:38:07.581 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:07 smithi144 ceph-mon[35650]: pgmap v421: 129 pgs: 5 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 110 active+clean; 59 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:08.683 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-14T15:38:08.683 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-14T15:38:08.683 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-14T15:38:08.684 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-14T15:38:08.684 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-05-14T15:38:08.684 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-14T15:38:08.684 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T15:38:08.684 DEBUG:tasks.check_counter:Getting stats from i 2024-05-14T15:38:08.684 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-14T15:38:09.080 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:38:10.256 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:09 smithi193 ceph-mon[36001]: pgmap v422: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 59 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:10.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:09 smithi144 ceph-mon[35650]: pgmap v422: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 59 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:10.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:09 smithi005 ceph-mon[31559]: pgmap v422: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 59 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:11.146 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T15:38:11.147 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:38:11.147 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 425, 2024-05-14T15:38:11.147 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 402, 2024-05-14T15:38:11.147 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1827112, 2024-05-14T15:38:11.147 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 671191, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 50, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.073945673, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.018428028, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.058133191, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.006577359, 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:11.148 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 401, 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.021027813, 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000052438 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 48, 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000006240, 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000130 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1817392, 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 185600 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 26369, 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 26623, 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 6915053, 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 2760977, 2024-05-14T15:38:11.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 1.580059634, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.664688265, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 1.323679014, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000341419, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 26620, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.427953141, 2024-05-14T15:38:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000016076 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 26384, 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.014990852, 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000568 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 525920 2024-05-14T15:38:11.152 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 249420, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 124832, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 115652352, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 20242037, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 11, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-05-14T15:38:11.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 11.192928410, 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 4.294988103, 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 9.771442902, 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.005068977, 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 124831, 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.777129831, 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000014236 2024-05-14T15:38:11.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 374016, 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.053244128, 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000142 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 6496, 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 15040 2024-05-14T15:38:11.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 39, 2024-05-14T15:38:11.156 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001932126, 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000049541 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-05-14T15:38:11.157 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000006028, 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000003014 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 26346, 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 5, 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-05-14T15:38:11.158 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001021337, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000204267 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 26341, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-05-14T15:38:11.159 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 29, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 7, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 22, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 19, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 2, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 3, 2024-05-14T15:38:11.160 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 2, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 26371, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 19, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 26339, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 12, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-05-14T15:38:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-05-14T15:38:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 89, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 21253413, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 6, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-05-14T15:38:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T15:38:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 38, 2024-05-14T15:38:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 25, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 26, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-05-14T15:38:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-05-14T15:38:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-05-14T15:38:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 6, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 6, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-05-14T15:38:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-05-14T15:38:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4196123, 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.009613495, 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001922699 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-05-14T15:38:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 32, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 32, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 45, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 49, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 4, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 29, 2024-05-14T15:38:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 29, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 3, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 3, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 124540, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223632 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.173 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 26346, 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 26346, 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 25, 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.174 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000721061, 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000240353 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.175 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.176 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.177 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-05-14T15:38:11.177 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.177 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.177 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.177 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.177 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000300276, 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000150138 2024-05-14T15:38:11.178 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.179 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.180 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-05-14T15:38:11.181 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.181 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.181 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.181 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.182 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.183 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.184 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.185 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-05-14T15:38:11.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.187 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.190 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.191 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-05-14T15:38:11.192 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-05-14T15:38:11.193 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-05-14T15:38:11.194 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-05-14T15:38:11.194 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-05-14T15:38:11.194 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-05-14T15:38:11.194 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 4, 2024-05-14T15:38:11.194 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 1, 2024-05-14T15:38:11.194 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 756, 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:38:11.195 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:38:11.196 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:38:11.197 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T15:38:11.198 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 267920, 2024-05-14T15:38:11.199 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 125, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:38:11.200 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9952, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 193, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 157164, 2024-05-14T15:38:11.201 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 794, 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-05-14T15:38:11.202 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 44, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 12512, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 44, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 44, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.228996530, 2024-05-14T15:38:11.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005204466 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 44, 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 67 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 44, 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 0, 2024-05-14T15:38:11.204 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 44, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 0, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 5, 2024-05-14T15:38:11.205 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 9, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 6, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-05-14T15:38:11.206 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-05-14T15:38:11.207 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-05-14T15:38:11.208 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-05-14T15:38:11.209 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 111, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 23, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 8, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 11, 2024-05-14T15:38:11.210 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 3, 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 16, 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 1 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-05-14T15:38:11.211 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-05-14T15:38:11.212 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:38:11.213 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 276210, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 99934868, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 276210, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:11.214 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 276210, 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 99934868, 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.215 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 44, 2024-05-14T15:38:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 12512, 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 44, 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12512, 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.217 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:11.218 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 44, 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 44, 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 44, 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 44, 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-05-14T15:38:11.220 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.221 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55d716d025b0": { 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 6, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1819, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:11.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 6, 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5, 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1819, 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:11.223 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:11.224 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:11.224 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.224 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:11.224 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T15:38:11.934 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-14T15:38:11.934 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-14T15:38:11.935 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-14T15:38:11.935 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=38 2024-05-14T15:38:11.935 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-14T15:38:11.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-14T15:38:11.935 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T15:38:11.935 DEBUG:tasks.check_counter:Getting stats from l 2024-05-14T15:38:11.935 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-14T15:38:12.170 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T15:38:12.292 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:11 smithi193 ceph-mon[36001]: pgmap v423: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 59 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:12.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:11 smithi144 ceph-mon[35650]: pgmap v423: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 59 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:12.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:11 smithi005 ceph-mon[31559]: pgmap v423: 129 pgs: 19 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 55 active+clean; 59 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-05-14T15:38:13.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:13 smithi193 ceph-mon[36001]: pgmap v424: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-05-14T15:38:13.988 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:13 smithi193 ceph-mon[36001]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T15:38:14.132 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4, 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 4, 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3040, 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 448, 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-05-14T15:38:14.133 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 0, 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.001515777, 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.000039294, 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.000641325, 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-05-14T15:38:14.134 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000778802, 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000259600 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-14T15:38:14.135 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 303, 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 292, 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1734480, 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 179760, 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 2, 2024-05-14T15:38:14.136 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 1, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.057296192, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.016646039, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.046004878, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.006608845, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 291, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.012135776, 2024-05-14T15:38:14.137 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000041703 2024-05-14T15:38:14.138 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.138 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:14.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:14.138 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:14.138 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1734480, 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 179760 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2, 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 222, 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 320, 2024-05-14T15:38:14.139 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 196528, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.015359253, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.008788473, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.004162080, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T15:38:14.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 219, 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.017039446, 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000077805 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 196528 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 0, 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-05-14T15:38:14.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-05-14T15:38:14.143 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-05-14T15:38:14.144 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T15:38:14.145 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-05-14T15:38:14.146 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 149745, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 31, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-05-14T15:38:14.147 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 0, 2024-05-14T15:38:14.148 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 928, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 2, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-05-14T15:38:14.149 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 305, 2024-05-14T15:38:14.150 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1691695, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 305, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 305, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1691695, 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-05-14T15:38:14.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-05-14T15:38:14.152 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-05-14T15:38:14.152 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-05-14T15:38:14.152 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:14.152 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-05-14T15:38:14.152 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-05-14T15:38:14.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:13 smithi144 ceph-mon[35650]: pgmap v424: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-05-14T15:38:14.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:13 smithi144 ceph-mon[35650]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T15:38:14.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:13 smithi005 ceph-mon[31559]: pgmap v424: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-05-14T15:38:14.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:13 smithi005 ceph-mon[31559]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T15:38:14.787 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-14T15:38:14.787 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-14T15:38:14.788 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-14T15:38:14.788 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-14T15:38:14.788 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-14T15:38:14.788 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-14T15:38:14.788 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-14T15:38:14.788 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_e2b755a64f0d72176b57299d02e880cf0620d780/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-05-14T15:38:14.797 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-14T15:38:14.807 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T15:38:14.808 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:38:14.808 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T15:38:14.832 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:38:14.832 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T15:38:14.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T15:38:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:16.053 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:15 smithi193 ceph-mon[36001]: pgmap v426: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-05-14T15:38:16.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:15 smithi144 ceph-mon[35650]: pgmap v426: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-05-14T15:38:16.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:15 smithi005 ceph-mon[31559]: pgmap v426: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-05-14T15:38:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.241+0000 7f7fca022700 1 -- 172.21.15.5:0/2780567486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 msgr2=0x7f7fc41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.241+0000 7f7fca022700 1 --2- 172.21.15.5:0/2780567486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc41100c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b0d0 tx=0x7f7fbc00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.241+0000 7f7fca022700 1 -- 172.21.15.5:0/2780567486 shutdown_connections 2024-05-14T15:38:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.241+0000 7f7fca022700 1 --2- 172.21.15.5:0/2780567486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc41100c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.241+0000 7f7fca022700 1 --2- 172.21.15.5:0/2780567486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc41034e0 0x7f7fc41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.241+0000 7f7fca022700 1 --2- 172.21.15.5:0/2780567486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fc4107e10 0x7f7fc4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.242+0000 7f7fca022700 1 -- 172.21.15.5:0/2780567486 >> 172.21.15.5:0/2780567486 conn(0x7f7fc40fef30 msgr2=0x7f7fc4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.242+0000 7f7fca022700 1 -- 172.21.15.5:0/2780567486 shutdown_connections 2024-05-14T15:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.242+0000 7f7fca022700 1 -- 172.21.15.5:0/2780567486 wait complete. 2024-05-14T15:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.242+0000 7f7fca022700 1 Processor -- start 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 -- start start 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc41034e0 0x7f7fc40ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fc4107e10 0x7f7fc40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113390 con 0x7f7fc4108760 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113210 con 0x7f7fc41034e0 2024-05-14T15:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fca022700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4113510 con 0x7f7fc4107e10 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc40ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43014/0 (socket says 172.21.15.5:43014) 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 -- 172.21.15.5:0/3362076853 learned_addr learned my addr 172.21.15.5:0/3362076853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc37fe700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc41034e0 0x7f7fc40ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc2ffd700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fc4107e10 0x7f7fc40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 -- 172.21.15.5:0/3362076853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fc4107e10 msgr2=0x7f7fc40ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fc4107e10 0x7f7fc40ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 -- 172.21.15.5:0/3362076853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc41034e0 msgr2=0x7f7fc40ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc41034e0 0x7f7fc40ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.243+0000 7f7fc3fff700 1 -- 172.21.15.5:0/3362076853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc41a9150 con 0x7f7fc4108760 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.244+0000 7f7fc3fff700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc40ffe90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0095c0 tx=0x7f7fbc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.244+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc01f070 con 0x7f7fc4108760 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.244+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc41a93e0 con 0x7f7fc4108760 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.244+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fbc007e20 con 0x7f7fc4108760 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.244+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc01b4a0 con 0x7f7fc4108760 2024-05-14T15:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.244+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a9840 con 0x7f7fc4108760 2024-05-14T15:38:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.245+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc40512c0 con 0x7f7fc4108760 2024-05-14T15:38:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.247+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7fbc022070 con 0x7f7fc4108760 2024-05-14T15:38:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.247+0000 7f7fc0ff9700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7fac0b29f0 0x7f7fac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:17.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.248+0000 7f7fc37fe700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7fac0b29f0 0x7f7fac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:17.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.248+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9562+0+0 (secure 0 0 0) 0x7f7fbc00f020 con 0x7f7fc4108760 2024-05-14T15:38:17.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.248+0000 7f7fc37fe700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7fac0b29f0 0x7f7fac0b4f00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4005950 tx=0x7f7fb400b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:17.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.251+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fbc029070 con 0x7f7fc4108760 2024-05-14T15:38:17.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.503+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fc402e7b0 con 0x7f7fc4108760 2024-05-14T15:38:17.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.503+0000 7f7fc0ff9700 1 -- 172.21.15.5:0/3362076853 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v112) ==== 81+0+927 (secure 0 0 0) 0x7f7fbc0ab2b0 con 0x7f7fc4108760 2024-05-14T15:38:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2413445605 2024-05-15T15:24:50.611144+0000 2024-05-14T15:38:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/887023568 2024-05-15T15:24:50.611144+0000 2024-05-14T15:38:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/365942426 2024-05-15T15:24:50.611144+0000 2024-05-14T15:38:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4280832550 2024-05-15T15:24:50.611144+0000 2024-05-14T15:38:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2154399460 2024-05-15T15:13:41.367574+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2401136202 2024-05-15T15:13:13.551868+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2075859388 2024-05-15T15:13:41.367574+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1587175256 2024-05-15T15:13:13.551868+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4018221162 2024-05-15T15:13:41.367574+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2401136202 2024-05-15T15:13:13.551868+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2357267352 2024-05-15T15:13:41.367574+0000 2024-05-14T15:38:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1647912546 2024-05-15T15:13:13.551868+0000 2024-05-14T15:38:17.507 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/887023568 2024-05-15T15:24:50.611144+0000 2024-05-14T15:38:17.507 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3326284548 2024-05-15T15:24:50.611144+0000 2024-05-14T15:38:17.507 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2154399460 2024-05-15T15:13:41.367574+0000 2024-05-14T15:38:17.507 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1991676154 2024-05-15T15:13:13.551868+0000 2024-05-14T15:38:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7fac0b29f0 msgr2=0x7f7fac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7fac0b29f0 0x7f7fac0b4f00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4005950 tx=0x7f7fb400b410 comp rx=0 tx=0).stop 2024-05-14T15:38:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 msgr2=0x7f7fc40ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc40ffe90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0095c0 tx=0x7f7fbc007a20 comp rx=0 tx=0).stop 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 shutdown_connections 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f7fac0b29f0 0x7f7fac0b4f00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fc4108760 0x7f7fc40ffe90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc41034e0 0x7f7fc40ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 --2- 172.21.15.5:0/3362076853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fc4107e10 0x7f7fc40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 >> 172.21.15.5:0/3362076853 conn(0x7f7fc40fef30 msgr2=0x7f7fc406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 shutdown_connections 2024-05-14T15:38:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:17.506+0000 7f7fca022700 1 -- 172.21.15.5:0/3362076853 wait complete. 2024-05-14T15:38:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-05-14T15:38:18.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:17 smithi144 ceph-mon[35650]: pgmap v427: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-05-14T15:38:18.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:17 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3362076853' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:38:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:17 smithi193 ceph-mon[36001]: pgmap v427: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-05-14T15:38:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:17 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3362076853' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:38:18.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:17 smithi005 ceph-mon[31559]: pgmap v427: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-05-14T15:38:18.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:17 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3362076853' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T15:38:18.526 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-14T15:38:18.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-14T15:38:18.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:38:18.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.581 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-14T15:38:18.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.616 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-14T15:38:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-14T15:38:18.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-14T15:38:18.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-14T15:38:18.852 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T15:38:18.852 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-14T15:38:18.852 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T15:38:18.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19679 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/aa78e767aa34f6885a0585503ebdf4ecfb64706c10ddc08cad5d4adfc8b7f14f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/aa78e767aa34f6885a0585503ebdf4ecfb64706c10ddc08cad5d4adfc8b7f14f/diff,workdir=/var/lib/containers/storage/overlay/aa78e767aa34f6885a0585503ebdf4ecfb64706c10ddc08cad5d4adfc8b7f14f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6f1965a38d0ba642c7109a54ae7b125347ebd3341a05be817d07a10e2dba97d7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/6f1965a38d0ba642c7109a54ae7b125347ebd3341a05be817d07a10e2dba97d7/diff,workdir=/var/lib/containers/storage/overlay/6f1965a38d0ba642c7109a54ae7b125347ebd3341a05be817d07a10e2dba97d7/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/20aa1db8b63736ec8db45403c94218d4d15129f4bcfa7416debbf81a2476055d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/20aa1db8b63736ec8db45403c94218d4d15129f4bcfa7416debbf81a2476055d/diff,workdir=/var/lib/containers/storage/overlay/20aa1db8b63736ec8db45403c94218d4d15129f4bcfa7416debbf81a2476055d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/41d6ea8dbb8f6373fc6c7da8c55b0c4dccfee479674cd41bf1150e1987538403/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/41d6ea8dbb8f6373fc6c7da8c55b0c4dccfee479674cd41bf1150e1987538403/diff,workdir=/var/lib/containers/storage/overlay/41d6ea8dbb8f6373fc6c7da8c55b0c4dccfee479674cd41bf1150e1987538403/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/75371eb09afe4d8cbc1ae7de2d96d8cec7d97d850c353fb59733dd4a3e6bc782/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/75371eb09afe4d8cbc1ae7de2d96d8cec7d97d850c353fb59733dd4a3e6bc782/diff,workdir=/var/lib/containers/storage/overlay/75371eb09afe4d8cbc1ae7de2d96d8cec7d97d850c353fb59733dd4a3e6bc782/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/42e81d88822b707b81f7bf14297df818f909ff32b3adb867518fbfc1304e7da1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/42e81d88822b707b81f7bf14297df818f909ff32b3adb867518fbfc1304e7da1/diff,workdir=/var/lib/containers/storage/overlay/42e81d88822b707b81f7bf14297df818f909ff32b3adb867518fbfc1304e7da1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6cff9938832b60542e8e36b70f0d4e9a96343b92a3b7926085db26edfb1e9029/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/6cff9938832b60542e8e36b70f0d4e9a96343b92a3b7926085db26edfb1e9029/diff,workdir=/var/lib/containers/storage/overlay/6cff9938832b60542e8e36b70f0d4e9a96343b92a3b7926085db26edfb1e9029/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/aa7bce5051af6445b184021d889d0d280f027a2e6a41950e9c00b8d8fca9813c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/aa7bce5051af6445b184021d889d0d280f027a2e6a41950e9c00b8d8fca9813c/diff,workdir=/var/lib/containers/storage/overlay/aa7bce5051af6445b184021d889d0d280f027a2e6a41950e9c00b8d8fca9813c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b2b9f0547288b2813fa4a470d2db8099048d617616e3bd398b5d6d42922caf08/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/b2b9f0547288b2813fa4a470d2db8099048d617616e3bd398b5d6d42922caf08/diff,workdir=/var/lib/containers/storage/overlay/b2b9f0547288b2813fa4a470d2db8099048d617616e3bd398b5d6d42922caf08/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f9786a1ca46fe2d67959773ebeda261d01d44a9fed573b4e9e1f1494fc820a2f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VUKDKWKCBH47F7Y3WAK5W5F2FF:/var/lib/containers/storage/overlay/l/WVDW7V66OAJXR22FACX2S6IZQN,upperdir=/var/lib/containers/storage/overlay/f9786a1ca46fe2d67959773ebeda261d01d44a9fed573b4e9e1f1494fc820a2f/diff,workdir=/var/lib/containers/storage/overlay/f9786a1ca46fe2d67959773ebeda261d01d44a9fed573b4e9e1f1494fc820a2f/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 2024-05-14T15:38:18.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.870 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-14T15:38:18.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T15:38:18.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-14T15:38:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-14T15:38:18.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:38:18.996 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-14T15:38:19.007 INFO:tasks.cephadm:Teardown begin 2024-05-14T15:38:19.007 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T15:38:19.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:38:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:20.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:19 smithi005 ceph-mon[31559]: pgmap v428: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T15:38:20.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:19 smithi144 ceph-mon[35650]: pgmap v428: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T15:38:20.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:19 smithi193 ceph-mon[36001]: pgmap v428: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T15:38:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 -- 172.21.15.5:0/2566578901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 msgr2=0x7ff83c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 --2- 172.21.15.5:0/2566578901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c1111b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff83400b0d0 tx=0x7ff83400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 -- 172.21.15.5:0/2566578901 shutdown_connections 2024-05-14T15:38:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 --2- 172.21.15.5:0/2566578901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c1111b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 --2- 172.21.15.5:0/2566578901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff83c102ae0 0x7ff83c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 --2- 172.21.15.5:0/2566578901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff83c108f00 0x7ff83c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 -- 172.21.15.5:0/2566578901 >> 172.21.15.5:0/2566578901 conn(0x7ff83c0fe540 msgr2=0x7ff83c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 -- 172.21.15.5:0/2566578901 shutdown_connections 2024-05-14T15:38:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.075+0000 7ff8408d7700 1 -- 172.21.15.5:0/2566578901 wait complete. 2024-05-14T15:38:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 Processor -- start 2024-05-14T15:38:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 -- start start 2024-05-14T15:38:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff83c102ae0 0x7ff83c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff83c108f00 0x7ff83c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c1144f0 con 0x7ff83c109850 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c114370 con 0x7ff83c102ae0 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.076+0000 7ff8408d7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c114670 con 0x7ff83c108f00 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43028/0 (socket says 172.21.15.5:43028) 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 -- 172.21.15.5:0/188612746 learned_addr learned my addr 172.21.15.5:0/188612746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff839d9b700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff83c108f00 0x7ff83c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 -- 172.21.15.5:0/188612746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff83c108f00 msgr2=0x7ff83c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff83c108f00 0x7ff83c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 -- 172.21.15.5:0/188612746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff83c102ae0 msgr2=0x7ff83c19ade0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff83c102ae0 0x7ff83c19ade0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 -- 172.21.15.5:0/188612746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c19fcb0 con 0x7ff83c109850 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff83ad9d700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c19f610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff8340095c0 tx=0x7ff834007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83401f070 con 0x7ff83c109850 2024-05-14T15:38:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c19ff40 con 0x7ff83c109850 2024-05-14T15:38:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff834003c00 con 0x7ff83c109850 2024-05-14T15:38:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83401b6a0 con 0x7ff83c109850 2024-05-14T15:38:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.077+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c1a0420 con 0x7ff83c109850 2024-05-14T15:38:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.079+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c0512c0 con 0x7ff83c109850 2024-05-14T15:38:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.080+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff834022070 con 0x7ff83c109850 2024-05-14T15:38:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.081+0000 7ff82b7fe700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff8240b29f0 0x7ff8240b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.081+0000 7ff83a59c700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff8240b29f0 0x7ff8240b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.081+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9562+0+0 (secure 0 0 0) 0x7ff83400f020 con 0x7ff83c109850 2024-05-14T15:38:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.081+0000 7ff83a59c700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff8240b29f0 0x7ff8240b4f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00ba10 tx=0x7ff82c00b3f0 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.086+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff834029020 con 0x7ff83c109850 2024-05-14T15:38:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.353+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff83c02e5f0 con 0x7ff83c109850 2024-05-14T15:38:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.354+0000 7ff82b7fe700 1 -- 172.21.15.5:0/188612746 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11864 (secure 0 0 0) 0x7ff8340ab330 con 0x7ff83c109850 2024-05-14T15:38:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:38:21.356 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:35:24.885756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27547":{"gid":27547,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3068356254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3068356254},{"type":"v1","addr":"172.21.15.144:6835","nonce":3068356254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:38:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff8240b29f0 msgr2=0x7ff8240b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff8240b29f0 0x7ff8240b4f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00ba10 tx=0x7ff82c00b3f0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 msgr2=0x7ff83c19f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c19f610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff8340095c0 tx=0x7ff834007a20 comp rx=0 tx=0).stop 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 shutdown_connections 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7ff8240b29f0 0x7ff8240b4f00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff83c109850 0x7ff83c19f610 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff83c102ae0 0x7ff83c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 --2- 172.21.15.5:0/188612746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff83c108f00 0x7ff83c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 >> 172.21.15.5:0/188612746 conn(0x7ff83c0fe540 msgr2=0x7ff83c106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 shutdown_connections 2024-05-14T15:38:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:21.356+0000 7ff8408d7700 1 -- 172.21.15.5:0/188612746 wait complete. 2024-05-14T15:38:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-14T15:38:22.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:21 smithi005 ceph-mon[31559]: pgmap v429: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T15:38:22.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:21 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/188612746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:38:22.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T15:38:22.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:21 smithi144 ceph-mon[35650]: pgmap v429: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T15:38:22.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:21 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/188612746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:38:22.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:21 smithi193 ceph-mon[36001]: pgmap v429: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-05-14T15:38:22.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:21 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/188612746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:38:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:24.091 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:23 smithi005 ceph-mon[31559]: pgmap v430: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-05-14T15:38:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 -- 172.21.15.5:0/1438337037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 msgr2=0x7fbb9c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 --2- 172.21.15.5:0/1438337037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c108200 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbb840099c0 tx=0x7fbb84009cd0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 -- 172.21.15.5:0/1438337037 shutdown_connections 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 --2- 172.21.15.5:0/1438337037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c108740 0x7fbb9c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 --2- 172.21.15.5:0/1438337037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9c1034c0 0x7fbb9c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 --2- 172.21.15.5:0/1438337037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c108200 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 -- 172.21.15.5:0/1438337037 >> 172.21.15.5:0/1438337037 conn(0x7fbb9c0fedf0 msgr2=0x7fbb9c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 -- 172.21.15.5:0/1438337037 shutdown_connections 2024-05-14T15:38:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.206+0000 7fbba2359700 1 -- 172.21.15.5:0/1438337037 wait complete. 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 Processor -- start 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 -- start start 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9c1034c0 0x7fbb9c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c108740 0x7fbb9c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c113350 con 0x7fbb9c107df0 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1131d0 con 0x7fbb9c108740 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbba2359700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1134d0 con 0x7fbb9c1034c0 2024-05-14T15:38:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbb93fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.207+0000 7fbb93fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43044/0 (socket says 172.21.15.5:43044) 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 -- 172.21.15.5:0/3675163165 learned_addr learned my addr 172.21.15.5:0/3675163165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb9bfff700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9c1034c0 0x7fbb9c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbba08f6700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c108740 0x7fbb9c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 -- 172.21.15.5:0/3675163165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9c1034c0 msgr2=0x7fbb9c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9c1034c0 0x7fbb9c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 -- 172.21.15.5:0/3675163165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c108740 msgr2=0x7fbb9c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c108740 0x7fbb9c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 -- 172.21.15.5:0/3675163165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c19fd20 con 0x7fbb9c107df0 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb93fff700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c19b490 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84027070 con 0x7fbb9c107df0 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c19ffb0 con 0x7fbb9c107df0 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb84003f20 con 0x7fbb9c107df0 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84022cd0 con 0x7fbb9c107df0 2024-05-14T15:38:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.208+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1a7930 con 0x7fbb9c107df0 2024-05-14T15:38:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.209+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c0512c0 con 0x7fbb9c107df0 2024-05-14T15:38:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.211+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbb8402a070 con 0x7fbb9c107df0 2024-05-14T15:38:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.212+0000 7fbb99ffb700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb880b29f0 0x7fbb880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.212+0000 7fbb9bfff700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb880b29f0 0x7fbb880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.212+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9562+0+0 (secure 0 0 0) 0x7fbb840e3cd0 con 0x7fbb9c107df0 2024-05-14T15:38:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.213+0000 7fbb9bfff700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb880b29f0 0x7fbb880b4f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005950 tx=0x7fbb8c00b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:24.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.216+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb84031080 con 0x7fbb9c107df0 2024-05-14T15:38:24.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:23 smithi144 ceph-mon[35650]: pgmap v430: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-05-14T15:38:24.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:23 smithi193 ceph-mon[36001]: pgmap v430: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-05-14T15:38:24.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.483+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb9c1a7b00 con 0x7fbb9c107df0 2024-05-14T15:38:24.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.483+0000 7fbb99ffb700 1 -- 172.21.15.5:0/3675163165 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11864 (secure 0 0 0) 0x7fbb84013070 con 0x7fbb9c107df0 2024-05-14T15:38:24.485 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:38:24.485 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/1133415705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1133415705},{"type":"v1","addr":"172.21.15.5:6835","nonce":1133415705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3415932983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3415932983},{"type":"v1","addr":"172.21.15.5:6837","nonce":3415932983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/45851617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":45851617},{"type":"v1","addr":"172.21.15.5:6839","nonce":45851617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1161784438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1161784438},{"type":"v1","addr":"172.21.15.5:6841","nonce":1161784438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2582549673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2582549673},{"type":"v1","addr":"172.21.15.193:6833","nonce":2582549673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3788502407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3788502407},{"type":"v1","addr":"172.21.15.144:6833","nonce":3788502407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T15:20:57.810761+0000","modified":"2024-05-14T15:35:24.885756+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24515,"mds_1":24491,"mds_2":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24457":{"gid":24457,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/4153907810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4153907810},{"type":"v1","addr":"172.21.15.193:6835","nonce":4153907810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/1910760961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1910760961},{"type":"v1","addr":"172.21.15.193:6837","nonce":1910760961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":23,"addr":"172.21.15.193:6839/570860802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":570860802},{"type":"v1","addr":"172.21.15.193:6839","nonce":570860802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":23,"addr":"172.21.15.144:6837/214149479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":214149479},{"type":"v1","addr":"172.21.15.144:6837","nonce":214149479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/4183436774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":4183436774},{"type":"v1","addr":"172.21.15.144:6839","nonce":4183436774}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27547":{"gid":27547,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/3068356254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3068356254},{"type":"v1","addr":"172.21.15.144:6835","nonce":3068356254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24491,24515,24487]},"id":1}]} 2024-05-14T15:38:24.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb880b29f0 msgr2=0x7fbb880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb880b29f0 0x7fbb880b4f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005950 tx=0x7fbb8c00b410 comp rx=0 tx=0).stop 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 msgr2=0x7fbb9c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c19b490 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbb84006010 tx=0x7fbb840047b0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 shutdown_connections 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fbb880b29f0 0x7fbb880b4f00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c108740 0x7fbb9c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb9c1034c0 0x7fbb9c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 --2- 172.21.15.5:0/3675163165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c107df0 0x7fbb9c19b490 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.486+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 >> 172.21.15.5:0/3675163165 conn(0x7fbb9c0fedf0 msgr2=0x7fbb9c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.487+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 shutdown_connections 2024-05-14T15:38:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:24.487+0000 7fbba2359700 1 -- 172.21.15.5:0/3675163165 wait complete. 2024-05-14T15:38:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-14T15:38:25.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:24 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3675163165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:38:25.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:24 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:38:25.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T15:38:25.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:24 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3675163165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:38:25.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:24 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:38:25.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:24 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3675163165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T15:38:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:24 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T15:38:25.481 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:26.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:25 smithi144 ceph-mon[35650]: pgmap v431: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-05-14T15:38:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:25 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:38:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:25 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:38:26.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:25 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:38:26.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:25 smithi193 ceph-mon[36001]: pgmap v431: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-05-14T15:38:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:25 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:38:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:25 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:38:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:25 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:38:26.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:25 smithi005 ceph-mon[31559]: pgmap v431: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-05-14T15:38:26.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:25 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T15:38:26.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:25 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T15:38:26.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:25 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' 2024-05-14T15:38:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 -- 172.21.15.5:0/2709704354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734107e30 msgr2=0x7fd734108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/2709704354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734107e30 0x7fd734108240 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd7280099c0 tx=0x7fd728009cd0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 -- 172.21.15.5:0/2709704354 shutdown_connections 2024-05-14T15:38:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/2709704354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd734108780 0x7fd7341100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/2709704354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734103550 0x7fd734103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/2709704354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734107e30 0x7fd734108240 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.296+0000 7fd73a7b2700 1 -- 172.21.15.5:0/2709704354 >> 172.21.15.5:0/2709704354 conn(0x7fd734078e00 msgr2=0x7fd734079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.297+0000 7fd73a7b2700 1 -- 172.21.15.5:0/2709704354 shutdown_connections 2024-05-14T15:38:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.297+0000 7fd73a7b2700 1 -- 172.21.15.5:0/2709704354 wait complete. 2024-05-14T15:38:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.298+0000 7fd73a7b2700 1 Processor -- start 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.298+0000 7fd73a7b2700 1 -- start start 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd73a7b2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd734103550 0x7fd73419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd73a7b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734107e30 0x7fd73419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd73a7b2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 0x7fd73419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd73a7b2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd734113390 con 0x7fd734108780 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd73a7b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd734113210 con 0x7fd734107e30 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd73a7b2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd734113510 con 0x7fd734103550 2024-05-14T15:38:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 0x7fd73419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 0x7fd73419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57082/0 (socket says 172.21.15.5:57082) 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 -- 172.21.15.5:0/313094160 learned_addr learned my addr 172.21.15.5:0/313094160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd733fff700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd734103550 0x7fd73419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd7337fe700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734107e30 0x7fd73419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 -- 172.21.15.5:0/313094160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd734103550 msgr2=0x7fd73419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd734103550 0x7fd73419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 -- 172.21.15.5:0/313094160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734107e30 msgr2=0x7fd73419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734107e30 0x7fd73419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd738d4f700 1 -- 172.21.15.5:0/313094160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73419fd80 con 0x7fd734108780 2024-05-14T15:38:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.299+0000 7fd7337fe700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734107e30 0x7fd73419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:38:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.300+0000 7fd738d4f700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 0x7fd73419f6e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd72c007ee0 tx=0x7fd72c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.300+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c00dd20 con 0x7fd734108780 2024-05-14T15:38:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.300+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7341a0070 con 0x7fd734108780 2024-05-14T15:38:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.300+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd72c004020 con 0x7fd734108780 2024-05-14T15:38:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.300+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c015660 con 0x7fd734108780 2024-05-14T15:38:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.300+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7341a7990 con 0x7fd734108780 2024-05-14T15:38:27.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.302+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7341049b0 con 0x7fd734108780 2024-05-14T15:38:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.303+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fd72c0041d0 con 0x7fd734108780 2024-05-14T15:38:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.304+0000 7fd7317fa700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd71c0b29f0 0x7fd71c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.304+0000 7fd733fff700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd71c0b29f0 0x7fd71c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.304+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9562+0+0 (secure 0 0 0) 0x7fd72c0e2380 con 0x7fd734108780 2024-05-14T15:38:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.305+0000 7fd733fff700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd71c0b29f0 0x7fd71c0b4f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd724005950 tx=0x7fd72400b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.309+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd72c010070 con 0x7fd734108780 2024-05-14T15:38:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.547+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd73402e5f0 con 0x7fd734108780 2024-05-14T15:38:27.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.549+0000 7fd7317fa700 1 -- 172.21.15.5:0/313094160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+25316 (secure 0 0 0) 0x7fd72c0a9950 con 0x7fd734108780 2024-05-14T15:38:27.551 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-14T15:38:27.551 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":112,"fsid":"4a7148fe-1204-11ef-bc98-c7b262605968","created":"2024-05-14T15:12:38.043878+0000","modified":"2024-05-14T15:38:13.294321+0000","last_up_change":"2024-05-14T15:19:51.926181+0000","last_in_change":"2024-05-14T15:19:31.682059+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T15:16:05.463462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T15:20:51.199481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T15:20:54.665379+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-14T15:21:04.523784+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ca22e76e-dbda-4a5c-be55-9740b288c6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6803","nonce":1440842370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6805","nonce":1440842370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6809","nonce":1440842370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1440842370},{"type":"v1","addr":"172.21.15.5:6807","nonce":1440842370}]},"public_addr":"172.21.15.5:6803/1440842370","cluster_addr":"172.21.15.5:6805/1440842370","heartbeat_back_addr":"172.21.15.5:6809/1440842370","heartbeat_front_addr":"172.21.15.5:6807/1440842370","state":["exists","up"]},{"osd":1,"uuid":"f8e57534-cfcb-479f-bcfc-64cadc8937cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6801","nonce":3346749766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6803","nonce":3346749766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6807","nonce":3346749766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3346749766},{"type":"v1","addr":"172.21.15.144:6805","nonce":3346749766}]},"public_addr":"172.21.15.144:6801/3346749766","cluster_addr":"172.21.15.144:6803/3346749766","heartbeat_back_addr":"172.21.15.144:6807/3346749766","heartbeat_front_addr":"172.21.15.144:6805/3346749766","state":["exists","up"]},{"osd":2,"uuid":"58e105d1-e047-41db-a978-c3baed2b8d25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6801","nonce":3480718798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6803","nonce":3480718798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6807","nonce":3480718798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3480718798},{"type":"v1","addr":"172.21.15.193:6805","nonce":3480718798}]},"public_addr":"172.21.15.193:6801/3480718798","cluster_addr":"172.21.15.193:6803/3480718798","heartbeat_back_addr":"172.21.15.193:6807/3480718798","heartbeat_front_addr":"172.21.15.193:6805/3480718798","state":["exists","up"]},{"osd":3,"uuid":"7530ff13-b342-47a6-9bf9-fa376975ab8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6811","nonce":431512675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6813","nonce":431512675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6817","nonce":431512675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":431512675},{"type":"v1","addr":"172.21.15.5:6815","nonce":431512675}]},"public_addr":"172.21.15.5:6811/431512675","cluster_addr":"172.21.15.5:6813/431512675","heartbeat_back_addr":"172.21.15.5:6817/431512675","heartbeat_front_addr":"172.21.15.5:6815/431512675","state":["exists","up"]},{"osd":4,"uuid":"c8db2181-5e0b-407f-9ce9-3a23a0898b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6809","nonce":3402609207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6811","nonce":3402609207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6815","nonce":3402609207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3402609207},{"type":"v1","addr":"172.21.15.144:6813","nonce":3402609207}]},"public_addr":"172.21.15.144:6809/3402609207","cluster_addr":"172.21.15.144:6811/3402609207","heartbeat_back_addr":"172.21.15.144:6815/3402609207","heartbeat_front_addr":"172.21.15.144:6813/3402609207","state":["exists","up"]},{"osd":5,"uuid":"3301b223-bb64-4c9e-a532-508e7e2e2e8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6809","nonce":2216132149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6811","nonce":2216132149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6815","nonce":2216132149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2216132149},{"type":"v1","addr":"172.21.15.193:6813","nonce":2216132149}]},"public_addr":"172.21.15.193:6809/2216132149","cluster_addr":"172.21.15.193:6811/2216132149","heartbeat_back_addr":"172.21.15.193:6815/2216132149","heartbeat_front_addr":"172.21.15.193:6813/2216132149","state":["exists","up"]},{"osd":6,"uuid":"024a549a-dc28-482e-a03d-4e4d5987e94b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6819","nonce":1238248767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6821","nonce":1238248767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6825","nonce":1238248767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1238248767},{"type":"v1","addr":"172.21.15.5:6823","nonce":1238248767}]},"public_addr":"172.21.15.5:6819/1238248767","cluster_addr":"172.21.15.5:6821/1238248767","heartbeat_back_addr":"172.21.15.5:6825/1238248767","heartbeat_front_addr":"172.21.15.5:6823/1238248767","state":["exists","up"]},{"osd":7,"uuid":"3aebc627-3769-4770-b617-721385d1cb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6817","nonce":3028125205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6819","nonce":3028125205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6823","nonce":3028125205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3028125205},{"type":"v1","addr":"172.21.15.144:6821","nonce":3028125205}]},"public_addr":"172.21.15.144:6817/3028125205","cluster_addr":"172.21.15.144:6819/3028125205","heartbeat_back_addr":"172.21.15.144:6823/3028125205","heartbeat_front_addr":"172.21.15.144:6821/3028125205","state":["exists","up"]},{"osd":8,"uuid":"bb0f1f87-2738-4488-8c2b-eac7983424ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6817","nonce":1885686165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6819","nonce":1885686165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6823","nonce":1885686165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1885686165},{"type":"v1","addr":"172.21.15.193:6821","nonce":1885686165}]},"public_addr":"172.21.15.193:6817/1885686165","cluster_addr":"172.21.15.193:6819/1885686165","heartbeat_back_addr":"172.21.15.193:6823/1885686165","heartbeat_front_addr":"172.21.15.193:6821/1885686165","state":["exists","up"]},{"osd":9,"uuid":"038f4941-b7d8-4fb3-8e34-dda9b11b012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6827","nonce":2591214668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6829","nonce":2591214668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6833","nonce":2591214668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2591214668},{"type":"v1","addr":"172.21.15.5:6831","nonce":2591214668}]},"public_addr":"172.21.15.5:6827/2591214668","cluster_addr":"172.21.15.5:6829/2591214668","heartbeat_back_addr":"172.21.15.5:6833/2591214668","heartbeat_front_addr":"172.21.15.5:6831/2591214668","state":["exists","up"]},{"osd":10,"uuid":"a017373b-14d7-440d-a525-b6bb24b2690d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6825","nonce":366654361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6827","nonce":366654361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6831","nonce":366654361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":366654361},{"type":"v1","addr":"172.21.15.144:6829","nonce":366654361}]},"public_addr":"172.21.15.144:6825/366654361","cluster_addr":"172.21.15.144:6827/366654361","heartbeat_back_addr":"172.21.15.144:6831/366654361","heartbeat_front_addr":"172.21.15.144:6829/366654361","state":["exists","up"]},{"osd":11,"uuid":"f9fa4c86-34f4-4c7d-9b8a-ff6e465091e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6825","nonce":1840598125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6827","nonce":1840598125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6831","nonce":1840598125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1840598125},{"type":"v1","addr":"172.21.15.193:6829","nonce":1840598125}]},"public_addr":"172.21.15.193:6825/1840598125","cluster_addr":"172.21.15.193:6827/1840598125","heartbeat_back_addr":"172.21.15.193:6831/1840598125","heartbeat_front_addr":"172.21.15.193:6829/1840598125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:15.520211+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:15:39.142904+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:02.767147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:27.961952+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:16:53.108035+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:17.749422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:17:43.005619+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:07.440058+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:32.550842+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:18:58.310269+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:23.893636+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T15:19:50.075977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2413445605":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/365942426":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/4280832550":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6801/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6801/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2075859388":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1587175256":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/4018221162":"2024-05-15T15:13:41.367574+0000","172.21.15.5:6800/2401136202":"2024-05-15T15:13:13.551868+0000","172.21.15.5:0/2357267352":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1647912546":"2024-05-15T15:13:13.551868+0000","172.21.15.5:6800/887023568":"2024-05-15T15:24:50.611144+0000","172.21.15.5:0/3326284548":"2024-05-15T15:24:50.611144+0000","172.21.15.5:6800/2154399460":"2024-05-15T15:13:41.367574+0000","172.21.15.5:0/1991676154":"2024-05-15T15:13:13.551868+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd71c0b29f0 msgr2=0x7fd71c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd71c0b29f0 0x7fd71c0b4f00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd724005950 tx=0x7fd72400b410 comp rx=0 tx=0).stop 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 msgr2=0x7fd73419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 0x7fd73419f6e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd72c007ee0 tx=0x7fd72c00d3b0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 shutdown_connections 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fd71c0b29f0 0x7fd71c0b4f00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd734108780 0x7fd73419f6e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd734103550 0x7fd73419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 --2- 172.21.15.5:0/313094160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd734107e30 0x7fd73419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 >> 172.21.15.5:0/313094160 conn(0x7fd734078e00 msgr2=0x7fd734075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 shutdown_connections 2024-05-14T15:38:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:27.552+0000 7fd73a7b2700 1 -- 172.21.15.5:0/313094160 wait complete. 2024-05-14T15:38:28.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:27 smithi005 ceph-mon[31559]: pgmap v432: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-05-14T15:38:28.190 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:27 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/313094160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:38:28.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-14T15:38:28.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:27 smithi144 ceph-mon[35650]: pgmap v432: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-05-14T15:38:28.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:27 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/313094160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:38:28.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:27 smithi193 ceph-mon[36001]: pgmap v432: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-05-14T15:38:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:27 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/313094160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T15:38:28.469 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:30.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:29 smithi005 ceph-mon[31559]: pgmap v433: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-14T15:38:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 -- 172.21.15.5:0/876948918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 msgr2=0x7fa9b4103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/876948918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b4103f40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400b0d0 tx=0x7fa9a400b3e0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 -- 172.21.15.5:0/876948918 shutdown_connections 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/876948918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b41087a0 0x7fa9b4110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/876948918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b4103f40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/876948918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b4107e50 0x7fa9b4108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 -- 172.21.15.5:0/876948918 >> 172.21.15.5:0/876948918 conn(0x7fa9b40fe520 msgr2=0x7fa9b4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 -- 172.21.15.5:0/876948918 shutdown_connections 2024-05-14T15:38:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.281+0000 7fa9b9b63700 1 -- 172.21.15.5:0/876948918 wait complete. 2024-05-14T15:38:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.282+0000 7fa9b9b63700 1 Processor -- start 2024-05-14T15:38:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.282+0000 7fa9b9b63700 1 -- start start 2024-05-14T15:38:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.282+0000 7fa9b9b63700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.282+0000 7fa9b9b63700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b4107e50 0x7fa9b419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.282+0000 7fa9b9b63700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b41087a0 0x7fa9b419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b9b63700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4114560 con 0x7fa9b4103b30 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b9b63700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41143e0 con 0x7fa9b41087a0 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b9b63700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41146e0 con 0x7fa9b4107e50 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57098/0 (socket says 172.21.15.5:57098) 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 -- 172.21.15.5:0/2561796883 learned_addr learned my addr 172.21.15.5:0/2561796883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b2ffd700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b4107e50 0x7fa9b419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b41087a0 0x7fa9b419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 -- 172.21.15.5:0/2561796883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b4107e50 msgr2=0x7fa9b419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b4107e50 0x7fa9b419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 -- 172.21.15.5:0/2561796883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b41087a0 msgr2=0x7fa9b419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b41087a0 0x7fa9b419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 -- 172.21.15.5:0/2561796883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b419fd80 con 0x7fa9b4103b30 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b3fff700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b41087a0 0x7fa9b419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b37fe700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b419aeb0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400d010 tx=0x7fa9a4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a401f070 con 0x7fa9b4103b30 2024-05-14T15:38:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41a0010 con 0x7fa9b4103b30 2024-05-14T15:38:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a4003c00 con 0x7fa9b4103b30 2024-05-14T15:38:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.283+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a401b6a0 con 0x7fa9b4103b30 2024-05-14T15:38:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.284+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b41a04f0 con 0x7fa9b4103b30 2024-05-14T15:38:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.285+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b410b7c0 con 0x7fa9b4103b30 2024-05-14T15:38:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.286+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa9a4022070 con 0x7fa9b4103b30 2024-05-14T15:38:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.287+0000 7fa9b0ff9700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa99c0b29f0 0x7fa99c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.287+0000 7fa9b2ffd700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa99c0b29f0 0x7fa99c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.287+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9562+0+0 (secure 0 0 0) 0x7fa9a400f020 con 0x7fa9b4103b30 2024-05-14T15:38:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.288+0000 7fa9b2ffd700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa99c0b29f0 0x7fa99c0b4f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40feac0 tx=0x7fa9a800b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.291+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9a4029020 con 0x7fa9b4103b30 2024-05-14T15:38:30.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:29 smithi144 ceph-mon[35650]: pgmap v433: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-14T15:38:30.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:29 smithi193 ceph-mon[36001]: pgmap v433: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-14T15:38:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.555+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa9b419c210 con 0x7fa9b4103b30 2024-05-14T15:38:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.558+0000 7fa9b0ff9700 1 -- 172.21.15.5:0/2561796883 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7fa9a40ab6b0 con 0x7fa9b4103b30 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.561+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa99c0b29f0 msgr2=0x7fa99c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.561+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa99c0b29f0 0x7fa99c0b4f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40feac0 tx=0x7fa9a800b410 comp rx=0 tx=0).stop 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.561+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 msgr2=0x7fa9b419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.561+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b419aeb0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400d010 tx=0x7fa9a4007a20 comp rx=0 tx=0).stop 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 shutdown_connections 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7fa99c0b29f0 0x7fa99c0b4f00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b41087a0 0x7fa9b419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9b4103b30 0x7fa9b419aeb0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 --2- 172.21.15.5:0/2561796883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9b4107e50 0x7fa9b419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 >> 172.21.15.5:0/2561796883 conn(0x7fa9b40fe520 msgr2=0x7fa9b4106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 shutdown_connections 2024-05-14T15:38:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:30.562+0000 7fa9b9b63700 1 -- 172.21.15.5:0/2561796883 wait complete. 2024-05-14T15:38:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T15:38:30.815 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a[31555]: 2024-05-14T15:38:30.556+0000 7f8434454700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T15:38:30.956 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: ignoring --setuser ceph since I am not root 2024-05-14T15:38:30.957 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: ignoring --setgroup ceph since I am not root 2024-05-14T15:38:30.957 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mds[66392]: -- 172.21.15.144:0/3719724913 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e83c2bfa40 con 0x55e83c307180 2024-05-14T15:38:30.957 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: starting mds.b at 2024-05-14T15:38:30.957 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setuser ceph since I am not root 2024-05-14T15:38:30.958 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: ignoring --setgroup ceph since I am not root 2024-05-14T15:38:30.958 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mds[65770]: -- 172.21.15.144:0/2680870803 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591caa7da40 con 0x5591caac5180 2024-05-14T15:38:30.958 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: starting mds.e at 2024-05-14T15:38:30.958 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h[66084]: ignoring --setuser ceph since I am not root 2024-05-14T15:38:30.958 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h[66084]: ignoring --setgroup ceph since I am not root 2024-05-14T15:38:30.958 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mds[66088]: -- 172.21.15.144:0/1081433492 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4b97a1a40 con 0x55d4b97e9600 2024-05-14T15:38:30.959 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:38:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h[66084]: starting mds.h at 2024-05-14T15:38:30.959 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c[66206]: ignoring --setuser ceph since I am not root 2024-05-14T15:38:30.959 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c[66206]: ignoring --setgroup ceph since I am not root 2024-05-14T15:38:30.959 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mds[66210]: -- 172.21.15.193:0/2414125321 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf432c1a40 con 0x55bf43309180 2024-05-14T15:38:30.959 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c[66206]: starting mds.c at 2024-05-14T15:38:30.960 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f[65905]: ignoring --setuser ceph since I am not root 2024-05-14T15:38:30.960 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f[65905]: ignoring --setgroup ceph since I am not root 2024-05-14T15:38:30.960 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mds[65909]: -- 172.21.15.193:0/857497484 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c500a53a40 con 0x55c500a9b600 2024-05-14T15:38:30.960 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f[65905]: starting mds.f at 2024-05-14T15:38:30.960 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i[66521]: ignoring --setuser ceph since I am not root 2024-05-14T15:38:30.961 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i[66521]: ignoring --setgroup ceph since I am not root 2024-05-14T15:38:30.961 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mds[66525]: -- 172.21.15.193:0/3364479499 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a1c029a40 con 0x563a1c071600 2024-05-14T15:38:30.961 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:38:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i[66521]: starting mds.i at 2024-05-14T15:38:31.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2561796883' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T15:38:31.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-mon[31559]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T15:38:31.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-mon[31559]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T15:38:31.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/2561796883' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T15:38:31.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-mon[31559]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T15:38:31.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:30 smithi005 ceph-mon[31559]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T15:38:31.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T15:38:31.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2561796883' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T15:38:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mon[35650]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T15:38:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mon[35650]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T15:38:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/2561796883' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T15:38:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mon[35650]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T15:38:31.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:30 smithi144 ceph-mon[35650]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T15:38:31.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2561796883' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T15:38:31.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mon[36001]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T15:38:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mon[36001]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T15:38:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/2561796883' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T15:38:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mon[36001]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T15:38:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:30 smithi193 ceph-mon[36001]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T15:38:31.525 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:32.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: pgmap v435: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-14T15:38:32.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6834/112954201,v1:172.21.15.144:6835/112954201] up:boot 2024-05-14T15:38:32.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6834/3850900126,v1:172.21.15.193:6835/3850900126] up:boot 2024-05-14T15:38:32.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6836/3385906079,v1:172.21.15.144:6837/3385906079] up:boot 2024-05-14T15:38:32.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6836/2508346181,v1:172.21.15.193:6837/2508346181] up:boot 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.193:6838/1512162979,v1:172.21.15.193:6839/1512162979] up:boot 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: mds.? [v2:172.21.15.144:6838/2760800559,v1:172.21.15.144:6839/2760800559] up:boot 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:38:32.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:31 smithi193 ceph-mon[36001]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:38:32.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: pgmap v435: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-14T15:38:32.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6834/112954201,v1:172.21.15.144:6835/112954201] up:boot 2024-05-14T15:38:32.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6834/3850900126,v1:172.21.15.193:6835/3850900126] up:boot 2024-05-14T15:38:32.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6836/3385906079,v1:172.21.15.144:6837/3385906079] up:boot 2024-05-14T15:38:32.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6836/2508346181,v1:172.21.15.193:6837/2508346181] up:boot 2024-05-14T15:38:32.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.193:6838/1512162979,v1:172.21.15.193:6839/1512162979] up:boot 2024-05-14T15:38:32.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: mds.? [v2:172.21.15.144:6838/2760800559,v1:172.21.15.144:6839/2760800559] up:boot 2024-05-14T15:38:32.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T15:38:32.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:38:32.299 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:38:32.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:38:32.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:38:32.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:38:32.300 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:31 smithi144 ceph-mon[35650]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:38:32.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: pgmap v435: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6834/112954201,v1:172.21.15.144:6835/112954201] up:boot 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6834/3850900126,v1:172.21.15.193:6835/3850900126] up:boot 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6836/3385906079,v1:172.21.15.144:6837/3385906079] up:boot 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6836/2508346181,v1:172.21.15.193:6837/2508346181] up:boot 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.193:6838/1512162979,v1:172.21.15.193:6839/1512162979] up:boot 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: mds.? [v2:172.21.15.144:6838/2760800559,v1:172.21.15.144:6839/2760800559] up:boot 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T15:38:32.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T15:38:32.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T15:38:32.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T15:38:32.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T15:38:32.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T15:38:32.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:31 smithi005 ceph-mon[31559]: from='mgr.15246 172.21.15.5:0/2067781234' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T15:38:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.212+0000 7f254e312700 1 -- 172.21.15.5:0/2109504910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 msgr2=0x7f254810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.212+0000 7f254e312700 1 --2- 172.21.15.5:0/2109504910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f254810e3a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2530009990 tx=0x7f2530009ca0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 -- 172.21.15.5:0/2109504910 shutdown_connections 2024-05-14T15:38:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 --2- 172.21.15.5:0/2109504910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f254810e8e0 0x7f2548110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 --2- 172.21.15.5:0/2109504910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2548105500 0x7f2548105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 --2- 172.21.15.5:0/2109504910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f254810e3a0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 -- 172.21.15.5:0/2109504910 >> 172.21.15.5:0/2109504910 conn(0x7f25480fedc0 msgr2=0x7f25481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 -- 172.21.15.5:0/2109504910 shutdown_connections 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 -- 172.21.15.5:0/2109504910 wait complete. 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 Processor -- start 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.213+0000 7f254e312700 1 -- start start 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254e312700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2548105500 0x7f25481031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254e312700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f2548103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254e312700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254810e8e0 0x7f2548103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254e312700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2548114440 con 0x7f2548105e50 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254e312700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25481142c0 con 0x7f2548105500 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254e312700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25481145c0 con 0x7f254810e8e0 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f2548103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f2548103730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57112/0 (socket says 172.21.15.5:57112) 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 -- 172.21.15.5:0/1415025598 learned_addr learned my addr 172.21.15.5:0/1415025598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254c8af700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254810e8e0 0x7f2548103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f2547fff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2548105500 0x7f25481031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 -- 172.21.15.5:0/1415025598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254810e8e0 msgr2=0x7f2548103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254810e8e0 0x7f2548103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 -- 172.21.15.5:0/1415025598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2548105500 msgr2=0x7f25481031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2548105500 0x7f25481031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 -- 172.21.15.5:0/1415025598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25481a9140 con 0x7f2548105e50 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f2547fff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2548105500 0x7f25481031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f254c8af700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254810e8e0 0x7f2548103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.214+0000 7f253ffff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f2548103730 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2530000c00 tx=0x7f2530009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.215+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530027070 con 0x7f2548105e50 2024-05-14T15:38:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.215+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2530022470 con 0x7f2548105e50 2024-05-14T15:38:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.215+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530017650 con 0x7f2548105e50 2024-05-14T15:38:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.215+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25481a93d0 con 0x7f2548105e50 2024-05-14T15:38:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.215+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a98b0 con 0x7f2548105e50 2024-05-14T15:38:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.216+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25480512c0 con 0x7f2548105e50 2024-05-14T15:38:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.218+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f253002a070 con 0x7f2548105e50 2024-05-14T15:38:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.219+0000 7f2545ffb700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25340b29f0 0x7f25340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.219+0000 7f2547fff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25340b29f0 0x7f25340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.219+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9732+0+0 (secure 0 0 0) 0x7f25300e3e20 con 0x7f2548105e50 2024-05-14T15:38:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.220+0000 7f2547fff700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25340b29f0 0x7f25340b4f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2538009cb0 tx=0x7f2538009260 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.223+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2530013070 con 0x7f2548105e50 2024-05-14T15:38:33.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.484+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f254802e5f0 con 0x7f2548105e50 2024-05-14T15:38:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.959+0000 7f2545ffb700 1 -- 172.21.15.5:0/1415025598 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f25300ab370 con 0x7f2548105e50 2024-05-14T15:38:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.962+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25340b29f0 msgr2=0x7f25340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25340b29f0 0x7f25340b4f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2538009cb0 tx=0x7f2538009260 comp rx=0 tx=0).stop 2024-05-14T15:38:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 msgr2=0x7f2548103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f2548103730 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2530000c00 tx=0x7f2530009580 comp rx=0 tx=0).stop 2024-05-14T15:38:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 shutdown_connections 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f25340b29f0 0x7f25340b4f00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254810e8e0 0x7f2548103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2548105500 0x7f25481031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 --2- 172.21.15.5:0/1415025598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548105e50 0x7f2548103730 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 >> 172.21.15.5:0/1415025598 conn(0x7f25480fedc0 msgr2=0x7f25480ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 shutdown_connections 2024-05-14T15:38:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:33.963+0000 7f254e312700 1 -- 172.21.15.5:0/1415025598 wait complete. 2024-05-14T15:38:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:33 smithi144 ceph-mon[35650]: pgmap v436: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T15:38:34.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:33 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1415025598' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:38:34.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:33 smithi193 ceph-mon[36001]: pgmap v436: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T15:38:34.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:33 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1415025598' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:38:34.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:33 smithi005 ceph-mon[31559]: pgmap v436: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T15:38:34.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:33 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1415025598' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T15:38:34.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T15:38:34.989 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:35.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:34 smithi005 ceph-mon[31559]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T15:38:35.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:34 smithi005 ceph-mon[31559]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T15:38:35.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:34 smithi005 ceph-mon[31559]: Cluster is now healthy 2024-05-14T15:38:35.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:34 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1415025598' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T15:38:35.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:34 smithi005 ceph-mon[31559]: fsmap 12 up:standby 2024-05-14T15:38:35.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:34 smithi144 ceph-mon[35650]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T15:38:35.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:34 smithi144 ceph-mon[35650]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T15:38:35.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:34 smithi144 ceph-mon[35650]: Cluster is now healthy 2024-05-14T15:38:35.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:34 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1415025598' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T15:38:35.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:34 smithi144 ceph-mon[35650]: fsmap 12 up:standby 2024-05-14T15:38:35.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:34 smithi193 ceph-mon[36001]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T15:38:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:34 smithi193 ceph-mon[36001]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T15:38:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:34 smithi193 ceph-mon[36001]: Cluster is now healthy 2024-05-14T15:38:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:34 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1415025598' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T15:38:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:34 smithi193 ceph-mon[36001]: fsmap 12 up:standby 2024-05-14T15:38:36.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:35 smithi005 ceph-mon[31559]: pgmap v437: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T15:38:36.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:35 smithi144 ceph-mon[35650]: pgmap v437: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T15:38:36.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:35 smithi193 ceph-mon[36001]: pgmap v437: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-14T15:38:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 -- 172.21.15.5:0/362302630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514103960 msgr2=0x7f7514103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 --2- 172.21.15.5:0/362302630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514103960 0x7f7514103d70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7500009990 tx=0x7f7500009ca0 comp rx=0 tx=0).stop 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 -- 172.21.15.5:0/362302630 shutdown_connections 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 --2- 172.21.15.5:0/362302630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75141042b0 0x7f7514110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 --2- 172.21.15.5:0/362302630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514108200 0x7f7514108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 --2- 172.21.15.5:0/362302630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514103960 0x7f7514103d70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 -- 172.21.15.5:0/362302630 >> 172.21.15.5:0/362302630 conn(0x7f75140fe4e0 msgr2=0x7f7514100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 -- 172.21.15.5:0/362302630 shutdown_connections 2024-05-14T15:38:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.843+0000 7f751b237700 1 -- 172.21.15.5:0/362302630 wait complete. 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.844+0000 7f751b237700 1 Processor -- start 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.844+0000 7f751b237700 1 -- start start 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.844+0000 7f751b237700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514103960 0x7f751419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f751b237700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75141042b0 0x7f751419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f751b237700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 0x7f751419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f751b237700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7514113380 con 0x7f7514108200 2024-05-14T15:38:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f751b237700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7514113200 con 0x7f7514103960 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f751b237700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7514113500 con 0x7f75141042b0 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 0x7f751419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 0x7f751419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56516/0 (socket says 172.21.15.5:56516) 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 -- 172.21.15.5:0/1492997849 learned_addr learned my addr 172.21.15.5:0/1492997849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f7518fd3700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514103960 0x7f751419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f7513fff700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75141042b0 0x7f751419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 -- 172.21.15.5:0/1492997849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75141042b0 msgr2=0x7f751419c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75141042b0 0x7f751419c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 -- 172.21.15.5:0/1492997849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514103960 msgr2=0x7f751419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514103960 0x7f751419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 -- 172.21.15.5:0/1492997849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75141a6f50 con 0x7f7514108200 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.845+0000 7f75197d4700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 0x7f751419c6a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f750c00ada0 tx=0x7f750c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.846+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c017020 con 0x7f7514108200 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.846+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7514199a30 con 0x7f7514108200 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.846+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c00cd80 con 0x7f7514108200 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.846+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c005260 con 0x7f7514108200 2024-05-14T15:38:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.846+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7514199f40 con 0x7f7514108200 2024-05-14T15:38:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.847+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751410b6f0 con 0x7f7514108200 2024-05-14T15:38:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.848+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f750c00a090 con 0x7f7514108200 2024-05-14T15:38:36.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.849+0000 7f7511ffb700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f75040b29f0 0x7f75040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:36.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.849+0000 7f7518fd3700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f75040b29f0 0x7f75040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:36.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.849+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9732+0+0 (secure 0 0 0) 0x7f750c0e2800 con 0x7f7514108200 2024-05-14T15:38:36.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.850+0000 7f7518fd3700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f75040b29f0 0x7f75040b4f00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f751406c2d0 tx=0x7f750800a380 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:36.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:36.853+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f750c021020 con 0x7f7514108200 2024-05-14T15:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.069+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75140512c0 con 0x7f7514108200 2024-05-14T15:38:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.982+0000 7f7511ffb700 1 -- 172.21.15.5:0/1492997849 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) ==== 181+0+0 (secure 0 0 0) 0x7f750c0a9db0 con 0x7f7514108200 2024-05-14T15:38:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.986+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f75040b29f0 msgr2=0x7f75040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.986+0000 7f751b237700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f75040b29f0 0x7f75040b4f00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f751406c2d0 tx=0x7f750800a380 comp rx=0 tx=0).stop 2024-05-14T15:38:37.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.986+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 msgr2=0x7f751419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:37.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.986+0000 7f751b237700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 0x7f751419c6a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f750c00ada0 tx=0x7f750c00c330 comp rx=0 tx=0).stop 2024-05-14T15:38:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.987+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 shutdown_connections 2024-05-14T15:38:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.987+0000 7f751b237700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f75040b29f0 0x7f75040b4f00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.987+0000 7f751b237700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7514108200 0x7f751419c6a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.987+0000 7f751b237700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7514103960 0x7f751419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.987+0000 7f751b237700 1 --2- 172.21.15.5:0/1492997849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75141042b0 0x7f751419c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.987+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 >> 172.21.15.5:0/1492997849 conn(0x7f75140fe4e0 msgr2=0x7f75141076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.988+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 shutdown_connections 2024-05-14T15:38:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:37.988+0000 7f751b237700 1 -- 172.21.15.5:0/1492997849 wait complete. 2024-05-14T15:38:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-14T15:38:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:37 smithi144 ceph-mon[35650]: pgmap v438: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3 op/s 2024-05-14T15:38:38.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:37 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1492997849' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:38.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:37 smithi193 ceph-mon[36001]: pgmap v438: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3 op/s 2024-05-14T15:38:38.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:37 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1492997849' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:38.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:37 smithi005 ceph-mon[31559]: pgmap v438: 129 pgs: 129 active+clean; 58 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3 op/s 2024-05-14T15:38:38.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:37 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1492997849' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:38.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T15:38:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:39.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:38 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/1492997849' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:39.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:38 smithi005 ceph-mon[31559]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T15:38:39.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:38 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/1492997849' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:39.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:38 smithi144 ceph-mon[35650]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T15:38:39.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:38 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/1492997849' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:39.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:38 smithi193 ceph-mon[36001]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T15:38:40.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:39 smithi005 ceph-mon[31559]: pgmap v440: 97 pgs: 97 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:40.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:39 smithi144 ceph-mon[35650]: pgmap v440: 97 pgs: 97 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:40.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:40 smithi193 ceph-mon[36001]: pgmap v440: 97 pgs: 97 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 -- 172.21.15.5:0/2350608540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c104610 msgr2=0x7f450c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 --2- 172.21.15.5:0/2350608540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c104610 0x7f450c104a20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00b0d0 tx=0x7f44fc00b3e0 comp rx=0 tx=0).stop 2024-05-14T15:38:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 -- 172.21.15.5:0/2350608540 shutdown_connections 2024-05-14T15:38:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 --2- 172.21.15.5:0/2350608540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450c109890 0x7f450c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 --2- 172.21.15.5:0/2350608540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c104610 0x7f450c104a20 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 --2- 172.21.15.5:0/2350608540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c108f40 0x7f450c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.649+0000 7f45108ed700 1 -- 172.21.15.5:0/2350608540 >> 172.21.15.5:0/2350608540 conn(0x7f450c07af70 msgr2=0x7f450c07b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.650+0000 7f45108ed700 1 -- 172.21.15.5:0/2350608540 shutdown_connections 2024-05-14T15:38:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.650+0000 7f45108ed700 1 -- 172.21.15.5:0/2350608540 wait complete. 2024-05-14T15:38:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.650+0000 7f45108ed700 1 Processor -- start 2024-05-14T15:38:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.650+0000 7f45108ed700 1 -- start start 2024-05-14T15:38:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f45108ed700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c104610 0x7f450c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f45108ed700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450c108f40 0x7f450c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f45108ed700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 0x7f450c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f45108ed700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f450c1144e0 con 0x7f450c109890 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f45108ed700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f450c114360 con 0x7f450c104610 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f45108ed700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f450c114660 con 0x7f450c108f40 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 0x7f450c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 0x7f450c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56526/0 (socket says 172.21.15.5:56526) 2024-05-14T15:38:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 -- 172.21.15.5:0/3527504192 learned_addr learned my addr 172.21.15.5:0/3527504192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f4509d9b700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450c108f40 0x7f450c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450a59c700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c104610 0x7f450c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 -- 172.21.15.5:0/3527504192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450c108f40 msgr2=0x7f450c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450c108f40 0x7f450c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 -- 172.21.15.5:0/3527504192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c104610 msgr2=0x7f450c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c104610 0x7f450c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.651+0000 7f450ad9d700 1 -- 172.21.15.5:0/3527504192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c19fce0 con 0x7f450c109890 2024-05-14T15:38:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.652+0000 7f450ad9d700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 0x7f450c19f640 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f450400d980 tx=0x7f450400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.652+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504017070 con 0x7f450c109890 2024-05-14T15:38:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.652+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4504005020 con 0x7f450c109890 2024-05-14T15:38:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.652+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c19ffd0 con 0x7f450c109890 2024-05-14T15:38:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.652+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450400fa10 con 0x7f450c109890 2024-05-14T15:38:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.652+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c1a78f0 con 0x7f450c109890 2024-05-14T15:38:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.653+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c0512c0 con 0x7f450c109890 2024-05-14T15:38:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.655+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4504004a30 con 0x7f450c109890 2024-05-14T15:38:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.655+0000 7f45037fe700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44f80b2a70 0x7f44f80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.656+0000 7f450a59c700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44f80b2a70 0x7f44f80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.656+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9120+0+0 (secure 0 0 0) 0x7f45040e29b0 con 0x7f450c109890 2024-05-14T15:38:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.656+0000 7f450a59c700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44f80b2a70 0x7f44f80b4f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009900 tx=0x7f44fc00b040 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:40.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.659+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4504020020 con 0x7f450c109890 2024-05-14T15:38:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:40.871+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f450c02e5f0 con 0x7f450c109890 2024-05-14T15:38:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.005+0000 7f45037fe700 1 -- 172.21.15.5:0/3527504192 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) ==== 169+0+0 (secure 0 0 0) 0x7f45040aa1c0 con 0x7f450c109890 2024-05-14T15:38:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44f80b2a70 msgr2=0x7f44f80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44f80b2a70 0x7f44f80b4f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009900 tx=0x7f44fc00b040 comp rx=0 tx=0).stop 2024-05-14T15:38:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 msgr2=0x7f450c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 0x7f450c19f640 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f450400d980 tx=0x7f450400dc90 comp rx=0 tx=0).stop 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 shutdown_connections 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f44f80b2a70 0x7f44f80b4f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450c109890 0x7f450c19f640 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f450c104610 0x7f450c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 --2- 172.21.15.5:0/3527504192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450c108f40 0x7f450c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 >> 172.21.15.5:0/3527504192 conn(0x7f450c07af70 msgr2=0x7f450c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 shutdown_connections 2024-05-14T15:38:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:41.010+0000 7f45108ed700 1 -- 172.21.15.5:0/3527504192 wait complete. 2024-05-14T15:38:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-14T15:38:41.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:41 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3527504192' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:41.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:41 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3527504192' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:41.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:41 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3527504192' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:41.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780 shell --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-14T15:38:41.879 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T15:38:42.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:42 smithi005 ceph-mon[31559]: pgmap v441: 97 pgs: 97 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:42.253 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:42 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/3527504192' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:42.253 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:42 smithi005 ceph-mon[31559]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T15:38:42.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:42 smithi144 ceph-mon[35650]: pgmap v441: 97 pgs: 97 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:42 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/3527504192' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:42.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:42 smithi144 ceph-mon[35650]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T15:38:42.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:42 smithi193 ceph-mon[36001]: pgmap v441: 97 pgs: 97 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:42 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/3527504192' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:42.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:42 smithi193 ceph-mon[36001]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T15:38:43.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 -- 172.21.15.5:0/1675537442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98108760 msgr2=0x7f2f981100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/1675537442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98108760 0x7f2f981100c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b0d0 tx=0x7f2f9000b3e0 comp rx=0 tx=0).stop 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 -- 172.21.15.5:0/1675537442 shutdown_connections 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/1675537442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98108760 0x7f2f981100c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/1675537442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f981034e0 0x7f2f981038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/1675537442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f98107e10 0x7f2f98108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 -- 172.21.15.5:0/1675537442 >> 172.21.15.5:0/1675537442 conn(0x7f2f980fef30 msgr2=0x7f2f98101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 -- 172.21.15.5:0/1675537442 shutdown_connections 2024-05-14T15:38:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.577+0000 7f2f9d21b700 1 -- 172.21.15.5:0/1675537442 wait complete. 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.578+0000 7f2f9d21b700 1 Processor -- start 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.578+0000 7f2f9d21b700 1 -- start start 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.578+0000 7f2f9d21b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f981034e0 0x7f2f9819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9d21b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 0x7f2f9819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9d21b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f98108760 0x7f2f9819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9d21b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f981144e0 con 0x7f2f98107e10 2024-05-14T15:38:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9d21b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98114360 con 0x7f2f98108760 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9d21b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98114660 con 0x7f2f981034e0 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 0x7f2f9819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 0x7f2f9819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56536/0 (socket says 172.21.15.5:56536) 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 -- 172.21.15.5:0/956566178 learned_addr learned my addr 172.21.15.5:0/956566178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f96d9d700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f981034e0 0x7f2f9819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9759e700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f98108760 0x7f2f9819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 -- 172.21.15.5:0/956566178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f981034e0 msgr2=0x7f2f9819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f981034e0 0x7f2f9819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 -- 172.21.15.5:0/956566178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f98108760 msgr2=0x7f2f9819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f98108760 0x7f2f9819f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 -- 172.21.15.5:0/956566178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9819fc60 con 0x7f2f98107e10 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f9659c700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 0x7f2f9819b330 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00d980 tx=0x7f2f8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.579+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c017070 con 0x7f2f98107e10 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.580+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9819ff50 con 0x7f2f98107e10 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.580+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f8c00fd40 con 0x7f2f98107e10 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.580+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00e3b0 con 0x7f2f98107e10 2024-05-14T15:38:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.580+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981a7870 con 0x7f2f98107e10 2024-05-14T15:38:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.581+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f980512c0 con 0x7f2f98107e10 2024-05-14T15:38:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.582+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2f8c010610 con 0x7f2f98107e10 2024-05-14T15:38:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.583+0000 7f2f7ffff700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f800b29f0 0x7f2f800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T15:38:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.583+0000 7f2f96d9d700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f800b29f0 0x7f2f800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T15:38:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.583+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8561+0+0 (secure 0 0 0) 0x7f2f8c0e2010 con 0x7f2f98107e10 2024-05-14T15:38:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.584+0000 7f2f96d9d700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f800b29f0 0x7f2f800b4f00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005950 tx=0x7f2f8800b410 comp rx=0 tx=0).ready entity=mgr.15246 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T15:38:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.587+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f8c0a99c0 con 0x7f2f98107e10 2024-05-14T15:38:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:43.800+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f9802e5f0 con 0x7f2f98107e10 2024-05-14T15:38:44.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.009+0000 7f2f7ffff700 1 -- 172.21.15.5:0/956566178 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v116) ==== 178+0+0 (secure 0 0 0) 0x7f2f8c0a92c0 con 0x7f2f98107e10 2024-05-14T15:38:44.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.013+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f800b29f0 msgr2=0x7f2f800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:44.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.013+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f800b29f0 0x7f2f800b4f00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f88005950 tx=0x7f2f8800b410 comp rx=0 tx=0).stop 2024-05-14T15:38:44.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.013+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 msgr2=0x7f2f9819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T15:38:44.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.013+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 0x7f2f9819b330 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00d980 tx=0x7f2f8c00dc90 comp rx=0 tx=0).stop 2024-05-14T15:38:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 shutdown_connections 2024-05-14T15:38:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:6800/2405629464,v1:172.21.15.5:6801/2405629464] conn(0x7f2f800b29f0 0x7f2f800b4f00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f98108760 0x7f2f9819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f981034e0 0x7f2f9819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 --2- 172.21.15.5:0/956566178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f98107e10 0x7f2f9819b330 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T15:38:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 >> 172.21.15.5:0/956566178 conn(0x7f2f980fef30 msgr2=0x7f2f98107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T15:38:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 shutdown_connections 2024-05-14T15:38:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-14T15:38:44.015+0000 7f2f9d21b700 1 -- 172.21.15.5:0/956566178 wait complete. 2024-05-14T15:38:44.018 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' removed 2024-05-14T15:38:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:44 smithi144 ceph-mon[35650]: pgmap v443: 33 pgs: 33 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:44.298 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:44 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/956566178' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:44.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:44 smithi193 ceph-mon[36001]: pgmap v443: 33 pgs: 33 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:44.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:44 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/956566178' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:44.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:44 smithi005 ceph-mon[31559]: pgmap v443: 33 pgs: 33 active+clean; 48 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T15:38:44.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:44 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/956566178' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T15:38:44.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:38:44.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:38:44.809 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:38:44.836 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-14T15:38:44.836 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T15:38:44.851 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T15:38:44.866 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T15:38:44.890 INFO:tasks.cephadm:Stopping all daemons... 2024-05-14T15:38:44.891 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-14T15:38:44.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a 2024-05-14T15:38:45.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:44 smithi005 systemd[1]: Stopping Ceph mon.a for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:45.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:45 smithi144 ceph-mon[35650]: from='client.? 172.21.15.5:0/956566178' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:45.297 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:45 smithi144 ceph-mon[35650]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T15:38:45.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:45 smithi193 ceph-mon[36001]: from='client.? 172.21.15.5:0/956566178' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:45.303 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:45 smithi193 ceph-mon[36001]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T15:38:45.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 ceph-mon[31559]: from='client.? 172.21.15.5:0/956566178' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T15:38:45.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 ceph-mon[31559]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T15:38:45.315 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a[31555]: 2024-05-14T15:38:45.206+0000 7f8439c5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:38:45.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a[31555]: 2024-05-14T15:38:45.206+0000 7f8439c5f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-14T15:38:45.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 podman[124376]: 2024-05-14 15:38:45.380672815 +0000 UTC m=+0.290048374 container died 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:38:46.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 podman[124376]: 2024-05-14 15:38:45.822624277 +0000 UTC m=+0.731999836 container cleanup 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:38:46.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:45 smithi005 bash[124376]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a 2024-05-14T15:38:46.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:May 14 15:38:46 smithi005 podman[124388]: 2024-05-14 15:38:46.057596265 +0000 UTC m=+0.667976407 container remove 1757122c7e39391cec0f4bf6ea721c918aeda3f34aa0b68adbca73417bcebdf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-14T15:38:46.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.a.service' 2024-05-14T15:38:46.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:38:46.367 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-14T15:38:46.367 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-14T15:38:46.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.b 2024-05-14T15:38:46.733 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:46 smithi144 systemd[1]: Stopping Ceph mon.b for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:47.047 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-b[35646]: 2024-05-14T15:38:46.731+0000 7ff97ba75700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:38:47.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-b[35646]: 2024-05-14T15:38:46.731+0000 7ff97ba75700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-14T15:38:47.048 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:46 smithi144 podman[73607]: 2024-05-14 15:38:46.875138919 +0000 UTC m=+0.270921461 container died baa0403d2691b85cd293fd6a7b74b353d1caa65f92143da9b536edf2cac07a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-b, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-14T15:38:47.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:47 smithi144 podman[73607]: 2024-05-14 15:38:47.257763006 +0000 UTC m=+0.653545549 container cleanup baa0403d2691b85cd293fd6a7b74b353d1caa65f92143da9b536edf2cac07a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-b, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-14T15:38:47.510 INFO:journalctl@ceph.mon.b.smithi144.stdout:May 14 15:38:47 smithi144 bash[73607]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-b 2024-05-14T15:38:47.638 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.b.service' 2024-05-14T15:38:47.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:38:47.679 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-14T15:38:47.680 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-14T15:38:47.680 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.c 2024-05-14T15:38:48.006 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:47 smithi193 systemd[1]: Stopping Ceph mon.c for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:48.269 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c[35995]: 2024-05-14T15:38:48.004+0000 7f954e4af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:38:48.269 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c[35995]: 2024-05-14T15:38:48.004+0000 7f954e4af700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-14T15:38:48.269 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 podman[73106]: 2024-05-14 15:38:48.24006288 +0000 UTC m=+0.365204026 container stop 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T15:38:48.270 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 podman[73106]: 2024-05-14 15:38:48.269083361 +0000 UTC m=+0.394224587 container died 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-14T15:38:48.908 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 podman[73106]: 2024-05-14 15:38:48.67224001 +0000 UTC m=+0.797381156 container cleanup 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T15:38:48.908 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 bash[73106]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c 2024-05-14T15:38:49.149 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.c.service' 2024-05-14T15:38:49.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:48 smithi193 podman[73117]: 2024-05-14 15:38:48.907631158 +0000 UTC m=+0.661773823 container remove 7f993fef78a0dca19eb40c55901be085c60b5ed054cacde2d06a26333938770e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-14T15:38:49.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:49 smithi193 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.c.service: Deactivated successfully. 2024-05-14T15:38:49.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:49 smithi193 systemd[1]: Stopped Ceph mon.c for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:38:49.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:May 14 15:38:49 smithi193 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mon.c.service: Consumed 20.277s CPU time. 2024-05-14T15:38:49.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:38:49.196 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-14T15:38:49.196 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-14T15:38:49.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x 2024-05-14T15:38:49.565 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:38:49 smithi005 systemd[1]: Stopping Ceph mgr.x for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:49.997 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:38:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x[31787]: 2024-05-14T15:38:49.635+0000 7fbc9900d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:38:49.998 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:38:49 smithi005 podman[124499]: 2024-05-14 15:38:49.677243631 +0000 UTC m=+0.274260614 container died 4921cde61604fab3fb17c9f66cda8d4e34b35d8cb9c74f487f6da62667d1ae6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T15:38:50.315 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:38:50 smithi005 podman[124499]: 2024-05-14 15:38:50.087144781 +0000 UTC m=+0.684161764 container cleanup 4921cde61604fab3fb17c9f66cda8d4e34b35d8cb9c74f487f6da62667d1ae6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e2b755a64f0d72176b57299d02e880cf0620d780, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-14T15:38:50.316 INFO:journalctl@ceph.mgr.x.smithi005.stdout:May 14 15:38:50 smithi005 bash[124499]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-x 2024-05-14T15:38:50.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-14T15:38:50.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:38:50.509 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-14T15:38:50.509 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-14T15:38:50.509 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.y 2024-05-14T15:38:50.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:38:50 smithi144 systemd[1]: Stopping Ceph mgr.y for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:51.189 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:38:50 smithi144 podman[73730]: 2024-05-14 15:38:50.893242425 +0000 UTC m=+0.187240057 container died 03589df7a821feba698be56207ea00af434302c8d17610774ff95e287ac66255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T15:38:51.547 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:38:51 smithi144 podman[73730]: 2024-05-14 15:38:51.296719299 +0000 UTC m=+0.590716931 container cleanup 03589df7a821feba698be56207ea00af434302c8d17610774ff95e287ac66255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T15:38:51.548 INFO:journalctl@ceph.mgr.y.smithi144.stdout:May 14 15:38:51 smithi144 bash[73730]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-y 2024-05-14T15:38:51.678 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-14T15:38:51.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:38:51.712 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-14T15:38:51.712 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-14T15:38:51.712 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.z 2024-05-14T15:38:52.043 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:38:51 smithi193 systemd[1]: Stopping Ceph mgr.z for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:52.303 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:38:52 smithi193 podman[73228]: 2024-05-14 15:38:52.041714003 +0000 UTC m=+0.141784563 container died a288382abd54f60b90ef243eee1ad431f8ae2489af2032b5a3c4fd8c0cf09949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:38:52.689 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:38:52 smithi193 podman[73228]: 2024-05-14 15:38:52.403391264 +0000 UTC m=+0.503461807 container cleanup a288382abd54f60b90ef243eee1ad431f8ae2489af2032b5a3c4fd8c0cf09949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-05-14T15:38:52.689 INFO:journalctl@ceph.mgr.z.smithi193.stdout:May 14 15:38:52 smithi193 bash[73228]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mgr-z 2024-05-14T15:38:52.831 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-14T15:38:52.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:38:52.864 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-14T15:38:52.864 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-14T15:38:52.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.0 2024-05-14T15:38:53.169 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:52 smithi005 systemd[1]: Stopping Ceph osd.0 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:38:53.565 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0[46433]: 2024-05-14T15:38:53.167+0000 7f37d81dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:38:53.566 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0[46433]: 2024-05-14T15:38:53.167+0000 7f37d81dd700 -1 osd.0 116 *** Got signal Terminated *** 2024-05-14T15:38:53.566 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0[46433]: 2024-05-14T15:38:53.167+0000 7f37d81dd700 -1 osd.0 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:38:59.053 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:58 smithi005 podman[124620]: 2024-05-14 15:38:58.745710976 +0000 UTC m=+5.683229347 container died 6d21e4de04528e96b400c5a2543f7d7161425513dc0c7dc3067af255f3aa3b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5) 2024-05-14T15:38:59.316 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:59 smithi005 podman[124620]: 2024-05-14 15:38:59.15229252 +0000 UTC m=+6.089810890 container cleanup 6d21e4de04528e96b400c5a2543f7d7161425513dc0c7dc3067af255f3aa3b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-14T15:38:59.316 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:59 smithi005 bash[124620]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0 2024-05-14T15:38:59.789 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:59 smithi005 podman[124641]: 2024-05-14 15:38:59.347460381 +0000 UTC m=+0.596919229 container remove 6d21e4de04528e96b400c5a2543f7d7161425513dc0c7dc3067af255f3aa3b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T15:39:00.066 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:59 smithi005 podman[124731]: 2024-05-14 15:38:59.691912305 +0000 UTC m=+0.018028675 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:00.066 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:38:59 smithi005 podman[124731]: 2024-05-14 15:38:59.806642643 +0000 UTC m=+0.132758992 container create fa444b1624c600af2a515b65504126a848bc7843dfe54772ebb4fc096910dbb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:00.380 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:39:00 smithi005 podman[124731]: 2024-05-14 15:39:00.105568454 +0000 UTC m=+0.431684815 container init fa444b1624c600af2a515b65504126a848bc7843dfe54772ebb4fc096910dbb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-14T15:39:00.381 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:39:00 smithi005 podman[124731]: 2024-05-14 15:39:00.109078155 +0000 UTC m=+0.435194519 container start fa444b1624c600af2a515b65504126a848bc7843dfe54772ebb4fc096910dbb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-14T15:39:00.381 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:39:00 smithi005 podman[124731]: 2024-05-14 15:39:00.209502852 +0000 UTC m=+0.535619223 container attach fa444b1624c600af2a515b65504126a848bc7843dfe54772ebb4fc096910dbb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-14T15:39:00.797 INFO:journalctl@ceph.osd.0.smithi005.stdout:May 14 15:39:00 smithi005 podman[124731]: 2024-05-14 15:39:00.437901197 +0000 UTC m=+0.764017557 container died fa444b1624c600af2a515b65504126a848bc7843dfe54772ebb4fc096910dbb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:01.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.0.service' 2024-05-14T15:39:01.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:01.140 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-14T15:39:01.141 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-14T15:39:01.141 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.1 2024-05-14T15:39:01.444 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:01 smithi144 systemd[1]: Stopping Ceph osd.1 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:01.797 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1[40230]: 2024-05-14T15:39:01.442+0000 7fbab9ce9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:01.798 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1[40230]: 2024-05-14T15:39:01.442+0000 7fbab9ce9700 -1 osd.1 116 *** Got signal Terminated *** 2024-05-14T15:39:01.798 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1[40230]: 2024-05-14T15:39:01.442+0000 7fbab9ce9700 -1 osd.1 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:07.290 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:06 smithi144 podman[73855]: 2024-05-14 15:39:06.991274237 +0000 UTC m=+5.654335598 container died 2adf885dae4643f4d767b45fa399ef55eabfaf88fb598c59460ff22dda082ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-05-14T15:39:07.547 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:07 smithi144 podman[73855]: 2024-05-14 15:39:07.40194971 +0000 UTC m=+6.065011077 container cleanup 2adf885dae4643f4d767b45fa399ef55eabfaf88fb598c59460ff22dda082ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-14T15:39:07.548 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:07 smithi144 bash[73855]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1 2024-05-14T15:39:07.970 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:07 smithi144 podman[73876]: 2024-05-14 15:39:07.645365962 +0000 UTC m=+0.645923470 container remove 2adf885dae4643f4d767b45fa399ef55eabfaf88fb598c59460ff22dda082ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T15:39:08.241 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:07 smithi144 podman[73970]: 2024-05-14 15:39:07.87441954 +0000 UTC m=+0.020058329 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:08.241 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:07 smithi144 podman[73970]: 2024-05-14 15:39:07.987945067 +0000 UTC m=+0.133583822 container create 9702e64037dd2c96f00a2f44f76590cb745c48ae85d50fa54744b8c507a8fb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T15:39:08.241 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:08 smithi144 podman[73970]: 2024-05-14 15:39:08.240448473 +0000 UTC m=+0.386087223 container init 9702e64037dd2c96f00a2f44f76590cb745c48ae85d50fa54744b8c507a8fb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-14T15:39:08.548 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:08 smithi144 podman[73970]: 2024-05-14 15:39:08.244245079 +0000 UTC m=+0.389883835 container start 9702e64037dd2c96f00a2f44f76590cb745c48ae85d50fa54744b8c507a8fb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-14T15:39:08.548 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:08 smithi144 podman[73970]: 2024-05-14 15:39:08.349129395 +0000 UTC m=+0.494768151 container attach 9702e64037dd2c96f00a2f44f76590cb745c48ae85d50fa54744b8c507a8fb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-14T15:39:09.047 INFO:journalctl@ceph.osd.1.smithi144.stdout:May 14 15:39:08 smithi144 podman[73970]: 2024-05-14 15:39:08.756822523 +0000 UTC m=+0.902461286 container died 9702e64037dd2c96f00a2f44f76590cb745c48ae85d50fa54744b8c507a8fb55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:09.430 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.1.service' 2024-05-14T15:39:09.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:09.471 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-14T15:39:09.471 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-14T15:39:09.471 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.2 2024-05-14T15:39:09.783 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:09 smithi193 systemd[1]: Stopping Ceph osd.2 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:10.046 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2[41149]: 2024-05-14T15:39:09.781+0000 7fa87d5e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:10.046 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2[41149]: 2024-05-14T15:39:09.781+0000 7fa87d5e0700 -1 osd.2 116 *** Got signal Terminated *** 2024-05-14T15:39:10.046 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2[41149]: 2024-05-14T15:39:09.781+0000 7fa87d5e0700 -1 osd.2 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:15.650 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:15 smithi193 podman[73352]: 2024-05-14 15:39:15.365736902 +0000 UTC m=+5.697663343 container died 54d65e28ca4d374d961782a66813f0db409cb208f1e6d5c6cf5f4db3ca76d1cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:39:16.018 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:15 smithi193 podman[73352]: 2024-05-14 15:39:15.741119919 +0000 UTC m=+6.073046357 container cleanup 54d65e28ca4d374d961782a66813f0db409cb208f1e6d5c6cf5f4db3ca76d1cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T15:39:16.018 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:15 smithi193 bash[73352]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2 2024-05-14T15:39:16.304 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73380]: 2024-05-14 15:39:16.017786961 +0000 UTC m=+0.649296626 container remove 54d65e28ca4d374d961782a66813f0db409cb208f1e6d5c6cf5f4db3ca76d1cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:16.664 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73468]: 2024-05-14 15:39:16.277081533 +0000 UTC m=+0.016938799 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:16.664 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73468]: 2024-05-14 15:39:16.402306635 +0000 UTC m=+0.142163878 container create 8b878d941e0052612279bba816f2510aeef71e2101739f0557d184a2ee3dcc77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-14T15:39:16.664 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73468]: 2024-05-14 15:39:16.663568982 +0000 UTC m=+0.403426233 container init 8b878d941e0052612279bba816f2510aeef71e2101739f0557d184a2ee3dcc77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T15:39:16.965 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73468]: 2024-05-14 15:39:16.66690715 +0000 UTC m=+0.406764408 container start 8b878d941e0052612279bba816f2510aeef71e2101739f0557d184a2ee3dcc77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-14T15:39:16.966 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73468]: 2024-05-14 15:39:16.806140262 +0000 UTC m=+0.545997523 container attach 8b878d941e0052612279bba816f2510aeef71e2101739f0557d184a2ee3dcc77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-14T15:39:17.268 INFO:journalctl@ceph.osd.2.smithi193.stdout:May 14 15:39:16 smithi193 podman[73468]: 2024-05-14 15:39:16.965527736 +0000 UTC m=+0.705384988 container died 8b878d941e0052612279bba816f2510aeef71e2101739f0557d184a2ee3dcc77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-14T15:39:17.603 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.2.service' 2024-05-14T15:39:17.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:17.649 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-14T15:39:17.650 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-14T15:39:17.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.3 2024-05-14T15:39:17.976 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:17 smithi005 systemd[1]: Stopping Ceph osd.3 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:18.262 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:17 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:39:17.975+0000 7f0729b4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:18.262 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:17 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:39:17.975+0000 7f0729b4a700 -1 osd.3 116 *** Got signal Terminated *** 2024-05-14T15:39:18.262 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:17 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:39:17.975+0000 7f0729b4a700 -1 osd.3 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:20.547 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:20 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:20.206+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:21.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:21 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:21.025+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:21.547 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:21 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:21.246+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:21.777 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:21.329+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:21.815 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:21 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:39:21.509+0000 7f07220f8700 -1 osd.3 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.780731+0000 front 2024-05-14T15:38:56.780777+0000 (oldest deadline 2024-05-14T15:39:20.880898+0000) 2024-05-14T15:39:22.053 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:21.776+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:22.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:22 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:22.004+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:22.547 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:22 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:22.198+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:22.803 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:22.319+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:22.815 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:22 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3[52935]: 2024-05-14T15:39:22.494+0000 7f07220f8700 -1 osd.3 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.780731+0000 front 2024-05-14T15:38:56.780777+0000 (oldest deadline 2024-05-14T15:39:20.880898+0000) 2024-05-14T15:39:23.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:22.818+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:23.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:23 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:23.016+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:23.547 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:23 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:23.215+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:23.753 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:23 smithi005 podman[124852]: 2024-05-14 15:39:23.443086827 +0000 UTC m=+5.590085553 container died f9c261300a33b96515342f66b1a542893b06a3718014c36541728f8765e93214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True) 2024-05-14T15:39:23.803 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:23.332+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:24.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:23 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:23.974+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:24.066 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:23 smithi005 podman[124852]: 2024-05-14 15:39:23.95245639 +0000 UTC m=+6.099455107 container cleanup f9c261300a33b96515342f66b1a542893b06a3718014c36541728f8765e93214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-14T15:39:24.066 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:23 smithi005 bash[124852]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3 2024-05-14T15:39:24.296 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:23.838+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:24.547 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:24 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:24.261+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:24.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:24.295+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:24.565 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:24 smithi005 podman[124882]: 2024-05-14 15:39:24.204243079 +0000 UTC m=+0.759864995 container remove f9c261300a33b96515342f66b1a542893b06a3718014c36541728f8765e93214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T15:39:24.916 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:24 smithi005 podman[124969]: 2024-05-14 15:39:24.545496678 +0000 UTC m=+0.017760585 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:24.916 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:24 smithi005 podman[124969]: 2024-05-14 15:39:24.671890962 +0000 UTC m=+0.144154845 container create 12e058cadc22ab856c8bf96ad959962a69054b2e6c6ce08e122321a0ffc903b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True) 2024-05-14T15:39:24.916 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:24 smithi005 podman[124969]: 2024-05-14 15:39:24.916027014 +0000 UTC m=+0.388290899 container init 12e058cadc22ab856c8bf96ad959962a69054b2e6c6ce08e122321a0ffc903b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:39:25.223 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:24 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:24.929+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:25.223 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:24 smithi005 podman[124969]: 2024-05-14 15:39:24.919696528 +0000 UTC m=+0.391960429 container start 12e058cadc22ab856c8bf96ad959962a69054b2e6c6ce08e122321a0ffc903b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-14T15:39:25.223 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:25 smithi005 podman[124969]: 2024-05-14 15:39:25.016151986 +0000 UTC m=+0.488415899 container attach 12e058cadc22ab856c8bf96ad959962a69054b2e6c6ce08e122321a0ffc903b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-14T15:39:25.250 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:24.877+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:25.547 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:25 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:25.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:25.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:25.249+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:25.565 INFO:journalctl@ceph.osd.3.smithi005.stdout:May 14 15:39:25 smithi005 podman[124969]: 2024-05-14 15:39:25.222859082 +0000 UTC m=+0.695122985 container died 12e058cadc22ab856c8bf96ad959962a69054b2e6c6ce08e122321a0ffc903b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T15:39:25.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.3.service' 2024-05-14T15:39:26.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:26.026 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-14T15:39:26.026 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-14T15:39:26.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.4 2024-05-14T15:39:26.278 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:25.832+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:26.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:25 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:25.927+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:26.340 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:26 smithi144 systemd[1]: Stopping Ceph osd.4 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:26.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:26.277+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:26.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:26 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:26.342+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:26.798 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:26 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:39:26.339+0000 7f68d5db2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:26.798 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:26 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:39:26.339+0000 7f68d5db2700 -1 osd.4 116 *** Got signal Terminated *** 2024-05-14T15:39:26.798 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:26 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:39:26.339+0000 7f68d5db2700 -1 osd.4 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:27.053 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:26.801+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:27.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:26 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:26.973+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:27.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:27.235+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:27.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:27 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:27.359+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:27.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:27 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:27.359+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:28.053 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:27.778+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:28.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:27 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:27.991+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:28.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:28 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:28.261+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:28.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:28 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:28.489+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:28.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:28 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:28.342+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:28.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:28 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:28.342+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:29.253 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:28 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:28.827+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:29.312 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:28 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:28.996+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:29.312 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:29 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:29.077+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:29.547 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:29 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:39:29.124+0000 7f68d2368700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.343537+0000 front 2024-05-14T15:39:05.343570+0000 (oldest deadline 2024-05-14T15:39:28.243773+0000) 2024-05-14T15:39:29.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:29 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:29.293+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:29.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:29 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:29.293+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:29.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:29 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:29.252+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:29.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:29 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:29.448+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:30.047 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:29 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:29.592+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:30.231 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:29 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:29.876+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:30.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:30 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:30.044+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:30.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:29 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:29.952+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:30.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:29 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:29.952+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:30.547 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:39:30.092+0000 7f68d2368700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.343537+0000 front 2024-05-14T15:39:05.343570+0000 (oldest deadline 2024-05-14T15:39:28.243773+0000) 2024-05-14T15:39:30.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:30.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:30.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:30.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:30.553 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:30.452+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:30.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:30.229+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:30.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:30.229+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:31.047 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:30 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:30.621+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:31.241 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:30 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:30.917+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:31.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:31 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:31.085+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:31.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:30 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:30.905+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:31.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:30 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:30.905+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:31.547 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:31 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4[47007]: 2024-05-14T15:39:31.116+0000 7f68d2368700 -1 osd.4 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.343537+0000 front 2024-05-14T15:39:05.343570+0000 (oldest deadline 2024-05-14T15:39:28.243773+0000) 2024-05-14T15:39:31.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:31 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:31.299+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:31.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:31 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:31.299+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:31.553 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:31 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:31.485+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:31.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:31 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:31.240+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:31.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:31 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:31.240+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:32.048 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:31 smithi144 podman[74080]: 2024-05-14 15:39:31.75998744 +0000 UTC m=+5.538669423 container died 86e2a6259ff75e0f05ab01d7564c07a6c008e91f09a21750800a7e3f3fc05144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:39:32.048 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:31 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:31.619+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:32.248 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:31 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:31.962+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:32.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:32 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:32.110+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:32.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:31 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:31.937+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:32.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:31 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:31.937+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:32.331 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 podman[74080]: 2024-05-14 15:39:32.147919787 +0000 UTC m=+5.926601767 container cleanup 86e2a6259ff75e0f05ab01d7564c07a6c008e91f09a21750800a7e3f3fc05144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-14T15:39:32.332 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 bash[74080]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4 2024-05-14T15:39:32.501 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:32 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:32.247+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:32.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:32 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:32.247+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:32.662 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 podman[74095]: 2024-05-14 15:39:32.399438528 +0000 UTC m=+0.633015312 container remove 86e2a6259ff75e0f05ab01d7564c07a6c008e91f09a21750800a7e3f3fc05144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-14T15:39:32.663 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:32 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:32.331+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:32.663 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:32 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:32.331+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:32.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:32 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:32.500+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:32.994 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:32 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:32.662+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:32.995 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 podman[74181]: 2024-05-14 15:39:32.620208023 +0000 UTC m=+0.017383515 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:32.995 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 podman[74181]: 2024-05-14 15:39:32.733701201 +0000 UTC m=+0.130876693 container create fd5a5a2cc9df0eea962e2ff5a8f59b50c59ad5c40bde1b7ba77732834f1d5bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T15:39:32.995 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 podman[74181]: 2024-05-14 15:39:32.994473479 +0000 UTC m=+0.391648973 container init fd5a5a2cc9df0eea962e2ff5a8f59b50c59ad5c40bde1b7ba77732834f1d5bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-14T15:39:33.279 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:32 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:32.993+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:33.279 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:32 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:32.993+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:33.290 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:32 smithi144 podman[74181]: 2024-05-14 15:39:32.997671713 +0000 UTC m=+0.394847206 container start fd5a5a2cc9df0eea962e2ff5a8f59b50c59ad5c40bde1b7ba77732834f1d5bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:39:33.290 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:33 smithi144 podman[74181]: 2024-05-14 15:39:33.094704503 +0000 UTC m=+0.491880000 container attach fd5a5a2cc9df0eea962e2ff5a8f59b50c59ad5c40bde1b7ba77732834f1d5bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:33.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:33 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:33.101+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:33.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:32 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:32.921+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:33.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:32 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:32.921+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:33.547 INFO:journalctl@ceph.osd.4.smithi144.stdout:May 14 15:39:33 smithi144 podman[74181]: 2024-05-14 15:39:33.290580709 +0000 UTC m=+0.687756202 container died fd5a5a2cc9df0eea962e2ff5a8f59b50c59ad5c40bde1b7ba77732834f1d5bbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True) 2024-05-14T15:39:33.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:33 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:33.340+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:33.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:33 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:33.341+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:33.553 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:33.465+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:33.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:33.278+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:33.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:33.278+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:33.857 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:33 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:33.648+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:33.880 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.4.service' 2024-05-14T15:39:33.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:33.918 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-14T15:39:33.918 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-14T15:39:33.919 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.5 2024-05-14T15:39:34.231 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:33 smithi193 systemd[1]: Stopping Ceph osd.5 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:34.231 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:33.965+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:34.231 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:33 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:33.965+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:34.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:34.060+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:34.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:33 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:33.953+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:34.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:33 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:33.953+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:34.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:34.228+0000 7f5dc3fba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:34.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:34.228+0000 7f5dc3fba700 -1 osd.5 116 *** Got signal Terminated *** 2024-05-14T15:39:34.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:34.228+0000 7f5dc3fba700 -1 osd.5 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:34.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:34.281+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:34.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:34.281+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:34.612 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:34 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:34.314+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:34.612 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:34 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:34.314+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:34.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:34.500+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:35.047 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:34 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:34.611+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:35.296 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:34.918+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:35.297 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:34 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:34.918+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:35.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:35 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:35.091+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:35.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:34.908+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:35.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:34.908+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:35.553 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:35 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:35.484+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:35.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:35 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:35.295+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:35.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:35 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:35.295+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:35.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:35 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:35.601+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:35.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:35 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:35.362+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:35.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:35 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:35.362+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:36.253 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:36 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:36.252+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:36.253 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:35 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:35.945+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:36.253 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:35 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:35.945+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:36.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:36.061+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:36.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:35 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:35.880+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:36.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:35 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:35.880+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:36.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:36 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:36.252+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:36.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:36 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:36.449+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:36.631 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:36 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:36.355+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:36.632 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:36 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:36.355+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:37.047 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:36 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:36.631+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:37.208 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:36 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:36.938+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:37.209 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:36 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:36.938+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:37.209 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:36 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:36.938+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:37.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:37 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:37.104+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:37.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:36.872+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:37.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:36.872+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:37.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:37.207+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:37.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:37.207+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:37.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:37.411+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:37.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:37.411+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:37.669 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:37 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:37.391+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:37.669 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:37 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:37.391+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:38.047 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:37 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:37.668+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:38.145 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:37 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:37.858+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:38.146 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:37 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:37.858+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:38.190 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:37.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:38.190 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:37.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:38.191 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:37 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:37.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:38.441 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:38.189+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:38.441 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:38.189+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:38.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:38.145+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:38.670 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:38 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:38.412+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:38.670 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:38 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:38.412+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:38.670 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:38 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:38.412+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:38.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:38.439+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:38.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:38.439+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:38.949 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:38 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:38.670+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:39.135 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:38.879+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:39.135 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:38.879+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:39.135 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:38 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:38.879+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:39.170 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:38.916+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:39.170 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:38.916+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:39.170 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:38 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:38.916+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:39.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:39.402+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:39.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:39.402+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:39.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:39.169+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:57.023114+0000 front 2024-05-14T15:38:57.023143+0000 (oldest deadline 2024-05-14T15:39:21.123017+0000) 2024-05-14T15:39:39.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:39.169+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.125037+0000 front 2024-05-14T15:39:05.125083+0000 (oldest deadline 2024-05-14T15:39:29.825176+0000) 2024-05-14T15:39:39.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5[48075]: 2024-05-14T15:39:39.169+0000 7f5dbc568700 -1 osd.5 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.626632+0000 front 2024-05-14T15:39:12.626660+0000 (oldest deadline 2024-05-14T15:39:38.526799+0000) 2024-05-14T15:39:39.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:39.134+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:39.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:39 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:39.637+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:39.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:39 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:39.637+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:39.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:39 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:39.431+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:39.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:39 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:39.431+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:39.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:39 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:39.431+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:39.956 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:39 smithi193 podman[73573]: 2024-05-14 15:39:39.697166693 +0000 UTC m=+5.584309507 container died b9deea07315f0f5c5c7973359f1dd3fe7f0b8db5d31c6f3975c3958a48da831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-14T15:39:39.956 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:39.917+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:39.956 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:39.917+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:39.956 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:39 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:39.917+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:40.157 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:39.850+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:40.157 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:39.850+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:40.157 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:39 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:39.850+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:40.265 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 podman[73573]: 2024-05-14 15:39:40.047936131 +0000 UTC m=+5.935078944 container cleanup b9deea07315f0f5c5c7973359f1dd3fe7f0b8db5d31c6f3975c3958a48da831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True) 2024-05-14T15:39:40.265 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 bash[73573]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5 2024-05-14T15:39:40.553 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 podman[73591]: 2024-05-14 15:39:40.264856194 +0000 UTC m=+0.561678193 container remove b9deea07315f0f5c5c7973359f1dd3fe7f0b8db5d31c6f3975c3958a48da831e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:39:40.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:40 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:40.425+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:40.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:40 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:40.425+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:40.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:40.156+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:40.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:40.156+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:40.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:40.663+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:40.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:40.663+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:40.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:40.481+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:40.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:40.481+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:40.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:40 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:40.481+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:40.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 podman[73680]: 2024-05-14 15:39:40.657538184 +0000 UTC m=+0.102794518 container create 042e8567383462e402daa787323cdb5077ec97ea34d994359f8dead650274098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-14T15:39:40.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 podman[73680]: 2024-05-14 15:39:40.571434625 +0000 UTC m=+0.016690972 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:40.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 podman[73680]: 2024-05-14 15:39:40.910808122 +0000 UTC m=+0.356064448 container init 042e8567383462e402daa787323cdb5077ec97ea34d994359f8dead650274098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-14T15:39:41.118 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:40.831+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:41.118 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:40.831+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:41.118 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:40 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:40.831+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:41.209 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:40 smithi193 podman[73680]: 2024-05-14 15:39:40.9143189 +0000 UTC m=+0.359575235 container start 042e8567383462e402daa787323cdb5077ec97ea34d994359f8dead650274098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T15:39:41.209 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:41 smithi193 podman[73680]: 2024-05-14 15:39:41.008376069 +0000 UTC m=+0.453632418 container attach 042e8567383462e402daa787323cdb5077ec97ea34d994359f8dead650274098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-14T15:39:41.209 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:40 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:40.926+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:41.210 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:40 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:40.926+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:41.210 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:40 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:40.926+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:41.537 INFO:journalctl@ceph.osd.5.smithi193.stdout:May 14 15:39:41 smithi193 podman[73680]: 2024-05-14 15:39:41.208916268 +0000 UTC m=+0.654172613 container died 042e8567383462e402daa787323cdb5077ec97ea34d994359f8dead650274098 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-14T15:39:41.538 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:41 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:41.457+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:41.538 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:41 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:41.457+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:41.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:41.117+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:41.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:41.117+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:41.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:41 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:41.667+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:41.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:41 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:41.667+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:41.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:41 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:41.490+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:41.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:41 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:41.490+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:41.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:41 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:41.490+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:41.817 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.5.service' 2024-05-14T15:39:41.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:41.856 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-14T15:39:41.856 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-14T15:39:41.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.6 2024-05-14T15:39:42.053 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:41 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:41.907+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:42.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:41 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:41.907+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:42.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:41 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:41.907+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:42.086 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:41 smithi005 systemd[1]: Stopping Ceph osd.6 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:42.086 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:41.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:42.086 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:41.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:42.086 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:41 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:41.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:42.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:42.083+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:42.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:42.083+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:42.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:42.224+0000 7fe640a1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:42.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:42.224+0000 7fe640a1c700 -1 osd.6 116 *** Got signal Terminated *** 2024-05-14T15:39:42.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:42.224+0000 7fe640a1c700 -1 osd.6 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:42.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:42.638+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:42.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:42.638+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:42.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:42.440+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:42.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:42.440+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:42.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:42 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:42.440+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:42.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:42.489+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:42.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:42.489+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:43.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:42.795+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:43.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:42.795+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:43.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:42.795+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:43.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:42.905+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:43.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:42.905+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:43.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:42 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:42.905+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:43.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:43.087+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:43.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:43.087+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:43.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:43.589+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:43.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:43.589+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:43.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:43.469+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:43.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:43.469+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:43.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:43 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:43.469+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:43.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:43.536+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:43.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:43.536+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:44.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:43.806+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:44.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:43.806+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:44.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:43 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:43.806+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:44.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:43.920+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:44.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:43.920+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:44.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:43 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:43.920+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:44.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:44 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:44.071+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:44.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:44 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:44.071+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:44.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:44.564+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:44.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:44.564+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:44.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:44.452+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:44.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:44.452+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:44.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:44.452+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:44.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:44.452+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:44.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:44 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:44.498+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:44.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:44 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:44.498+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:45.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:44 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:44.757+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:45.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:44 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:44.757+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:45.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:44 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:44.757+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:45.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:44 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:44.875+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:45.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:44 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:44.875+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:45.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:44 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:44.875+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:45.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:45.093+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:45.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:45.093+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:45.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:45.093+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.162617+0000 front 2024-05-14T15:39:22.162651+0000 (oldest deadline 2024-05-14T15:39:44.462771+0000) 2024-05-14T15:39:45.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:45.561+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:45.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:45.561+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:45.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:45.561+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:45.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:45.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:45.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:45.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:45.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:45.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:45.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:45 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:45.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:45.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:45.487+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:45.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:45.487+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:46.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:45.778+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:46.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:45.778+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:46.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:45.778+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:46.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:45 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:45.778+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:46.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:45.923+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:46.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:45.923+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:46.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:45 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:45.923+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:46.565 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:46.132+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:46.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:46.132+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:46.566 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:46.132+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.162617+0000 front 2024-05-14T15:39:22.162651+0000 (oldest deadline 2024-05-14T15:39:44.462771+0000) 2024-05-14T15:39:46.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:46.551+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:46.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:46.551+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:46.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:46.551+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:46.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:46.412+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:46.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:46.412+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:46.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:46.413+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:46.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:46 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:46.413+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:46.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:46.448+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:46.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:46.448+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:46.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:46.448+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:47.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:46.761+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:47.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:46.761+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:47.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:46.761+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:47.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:46 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:46.761+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:47.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:46.911+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:47.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:46.911+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:47.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:46 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:46.911+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:47.361 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:47.126+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.157394+0000 front 2024-05-14T15:39:03.157500+0000 (oldest deadline 2024-05-14T15:39:28.457484+0000) 2024-05-14T15:39:47.361 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:47.126+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:14.759893+0000 front 2024-05-14T15:39:14.759972+0000 (oldest deadline 2024-05-14T15:39:40.060081+0000) 2024-05-14T15:39:47.362 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6[59728]: 2024-05-14T15:39:47.126+0000 7fe63cfd2700 -1 osd.6 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.162617+0000 front 2024-05-14T15:39:22.162651+0000 (oldest deadline 2024-05-14T15:39:44.462771+0000) 2024-05-14T15:39:47.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:47.556+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:47.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:47.556+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:47.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:47.556+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:47.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:47.372+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:47.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:47.372+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:47.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:47.372+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:47.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:47.372+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:47.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:47.481+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:47.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:47.481+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:47.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:47.481+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:47.960 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:47 smithi005 podman[125075]: 2024-05-14 15:39:47.666384534 +0000 UTC m=+5.608217387 container died 7c9cce180cd6e144fc834b940a88866ba2738a05524439dd7a2b1df5c332d97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:39:47.960 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:47.739+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:47.961 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:47.739+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:47.961 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:47.739+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:47.961 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:47 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:47.739+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:48.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:47.911+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:48.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:47.911+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:48.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:47 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:47.911+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:48.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:48 smithi005 podman[125075]: 2024-05-14 15:39:48.151595221 +0000 UTC m=+6.093428073 container cleanup 7c9cce180cd6e144fc834b940a88866ba2738a05524439dd7a2b1df5c332d97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:48.315 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:48 smithi005 bash[125075]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6 2024-05-14T15:39:48.766 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:48 smithi005 podman[125098]: 2024-05-14 15:39:48.428313599 +0000 UTC m=+0.760870906 container remove 7c9cce180cd6e144fc834b940a88866ba2738a05524439dd7a2b1df5c332d97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:39:48.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:48.577+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:48.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:48.577+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:48.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:48.577+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:48.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:48.362+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:48.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:48.362+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:48.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:48.362+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:48.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:48 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:48.362+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:48.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:48.440+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:48.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:48.440+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:48.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:48.440+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:49.025 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:48 smithi005 podman[125191]: 2024-05-14 15:39:48.770731092 +0000 UTC m=+0.093086131 container create 644605f5c6c86d3a062b0663ef645107d193a4a38f9dc26283f9c63febbcb6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=) 2024-05-14T15:39:49.025 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:48 smithi005 podman[125191]: 2024-05-14 15:39:48.695300728 +0000 UTC m=+0.017655745 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:49.025 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:48.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:49.026 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:48.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:49.026 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:48.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:49.026 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:48 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:48.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:49.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:48.886+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:49.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:48.886+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:49.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:48 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:48.886+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:49.316 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:49 smithi005 podman[125191]: 2024-05-14 15:39:49.024010612 +0000 UTC m=+0.346365610 container init 644605f5c6c86d3a062b0663ef645107d193a4a38f9dc26283f9c63febbcb6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T15:39:49.316 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:49 smithi005 podman[125191]: 2024-05-14 15:39:49.028442248 +0000 UTC m=+0.350797256 container start 644605f5c6c86d3a062b0663ef645107d193a4a38f9dc26283f9c63febbcb6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T15:39:49.316 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:49 smithi005 podman[125191]: 2024-05-14 15:39:49.124346239 +0000 UTC m=+0.446701244 container attach 644605f5c6c86d3a062b0663ef645107d193a4a38f9dc26283f9c63febbcb6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-14T15:39:49.621 INFO:journalctl@ceph.osd.6.smithi005.stdout:May 14 15:39:49 smithi005 podman[125191]: 2024-05-14 15:39:49.338966375 +0000 UTC m=+0.661321382 container died 644605f5c6c86d3a062b0663ef645107d193a4a38f9dc26283f9c63febbcb6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:39:49.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:49.592+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:49.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:49.592+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:49.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:49.592+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:49.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:49.376+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:49.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:49.376+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:49.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:49.376+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:49.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:49 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:49.376+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:49.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:49.452+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:49.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:49.452+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:49.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:49.452+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:49.973 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:49.728+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:49.973 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:49.728+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:49.973 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:49.728+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:49.973 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:49 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:49.728+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:49.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.6.service' 2024-05-14T15:39:50.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:50.043 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-14T15:39:50.043 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-14T15:39:50.044 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.7 2024-05-14T15:39:50.297 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 systemd[1]: Stopping Ceph osd.7 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:50.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:49.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:50.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:49.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:50.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:49 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:49.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:50.599 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:50.333+0000 7f9eaf529700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:50.599 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:50.333+0000 7f9eaf529700 -1 osd.7 116 *** Got signal Terminated *** 2024-05-14T15:39:50.599 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:50.333+0000 7f9eaf529700 -1 osd.7 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:50.599 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:50.420+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:50.600 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:50.420+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:50.600 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:50.420+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:50.600 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:50.420+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:50.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:50.421+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:50.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:50.421+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:50.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:50.421+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:51.047 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:50.597+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:51.048 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:50.597+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:51.048 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:50 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:50.597+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:51.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:50.745+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:51.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:50.745+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:51.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:50.745+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:51.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:50 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:50.745+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:51.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:50.861+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:51.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:50.861+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:51.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:50 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:50.861+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:51.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:51.569+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:51.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:51.569+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:51.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:51.569+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:51.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:51.429+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:51.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:51.429+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:51.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:51.429+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:51.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:51.429+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:51.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:51.406+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:51.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:51.406+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:51.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:51.406+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:52.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:51.793+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:52.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:51.793+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:52.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:51.793+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:52.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:51 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:51.793+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:52.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:51.854+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:52.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:51.854+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:52.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:51 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:51.854+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:52.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:52.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:52.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:52.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:52.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:52.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:52.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:52.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:52.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:52.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:52.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:52.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:52.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:52 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:52.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:52.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:52.441+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:52.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:52.441+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:52.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:52.441+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:52.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:52.441+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:53.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:52 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:52.811+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:53.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:52 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:52.811+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:53.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:52 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:52.811+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:53.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:52 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:52.811+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:53.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:52.897+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:53.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:52.897+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:53.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:52.897+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:53.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:52 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:52.897+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:53.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:53.571+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:53.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:53.571+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:53.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:53.571+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:53.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:53.571+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:29.791550+0000 front 2024-05-14T15:39:29.791525+0000 (oldest deadline 2024-05-14T15:39:53.291653+0000) 2024-05-14T15:39:53.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:53.458+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:53.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:53.458+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:53.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:53.458+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:53.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:53.458+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:53.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:53 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:53.458+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:53.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:53.464+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:53.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:53.464+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:53.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:53.464+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:53.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:53.464+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:54.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:53.773+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:54.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:53.773+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:54.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:53.774+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:54.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:53.774+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:54.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:53 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:53.774+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:39:54.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:53.872+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:54.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:53.872+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:54.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:53.872+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:54.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:53 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:53.872+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:54.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:54.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.388074+0000 front 2024-05-14T15:39:03.387993+0000 (oldest deadline 2024-05-14T15:39:29.288227+0000) 2024-05-14T15:39:54.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:54.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:13.389416+0000 front 2024-05-14T15:39:13.389342+0000 (oldest deadline 2024-05-14T15:39:38.689472+0000) 2024-05-14T15:39:54.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:54.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:18.689980+0000 front 2024-05-14T15:39:18.690062+0000 (oldest deadline 2024-05-14T15:39:44.590067+0000) 2024-05-14T15:39:54.798 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7[53960]: 2024-05-14T15:39:54.607+0000 7f9ea7ad7700 -1 osd.7 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:29.791550+0000 front 2024-05-14T15:39:29.791525+0000 (oldest deadline 2024-05-14T15:39:53.291653+0000) 2024-05-14T15:39:54.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:54.498+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:54.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:54.498+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:54.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:54.498+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:54.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:54.498+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:54.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:54 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:54.498+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:54.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:54.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:54.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:54.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:54.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:54.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:54.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:54.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:55.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:54 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:54.790+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:55.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:54 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:54.790+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:55.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:54 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:54.790+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:55.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:54 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:54.790+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:55.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:54 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:54.790+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:39:55.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:54.842+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:55.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:54.842+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:55.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:54.842+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:55.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:54 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:54.842+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:55.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:55 smithi144 podman[74299]: 2024-05-14 15:39:55.73254132 +0000 UTC m=+5.502385416 container died c444e109f29806dcbdfa6af284dbe5db9d8a26bde6d5374a9aafcef71309c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-14T15:39:55.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:55.486+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:55.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:55.486+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:55.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:55.486+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:55.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:55.486+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:55.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:55.486+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:55.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:55.392+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:55.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:55.392+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:55.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:55.392+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:55.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:55.392+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:56.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:55.884+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:56.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:55.884+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:56.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:55.884+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:56.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:55 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:55.884+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:56.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:55 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:55.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:56.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:55 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:55.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:56.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:55 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:55.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:56.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:55 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:55.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:56.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:55 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:55.830+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:39:56.378 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:56 smithi144 podman[74299]: 2024-05-14 15:39:56.142483663 +0000 UTC m=+5.912327749 container cleanup c444e109f29806dcbdfa6af284dbe5db9d8a26bde6d5374a9aafcef71309c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-14T15:39:56.378 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:56 smithi144 bash[74299]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7 2024-05-14T15:39:56.797 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:56 smithi144 podman[74314]: 2024-05-14 15:39:56.377470036 +0000 UTC m=+0.640991395 container remove c444e109f29806dcbdfa6af284dbe5db9d8a26bde6d5374a9aafcef71309c8cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-14T15:39:56.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:56.500+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:56.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:56.500+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:56.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:56.500+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:56.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:56.500+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:56.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:56 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:56.500+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:56.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:56.437+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:56.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:56.437+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:56.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:56.437+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:56.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:56.437+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:57.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:56 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:56.804+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:57.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:56 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:56.804+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:57.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:56 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:56.804+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:57.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:56 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:56.804+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:57.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:56 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:56.804+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:39:57.114 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:56 smithi144 podman[74406]: 2024-05-14 15:39:56.739717775 +0000 UTC m=+0.017426901 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:39:57.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:56 smithi144 podman[74406]: 2024-05-14 15:39:56.861841263 +0000 UTC m=+0.139550420 container create 7e81a75c6c203f31d88d26bbd04183552507370ac71d935ab4594848a325589c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-14T15:39:57.115 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:57 smithi144 podman[74406]: 2024-05-14 15:39:57.114310639 +0000 UTC m=+0.392019755 container init 7e81a75c6c203f31d88d26bbd04183552507370ac71d935ab4594848a325589c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:39:57.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:56.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:57.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:56.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:57.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:56.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:57.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:56 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:56.899+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:57.389 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:57 smithi144 podman[74406]: 2024-05-14 15:39:57.117640617 +0000 UTC m=+0.395349733 container start 7e81a75c6c203f31d88d26bbd04183552507370ac71d935ab4594848a325589c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-14T15:39:57.389 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:57 smithi144 podman[74406]: 2024-05-14 15:39:57.214292276 +0000 UTC m=+0.492001406 container attach 7e81a75c6c203f31d88d26bbd04183552507370ac71d935ab4594848a325589c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-14T15:39:57.718 INFO:journalctl@ceph.osd.7.smithi144.stdout:May 14 15:39:57 smithi144 podman[74406]: 2024-05-14 15:39:57.423018961 +0000 UTC m=+0.700728079 container died 7e81a75c6c203f31d88d26bbd04183552507370ac71d935ab4594848a325589c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-14T15:39:57.719 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:57.455+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:57.719 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:57.455+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:57.719 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:57.455+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:57.719 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:57.455+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:57.719 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:57 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:57.455+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:57.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:57.458+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:57.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:57.458+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:57.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:57.458+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:57.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:57.458+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:58.039 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.7.service' 2024-05-14T15:39:58.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:57 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:57.798+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:58.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:57 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:57.798+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:58.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:57 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:57.798+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:58.067 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:57 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:57.798+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:58.067 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:57 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:57.798+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:39:58.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:39:58.075 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-14T15:39:58.076 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-14T15:39:58.076 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.8 2024-05-14T15:39:58.303 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 systemd[1]: Stopping Ceph osd.8 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:39:58.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:57.887+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:58.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:57.887+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:58.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:57.887+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:58.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:57 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:57.887+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:58.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:58.463+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:58.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:58.463+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:58.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:58.463+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:58.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:58.463+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:58.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:58 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:58.463+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:58.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.389+0000 7fd0ee2a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:39:58.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.389+0000 7fd0ee2a5700 -1 osd.8 116 *** Got signal Terminated *** 2024-05-14T15:39:58.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.389+0000 7fd0ee2a5700 -1 osd.8 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:39:58.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:58.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:58.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:58.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:58.416+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:39:59.081 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:58.816+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:39:59.081 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:58.816+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:39:59.082 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:58.816+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:39:59.082 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:58.816+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:39:59.082 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:58 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:58.816+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:39:59.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:58.912+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:39:59.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:58.912+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:39:59.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:58.912+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:39:59.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:58 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:58.912+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:39:59.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:59.423+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:39:59.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:59.423+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:39:59.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:59.423+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:39:59.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:59.423+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:39:59.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:39:59 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:39:59.423+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:39:59.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:59.443+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:39:59.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:59.443+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:39:59.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:59.443+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:39:59.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:39:59.443+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:40:00.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:59.921+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:00.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:59.921+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:00.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:59.921+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:00.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:39:59 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:39:59.921+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:00.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:59 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:59.827+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:00.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:59 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:59.827+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:00.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:59 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:59.827+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:00.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:59 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:59.827+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:00.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:39:59 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:39:59.827+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:00.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:00.464+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:00.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:00.464+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:00.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:00.464+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:00.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:00.464+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:00.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:00 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:00.464+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:00.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:00.400+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:40:00.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:00.400+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:40:00.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:00.400+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:40:00.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:00.400+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:40:01.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:00.809+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:01.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:00.809+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:01.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:00.809+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:01.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:00.809+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:01.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:00.809+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:01.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:00 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:00.809+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:01.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:00.880+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:01.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:00.880+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:01.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:00.880+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:01.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:00.880+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:01.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:01.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:01.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:01.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:01.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:01.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:01.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:01.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:01.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:01 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:01.450+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:01.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:01.388+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:40:01.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:01.388+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:40:01.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:01.388+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:40:01.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:01.388+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:40:01.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:01.388+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.470171+0000 front 2024-05-14T15:39:37.470196+0000 (oldest deadline 2024-05-14T15:40:00.970317+0000) 2024-05-14T15:40:02.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:01 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:01.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:02.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:01 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:01.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:02.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:01 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:01.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:02.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:01 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:01.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:02.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:01 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:01.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:02.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:01 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:01.765+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:02.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:01.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:02.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:01.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:02.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:01.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:02.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:01.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:02.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:01 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:01.860+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:02.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:02 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:02.456+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:02.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:02 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:02.456+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:02.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:02 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:02.456+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:02.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:02 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:02.456+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:02.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:02 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:02.456+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:02.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:02.436+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.664945+0000 front 2024-05-14T15:39:03.664854+0000 (oldest deadline 2024-05-14T15:39:28.364959+0000) 2024-05-14T15:40:02.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:02.436+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.466094+0000 front 2024-05-14T15:39:12.466042+0000 (oldest deadline 2024-05-14T15:39:36.566214+0000) 2024-05-14T15:40:02.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:02.436+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:22.868538+0000 front 2024-05-14T15:39:22.868550+0000 (oldest deadline 2024-05-14T15:39:46.268801+0000) 2024-05-14T15:40:02.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:02.436+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:26.269560+0000 front 2024-05-14T15:39:26.269396+0000 (oldest deadline 2024-05-14T15:39:52.169320+0000) 2024-05-14T15:40:02.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8[54617]: 2024-05-14T15:40:02.436+0000 7fd0ea85b700 -1 osd.8 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.470171+0000 front 2024-05-14T15:39:37.470196+0000 (oldest deadline 2024-05-14T15:40:00.970317+0000) 2024-05-14T15:40:03.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:02 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:02.786+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:03.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:02 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:02.786+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:03.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:02 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:02.786+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:03.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:02 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:02.786+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:03.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:02 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:02.786+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:03.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:02 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:02.786+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:03.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:02.898+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:03.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:02.898+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:03.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:02.898+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:03.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:02.898+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:03.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:02 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:02.898+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:03.800 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:03 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:03.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:03.800 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:03 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:03.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:03.800 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:03 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:03.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:03.800 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:03 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:03.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:03.801 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:03 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:03.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:03.801 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:03 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:03.427+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:04.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:03.768+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:04.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:03.768+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:04.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:03.768+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:04.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:03.768+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:04.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:03.768+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:04.067 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:03 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:03.768+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:04.275 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:04 smithi193 podman[73784]: 2024-05-14 15:40:04.000092213 +0000 UTC m=+5.731220597 container died 9be3e15fe630e1a744cc55429d8183356b610729b1dce8b713c20282d327e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:40:04.276 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:03.914+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:04.276 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:03.914+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:04.276 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:03.914+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:04.276 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:03.914+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:04.276 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:03.914+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:04.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:04 smithi193 podman[73784]: 2024-05-14 15:40:04.36506546 +0000 UTC m=+6.096193834 container cleanup 9be3e15fe630e1a744cc55429d8183356b610729b1dce8b713c20282d327e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-14T15:40:04.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:04 smithi193 bash[73784]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8 2024-05-14T15:40:04.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:04 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:04.467+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:04.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:04 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:04.467+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:04.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:04 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:04.467+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:04.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:04 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:04.467+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:04.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:04 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:04.467+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:04.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:04 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:04.467+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:04.940 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:04 smithi193 podman[73800]: 2024-05-14 15:40:04.625111494 +0000 UTC m=+0.622507854 container remove 9be3e15fe630e1a744cc55429d8183356b610729b1dce8b713c20282d327e1b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-14T15:40:05.065 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:04 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:04.770+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:05.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:04 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:04.770+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:05.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:04 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:04.770+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:05.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:04 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:04.770+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:05.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:04 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:04.770+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:05.066 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:04 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:04.770+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:05.246 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:04 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:04.939+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:05.246 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:04 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:04.939+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:05.246 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:04 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:04.939+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:05.246 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:04 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:04.939+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:05.246 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:04 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:04.939+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:05.247 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:04 smithi193 podman[73891]: 2024-05-14 15:40:04.885258407 +0000 UTC m=+0.017240014 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:40:05.247 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:05 smithi193 podman[73891]: 2024-05-14 15:40:05.000983189 +0000 UTC m=+0.132964774 container create f9e6f265268bc0f3b52ffed91b2f8607733ae018b38a56bbc9148a0bc5da72ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:40:05.247 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:05 smithi193 podman[73891]: 2024-05-14 15:40:05.245277209 +0000 UTC m=+0.377258796 container init f9e6f265268bc0f3b52ffed91b2f8607733ae018b38a56bbc9148a0bc5da72ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-14T15:40:05.538 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:05 smithi193 podman[73891]: 2024-05-14 15:40:05.248574547 +0000 UTC m=+0.380556141 container start f9e6f265268bc0f3b52ffed91b2f8607733ae018b38a56bbc9148a0bc5da72ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T15:40:05.538 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:05 smithi193 podman[73891]: 2024-05-14 15:40:05.353626584 +0000 UTC m=+0.485608176 container attach f9e6f265268bc0f3b52ffed91b2f8607733ae018b38a56bbc9148a0bc5da72ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-14T15:40:05.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:05 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:05.446+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:05.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:05 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:05.446+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:05.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:05 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:05.446+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:05.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:05 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:05.446+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:05.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:05 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:05.446+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:05.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:05 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:05.446+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:05.803 INFO:journalctl@ceph.osd.8.smithi193.stdout:May 14 15:40:05 smithi193 podman[73891]: 2024-05-14 15:40:05.538141866 +0000 UTC m=+0.670123461 container died f9e6f265268bc0f3b52ffed91b2f8607733ae018b38a56bbc9148a0bc5da72ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-14T15:40:06.099 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:05 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:05.901+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:06.100 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:05 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:05.901+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:06.100 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:05 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:05.901+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:06.100 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:05 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:05.901+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:06.100 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:05 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:05.901+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:06.119 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.8.service' 2024-05-14T15:40:06.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:06.160 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-14T15:40:06.160 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-14T15:40:06.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.9 2024-05-14T15:40:06.190 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:05 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:05.815+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:06.190 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:05 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:05.815+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:06.190 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:05 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:05.815+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:06.190 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:05 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:05.815+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:06.190 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:05 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:05.815+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:06.190 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:05 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:05.815+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:06.492 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 systemd[1]: Stopping Ceph osd.9 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:06.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:06.435+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:06.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:06.435+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:06.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:06.435+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:06.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:06.435+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:06.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:06.435+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:06.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:06 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:06.435+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:06.815 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.492+0000 7fb9ed8d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:06.815 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.492+0000 7fb9ed8d7700 -1 osd.9 116 *** Got signal Terminated *** 2024-05-14T15:40:06.815 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.492+0000 7fb9ed8d7700 -1 osd.9 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:40:07.301 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:06 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:06.932+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:07.302 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:06 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:06.932+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:07.302 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:06 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:06.932+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:07.302 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:06 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:06.932+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:07.302 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:06 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:06.932+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:07.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:07.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:07.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:07.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:07.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:07.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:06 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:06.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:07.699 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:07 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:07.425+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:07.699 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:07 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:07.425+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:07.699 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:07 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:07.425+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:07.699 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:07 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:07.425+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:07.699 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:07 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:07.425+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:07.700 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:07 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:07.425+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:08.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:07.964+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:08.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:07.964+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:08.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:07.964+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:08.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:07.964+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:08.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:07.964+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:08.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:07 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:07.891+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:08.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:07 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:07.891+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:08.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:07 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:07.891+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:08.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:07 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:07.891+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:08.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:07 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:07.891+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:08.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:07 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:07.891+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:08.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:08.449+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:08.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:08.449+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:08.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:08.449+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:08.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:08.449+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:08.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:08.449+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:08.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:08 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:08.449+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:09.228 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:08 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:08.977+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:09.229 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:08 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:08.977+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:09.229 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:08 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:08.977+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:09.229 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:08 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:08.977+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:09.229 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:08 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:08.977+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:09.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:08.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:09.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:08.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:09.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:08.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:09.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:08.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:09.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:08.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:09.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:08 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:08.861+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:09.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:09.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:09.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:09.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:09.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:09.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:09.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:09 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:09.415+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:10.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:09.982+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:10.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:09.982+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:10.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:09.982+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:10.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:09.982+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:10.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:09 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:09.982+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:10.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:09.867+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:10.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:09.867+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:10.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:09.867+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:10.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:09.867+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:10.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:09.867+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:10.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:09 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:09.867+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:10.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:10.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:10.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:10.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:10.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:10.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:10.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:10 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:10.428+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:11.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.024+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:11.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.024+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:11.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.024+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:11.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.024+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:11.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.024+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:11.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:10.833+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:56.989331+0000 front 2024-05-14T15:38:56.989366+0000 (oldest deadline 2024-05-14T15:39:20.489508+0000) 2024-05-14T15:40:11.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:10.833+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:05.790924+0000 front 2024-05-14T15:39:05.790804+0000 (oldest deadline 2024-05-14T15:39:29.191115+0000) 2024-05-14T15:40:11.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:10.833+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.692715+0000 front 2024-05-14T15:39:12.692426+0000 (oldest deadline 2024-05-14T15:39:38.592406+0000) 2024-05-14T15:40:11.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:10.833+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:19.693256+0000 front 2024-05-14T15:39:19.693230+0000 (oldest deadline 2024-05-14T15:39:44.993313+0000) 2024-05-14T15:40:11.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:10.833+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:30.194954+0000 front 2024-05-14T15:39:30.195013+0000 (oldest deadline 2024-05-14T15:39:53.695186+0000) 2024-05-14T15:40:11.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:10 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9[66686]: 2024-05-14T15:40:10.833+0000 7fb9e5e85700 -1 osd.9 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:38.996779+0000 front 2024-05-14T15:39:38.996710+0000 (oldest deadline 2024-05-14T15:40:00.696718+0000) 2024-05-14T15:40:11.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:11.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:11.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:11.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:11.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:11.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:11.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:11 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:11.381+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:12.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.997+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:12.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.997+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:12.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.997+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:12.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.997+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:12.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:11.997+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:12.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:11 smithi005 podman[125297]: 2024-05-14 15:40:11.91599893 +0000 UTC m=+5.547962635 container died 29c2fe55651c77573479660419369092376c7cc43f745330aa49b00d7dc1b206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T15:40:12.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:12.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:12.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:12.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:12.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:12.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:12.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:12 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:12.357+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:12.815 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:12 smithi005 podman[125297]: 2024-05-14 15:40:12.333289271 +0000 UTC m=+5.965252968 container cleanup 29c2fe55651c77573479660419369092376c7cc43f745330aa49b00d7dc1b206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T15:40:12.815 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:12 smithi005 bash[125297]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9 2024-05-14T15:40:12.815 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:12 smithi005 podman[125314]: 2024-05-14 15:40:12.534538807 +0000 UTC m=+0.611143155 container remove 29c2fe55651c77573479660419369092376c7cc43f745330aa49b00d7dc1b206 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-14T15:40:13.271 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:12 smithi005 podman[125399]: 2024-05-14 15:40:12.871142908 +0000 UTC m=+0.017626958 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:40:13.272 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:12 smithi005 podman[125399]: 2024-05-14 15:40:12.985477011 +0000 UTC m=+0.131961040 container create c3433bc0e6d4d345d7cc4b6cc77e0288ce855d37123c113b3a70d23b9f3002ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-14T15:40:13.272 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:13 smithi005 podman[125399]: 2024-05-14 15:40:13.196174229 +0000 UTC m=+0.342658270 container init c3433bc0e6d4d345d7cc4b6cc77e0288ce855d37123c113b3a70d23b9f3002ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:13.272 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:13 smithi005 podman[125399]: 2024-05-14 15:40:13.199613904 +0000 UTC m=+0.346097937 container start c3433bc0e6d4d345d7cc4b6cc77e0288ce855d37123c113b3a70d23b9f3002ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:13.272 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:13 smithi005 podman[125399]: 2024-05-14 15:40:13.271189263 +0000 UTC m=+0.417673334 container attach c3433bc0e6d4d345d7cc4b6cc77e0288ce855d37123c113b3a70d23b9f3002ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T15:40:13.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:13 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:13.000+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:13.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:13 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:13.000+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:13.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:13 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:13.000+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:13.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:13 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:13.000+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:13.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:13 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:13.000+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:13.565 INFO:journalctl@ceph.osd.9.smithi005.stdout:May 14 15:40:13 smithi005 podman[125399]: 2024-05-14 15:40:13.506287724 +0000 UTC m=+0.652771755 container died c3433bc0e6d4d345d7cc4b6cc77e0288ce855d37123c113b3a70d23b9f3002ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-14T15:40:13.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:13.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:13.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:13.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:13.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:13.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:13.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:13 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:13.324+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:14.140 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.9.service' 2024-05-14T15:40:14.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:14.222 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-14T15:40:14.223 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-14T15:40:14.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.10 2024-05-14T15:40:14.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:14 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:14.004+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:14.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:14 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:14.004+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:14.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:14 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:14.004+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:14.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:14 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:14.004+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:14.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:14 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:14.004+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:14.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:14 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:14.004+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:14.521 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 systemd[1]: Stopping Ceph osd.10 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:14.521 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:14.521 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:14.521 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:14.521 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:14.522 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:14.522 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:14.522 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.308+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:14.522 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.519+0000 7f8d70117700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:14.522 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.519+0000 7f8d70117700 -1 osd.10 116 *** Got signal Terminated *** 2024-05-14T15:40:14.522 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:14 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:14.519+0000 7f8d70117700 -1 osd.10 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:40:15.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:15 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:15.017+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:15.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:15 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:15.017+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:15.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:15 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:15.017+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:15.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:15 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:15.017+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:15.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:15 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:15.017+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:15.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:15 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:15.017+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:15.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:15.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:15.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:15.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:15.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:15.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:15.799 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:15 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:15.325+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:16.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:16 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:16.050+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:16.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:16 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:16.050+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:16.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:16 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:16.050+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:16.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:16 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:16.050+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:16.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:16 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:16.050+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:16.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:16 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:16.050+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:16.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:16.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:16.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:16.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:16.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:16.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:16.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:16 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:16.306+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:17.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.008+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:17.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.008+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:17.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.008+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:17.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.008+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:17.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.008+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:17.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.008+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:17.752 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:17.752 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:17.752 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:17.752 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:17.752 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:17.753 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:17.753 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:17 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:17.297+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:18.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:18.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:18.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:18.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:18.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:18.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:18.305 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:17 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:17.969+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:18.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:18.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:18.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:18.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:18.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:18.548 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:18.549 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:18 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:18.270+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:19.259 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:19.260 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:19.260 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:19.260 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:19.260 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:19.260 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:19.260 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:18 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:18.989+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:19.797 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:53.787036+0000 front 2024-05-14T15:38:53.786959+0000 (oldest deadline 2024-05-14T15:39:19.686949+0000) 2024-05-14T15:40:19.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:03.188066+0000 front 2024-05-14T15:39:03.188094+0000 (oldest deadline 2024-05-14T15:39:27.288227+0000) 2024-05-14T15:40:19.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.589531+0000 front 2024-05-14T15:39:12.589650+0000 (oldest deadline 2024-05-14T15:39:37.889461+0000) 2024-05-14T15:40:19.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-05-14T15:39:17.890199+0000 front 2024-05-14T15:39:17.890047+0000 (oldest deadline 2024-05-14T15:39:43.790042+0000) 2024-05-14T15:40:19.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.891155+0000 front 2024-05-14T15:39:27.891174+0000 (oldest deadline 2024-05-14T15:39:53.191255+0000) 2024-05-14T15:40:19.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.193272+0000 front 2024-05-14T15:39:37.193375+0000 (oldest deadline 2024-05-14T15:40:03.092760+0000) 2024-05-14T15:40:19.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:19 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10[60728]: 2024-05-14T15:40:19.307+0000 7f8d6c6cd700 -1 osd.10 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:45.294349+0000 front 2024-05-14T15:39:45.294333+0000 (oldest deadline 2024-05-14T15:40:08.794079+0000) 2024-05-14T15:40:20.303 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:20.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:20.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:20.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:20.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:20.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:20.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:20 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:20.021+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:20.337 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:20 smithi144 podman[74525]: 2024-05-14 15:40:20.080309875 +0000 UTC m=+5.662943305 container died b983405ec52c43f800e0f534b0510c931a7eda80ac52dfc6b2a4ed23ace02d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-14T15:40:20.604 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:20 smithi144 podman[74525]: 2024-05-14 15:40:20.419640077 +0000 UTC m=+6.002273507 container cleanup b983405ec52c43f800e0f534b0510c931a7eda80ac52dfc6b2a4ed23ace02d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-14T15:40:20.604 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:20 smithi144 bash[74525]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10 2024-05-14T15:40:21.030 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:20 smithi144 podman[74542]: 2024-05-14 15:40:20.604118148 +0000 UTC m=+0.519919755 container remove b983405ec52c43f800e0f534b0510c931a7eda80ac52dfc6b2a4ed23ace02d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:21.297 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:21 smithi144 podman[74629]: 2024-05-14 15:40:21.029911993 +0000 UTC m=+0.107482446 container create ce88b1c03c7f6c2eed181a826c82739d6df4d57fd7c1484f4ac8386769bbd6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T15:40:21.298 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:21 smithi144 podman[74629]: 2024-05-14 15:40:20.939891063 +0000 UTC m=+0.017461536 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:40:21.298 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:21 smithi144 podman[74629]: 2024-05-14 15:40:21.250138053 +0000 UTC m=+0.327708515 container init ce88b1c03c7f6c2eed181a826c82739d6df4d57fd7c1484f4ac8386769bbd6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:21.298 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:21 smithi144 podman[74629]: 2024-05-14 15:40:21.253356076 +0000 UTC m=+0.330926528 container start ce88b1c03c7f6c2eed181a826c82739d6df4d57fd7c1484f4ac8386769bbd6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:21.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:21 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:21.055+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:21.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:21 smithi144 podman[74629]: 2024-05-14 15:40:21.338551371 +0000 UTC m=+0.416121846 container attach ce88b1c03c7f6c2eed181a826c82739d6df4d57fd7c1484f4ac8386769bbd6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-14T15:40:21.798 INFO:journalctl@ceph.osd.10.smithi144.stdout:May 14 15:40:21 smithi144 podman[74629]: 2024-05-14 15:40:21.538861924 +0000 UTC m=+0.616432392 container died ce88b1c03c7f6c2eed181a826c82739d6df4d57fd7c1484f4ac8386769bbd6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-14T15:40:22.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.10.service' 2024-05-14T15:40:22.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:22.142 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-14T15:40:22.142 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-14T15:40:22.142 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.11 2024-05-14T15:40:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:22.481 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.091+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 systemd[1]: Stopping Ceph osd.11 for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.479+0000 7f5b8bb47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.479+0000 7f5b8bb47700 -1 osd.11 116 *** Got signal Terminated *** 2024-05-14T15:40:22.482 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:22 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:22.479+0000 7f5b8bb47700 -1 osd.11 116 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T15:40:23.553 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:23.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:23.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:23.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:23.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:23.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:23.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:23 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:23.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:24.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:24.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:24.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:24.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:24.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:24.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:24.305 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:24 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:24.044+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:25.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:25.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:25.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:25.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:25.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:25.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:25.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:25 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:25.090+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:26.553 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:26.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:26.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:26.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:26.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:26.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:26.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:26 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:26.127+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:27.553 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-05-14T15:38:55.774439+0000 front 2024-05-14T15:38:55.774564+0000 (oldest deadline 2024-05-14T15:39:21.074497+0000) 2024-05-14T15:40:27.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-05-14T15:39:02.176020+0000 front 2024-05-14T15:39:02.175862+0000 (oldest deadline 2024-05-14T15:39:32.176691+0000) 2024-05-14T15:40:27.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-05-14T15:39:12.177152+0000 front 2024-05-14T15:39:12.177185+0000 (oldest deadline 2024-05-14T15:39:36.277292+0000) 2024-05-14T15:40:27.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-05-14T15:39:27.380131+0000 front 2024-05-14T15:39:27.380077+0000 (oldest deadline 2024-05-14T15:39:52.680127+0000) 2024-05-14T15:40:27.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-05-14T15:39:37.281781+0000 front 2024-05-14T15:39:37.281916+0000 (oldest deadline 2024-05-14T15:40:01.381966+0000) 2024-05-14T15:40:27.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-05-14T15:39:41.382590+0000 front 2024-05-14T15:39:41.382482+0000 (oldest deadline 2024-05-14T15:40:13.183044+0000) 2024-05-14T15:40:27.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11[61190]: 2024-05-14T15:40:27.109+0000 7f5b840f5700 -1 osd.11 116 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-05-14T15:39:53.183697+0000 front 2024-05-14T15:39:53.183682+0000 (oldest deadline 2024-05-14T15:40:17.283664+0000) 2024-05-14T15:40:28.202 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:27 smithi193 podman[74008]: 2024-05-14 15:40:27.848482513 +0000 UTC m=+5.512766780 container died 9bb683fee1a52414c6d8eeaf6a547d25f5a46ae233095b8fb4cb418dcc37da4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T15:40:28.469 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:28 smithi193 podman[74008]: 2024-05-14 15:40:28.292685962 +0000 UTC m=+5.956970227 container cleanup 9bb683fee1a52414c6d8eeaf6a547d25f5a46ae233095b8fb4cb418dcc37da4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-14T15:40:28.470 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:28 smithi193 bash[74008]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11 2024-05-14T15:40:28.803 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:28 smithi193 podman[74023]: 2024-05-14 15:40:28.469064678 +0000 UTC m=+0.618918907 container remove 9bb683fee1a52414c6d8eeaf6a547d25f5a46ae233095b8fb4cb418dcc37da4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=) 2024-05-14T15:40:29.214 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:28 smithi193 podman[74111]: 2024-05-14 15:40:28.821650564 +0000 UTC m=+0.017161912 image pull bc14c6192b1a9279701ead435afd6a28db099db39c71ce5b6f404961d71ff345 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430 2024-05-14T15:40:29.214 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:28 smithi193 podman[74111]: 2024-05-14 15:40:28.936603412 +0000 UTC m=+0.132114796 container create c3e0bdbc34c9911a9aa28505ed8fb9f88519953d9507ab23f774ef2309c3823a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-14T15:40:29.215 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:29 smithi193 podman[74111]: 2024-05-14 15:40:29.13951957 +0000 UTC m=+0.335030922 container init c3e0bdbc34c9911a9aa28505ed8fb9f88519953d9507ab23f774ef2309c3823a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T15:40:29.215 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:29 smithi193 podman[74111]: 2024-05-14 15:40:29.142827141 +0000 UTC m=+0.338338478 container start c3e0bdbc34c9911a9aa28505ed8fb9f88519953d9507ab23f774ef2309c3823a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-14T15:40:29.215 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:29 smithi193 podman[74111]: 2024-05-14 15:40:29.21418826 +0000 UTC m=+0.409699613 container attach c3e0bdbc34c9911a9aa28505ed8fb9f88519953d9507ab23f774ef2309c3823a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-14T15:40:29.553 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:29 smithi193 podman[74111]: 2024-05-14 15:40:29.439146629 +0000 UTC m=+0.634657971 container died c3e0bdbc34c9911a9aa28505ed8fb9f88519953d9507ab23f774ef2309c3823a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-14T15:40:30.017 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:30 smithi193 podman[74137]: 2024-05-14 15:40:30.00117203 +0000 UTC m=+0.556097592 container remove c3e0bdbc34c9911a9aa28505ed8fb9f88519953d9507ab23f774ef2309c3823a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:30.017 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:30 smithi193 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-14T15:40:30.017 INFO:journalctl@ceph.osd.11.smithi193.stdout:May 14 15:40:30 smithi193 systemd[1]: Stopped Ceph osd.11 for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:40:30.020 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@osd.11.service' 2024-05-14T15:40:30.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:30.057 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-14T15:40:30.057 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-14T15:40:30.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.a 2024-05-14T15:40:30.377 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:30 smithi005 systemd[1]: Stopping Ceph mds.a for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:30.650 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:30 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-a[73951]: 2024-05-14T15:40:30.376+0000 7fa8d80f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:30.650 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:30 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-a[73951]: 2024-05-14T15:40:30.376+0000 7fa8d80f2700 -1 mds.a *** got signal Terminated *** 2024-05-14T15:40:33.761 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:33 smithi005 podman[125507]: 2024-05-14 15:40:33.493617023 +0000 UTC m=+3.236232425 container died 29d56efefc52be3d9de07d656c2bb533cc3f39402dfa7208f555f1657e9331a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5) 2024-05-14T15:40:34.065 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:33 smithi005 podman[125507]: 2024-05-14 15:40:33.926276707 +0000 UTC m=+3.668892121 container cleanup 29d56efefc52be3d9de07d656c2bb533cc3f39402dfa7208f555f1657e9331a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-14T15:40:34.066 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:33 smithi005 bash[125507]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-a 2024-05-14T15:40:34.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.a.service' 2024-05-14T15:40:34.398 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:34 smithi005 podman[125532]: 2024-05-14 15:40:34.135871339 +0000 UTC m=+0.638003833 container remove 29d56efefc52be3d9de07d656c2bb533cc3f39402dfa7208f555f1657e9331a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:34.398 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:34 smithi005 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-14T15:40:34.398 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:34 smithi005 systemd[1]: Stopped Ceph mds.a for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:40:34.398 INFO:journalctl@ceph.mds.a.smithi005.stdout:May 14 15:40:34 smithi005 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.a.service: Consumed 1.167s CPU time. 2024-05-14T15:40:34.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:34.411 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-14T15:40:34.412 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-14T15:40:34.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.d 2024-05-14T15:40:34.731 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 14 15:40:34 smithi005 systemd[1]: Stopping Ceph mds.d for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:35.065 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 14 15:40:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-d[72697]: 2024-05-14T15:40:34.729+0000 7fc196ba5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:35.065 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 14 15:40:34 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-d[72697]: 2024-05-14T15:40:34.729+0000 7fc196ba5700 -1 mds.d *** got signal Terminated *** 2024-05-14T15:40:36.023 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 14 15:40:35 smithi005 podman[125637]: 2024-05-14 15:40:35.705096613 +0000 UTC m=+1.070535896 container died 5536fdb611a468cf774cc58c9c494200ae7d9bc12a5ebbaeedf90014d1427168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-d, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a) 2024-05-14T15:40:36.315 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 14 15:40:36 smithi005 podman[125637]: 2024-05-14 15:40:36.097169091 +0000 UTC m=+1.462608363 container cleanup 5536fdb611a468cf774cc58c9c494200ae7d9bc12a5ebbaeedf90014d1427168 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-d, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-05-14T15:40:36.315 INFO:journalctl@ceph.mds.d.smithi005.stdout:May 14 15:40:36 smithi005 bash[125637]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-d 2024-05-14T15:40:36.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.d.service' 2024-05-14T15:40:36.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:36.506 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-14T15:40:36.506 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-14T15:40:36.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.g 2024-05-14T15:40:36.835 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:40:36 smithi005 systemd[1]: Stopping Ceph mds.g for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:37.316 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:40:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-g[73493]: 2024-05-14T15:40:36.833+0000 7f943a65d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:37.316 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:40:36 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-g[73493]: 2024-05-14T15:40:36.833+0000 7f943a65d700 -1 mds.g *** got signal Terminated *** 2024-05-14T15:40:41.113 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:40:40 smithi005 podman[125757]: 2024-05-14 15:40:40.846524104 +0000 UTC m=+4.105941116 container died 95767b1fd925464bda9a9138f4d2dfec38d071d5cfa14b00439385c59d2bcaf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True) 2024-05-14T15:40:41.552 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:40:41 smithi005 podman[125757]: 2024-05-14 15:40:41.333939029 +0000 UTC m=+4.593356032 container cleanup 95767b1fd925464bda9a9138f4d2dfec38d071d5cfa14b00439385c59d2bcaf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-g, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-14T15:40:41.552 INFO:journalctl@ceph.mds.g.smithi005.stdout:May 14 15:40:41 smithi005 bash[125757]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-g 2024-05-14T15:40:41.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.g.service' 2024-05-14T15:40:41.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:41.722 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-14T15:40:41.722 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-14T15:40:41.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.j 2024-05-14T15:40:41.815 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:40:41 smithi005 systemd[1]: Stopping Ceph mds.j for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:42.315 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:40:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-j[72254]: 2024-05-14T15:40:42.036+0000 7fc3c234f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:42.316 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:40:42 smithi005 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-j[72254]: 2024-05-14T15:40:42.036+0000 7fc3c234f700 -1 mds.j *** got signal Terminated *** 2024-05-14T15:40:43.713 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:40:43 smithi005 podman[125876]: 2024-05-14 15:40:43.40330213 +0000 UTC m=+1.461159405 container died e865c0ba747275dbe5bfd2c65bd16da49c22d418cf5e29c2c36b2c6252b5a2c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-14T15:40:44.014 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:40:43 smithi005 podman[125876]: 2024-05-14 15:40:43.804386615 +0000 UTC m=+1.862243890 container cleanup e865c0ba747275dbe5bfd2c65bd16da49c22d418cf5e29c2c36b2c6252b5a2c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-j, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:40:44.014 INFO:journalctl@ceph.mds.j.smithi005.stdout:May 14 15:40:43 smithi005 bash[125876]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-j 2024-05-14T15:40:44.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.j.service' 2024-05-14T15:40:44.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:44.207 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-14T15:40:44.207 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-14T15:40:44.208 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.b 2024-05-14T15:40:44.503 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:40:44 smithi144 systemd[1]: Stopping Ceph mds.b for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:44.763 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:40:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: 2024-05-14T15:40:44.502+0000 7f9956233700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:44.763 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:40:44 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b[66388]: 2024-05-14T15:40:44.502+0000 7f9956233700 -1 mds.b *** got signal Terminated *** 2024-05-14T15:40:46.928 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:40:46 smithi144 podman[74735]: 2024-05-14 15:40:46.65374341 +0000 UTC m=+2.256016674 container died 099fd1712df50f41e3dd0695f2ae0b3ea8ce6fa4c60c57e03893a866d5eed1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True) 2024-05-14T15:40:47.287 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:40:47 smithi144 podman[74735]: 2024-05-14 15:40:47.019206348 +0000 UTC m=+2.621479616 container cleanup 099fd1712df50f41e3dd0695f2ae0b3ea8ce6fa4c60c57e03893a866d5eed1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-14T15:40:47.287 INFO:journalctl@ceph.mds.b.smithi144.stdout:May 14 15:40:47 smithi144 bash[74735]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-b 2024-05-14T15:40:47.427 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.b.service' 2024-05-14T15:40:47.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:47.461 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-14T15:40:47.462 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-14T15:40:47.462 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.e 2024-05-14T15:40:47.797 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:40:47 smithi144 systemd[1]: Stopping Ceph mds.e for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:48.297 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:40:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: 2024-05-14T15:40:47.798+0000 7fcb98259700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:48.298 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:40:47 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e[65766]: 2024-05-14T15:40:47.798+0000 7fcb98259700 -1 mds.e *** got signal Terminated *** 2024-05-14T15:40:50.945 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:40:50 smithi144 podman[74870]: 2024-05-14 15:40:50.66949189 +0000 UTC m=+2.988402612 container died e8a99d2c13403af65a5e0d98f9a288663edb39878faf4aa272152b3085fa1f26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-14T15:40:51.290 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:40:51 smithi144 podman[74870]: 2024-05-14 15:40:51.047837068 +0000 UTC m=+3.366747786 container cleanup e8a99d2c13403af65a5e0d98f9a288663edb39878faf4aa272152b3085fa1f26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-14T15:40:51.291 INFO:journalctl@ceph.mds.e.smithi144.stdout:May 14 15:40:51 smithi144 bash[74870]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-e 2024-05-14T15:40:51.442 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.e.service' 2024-05-14T15:40:51.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:51.474 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-14T15:40:51.475 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-14T15:40:51.475 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.h 2024-05-14T15:40:52.047 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:51 smithi144 systemd[1]: Stopping Ceph mds.h for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:52.047 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h[66084]: 2024-05-14T15:40:51.784+0000 7f8dbd718700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:52.047 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:51 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h[66084]: 2024-05-14T15:40:51.784+0000 7f8dbd718700 -1 mds.h *** got signal Terminated *** 2024-05-14T15:40:54.937 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:54 smithi144 podman[74989]: 2024-05-14 15:40:54.681629253 +0000 UTC m=+2.991816203 container died 4b34cf157ad684386cab930f423e685275034d1de3b650f4ee9aa83e21aad11f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS) 2024-05-14T15:40:55.195 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:54 smithi144 podman[74989]: 2024-05-14 15:40:54.993508436 +0000 UTC m=+3.303695388 container cleanup 4b34cf157ad684386cab930f423e685275034d1de3b650f4ee9aa83e21aad11f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T15:40:55.195 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:54 smithi144 bash[74989]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h 2024-05-14T15:40:55.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.h.service' 2024-05-14T15:40:55.445 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:55 smithi144 podman[75004]: 2024-05-14 15:40:55.194599474 +0000 UTC m=+0.515128906 container remove 4b34cf157ad684386cab930f423e685275034d1de3b650f4ee9aa83e21aad11f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-14T15:40:55.446 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:55 smithi144 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-14T15:40:55.446 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:55 smithi144 systemd[1]: Stopped Ceph mds.h for 4a7148fe-1204-11ef-bc98-c7b262605968. 2024-05-14T15:40:55.446 INFO:journalctl@ceph.mds.h.smithi144.stdout:May 14 15:40:55 smithi144 systemd[1]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.h.service: Consumed 53.839s CPU time. 2024-05-14T15:40:55.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:55.455 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-14T15:40:55.456 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-14T15:40:55.456 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.k 2024-05-14T15:40:55.763 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:40:55 smithi144 systemd[1]: Stopping Ceph mds.k for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:40:56.047 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:40:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-k[65455]: 2024-05-14T15:40:55.787+0000 7fd076873700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:40:56.047 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:40:55 smithi144 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-k[65455]: 2024-05-14T15:40:55.787+0000 7fd076873700 -1 mds.k *** got signal Terminated *** 2024-05-14T15:40:59.464 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:40:59 smithi144 podman[75113]: 2024-05-14 15:40:59.076555766 +0000 UTC m=+3.407643840 container died 9cb95246a8f0118b1e5a2e9ca3c038ed6045bf760c2e240d722087d992c999d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-14T15:40:59.797 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:40:59 smithi144 podman[75113]: 2024-05-14 15:40:59.572041357 +0000 UTC m=+3.903129422 container cleanup 9cb95246a8f0118b1e5a2e9ca3c038ed6045bf760c2e240d722087d992c999d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-14T15:40:59.797 INFO:journalctl@ceph.mds.k.smithi144.stdout:May 14 15:40:59 smithi144 bash[75113]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-k 2024-05-14T15:40:59.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.k.service' 2024-05-14T15:40:59.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:40:59.978 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-14T15:40:59.978 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-14T15:40:59.978 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.c 2024-05-14T15:41:00.301 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:41:00 smithi193 systemd[1]: Stopping Ceph mds.c for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:41:00.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:41:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c[66206]: 2024-05-14T15:41:00.300+0000 7f68b377c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:41:00.553 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:41:00 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c[66206]: 2024-05-14T15:41:00.300+0000 7f68b377c700 -1 mds.c *** got signal Terminated *** 2024-05-14T15:41:02.935 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:41:02 smithi193 podman[74218]: 2024-05-14 15:41:02.665866392 +0000 UTC m=+2.496163668 container died 9e98e3d9fb013e6aa0e42f2a6bbbd5243e8b66051e3ffca559bc914e3ae06e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T15:41:03.294 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:41:03 smithi193 podman[74218]: 2024-05-14 15:41:03.034022168 +0000 UTC m=+2.864319443 container cleanup 9e98e3d9fb013e6aa0e42f2a6bbbd5243e8b66051e3ffca559bc914e3ae06e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True) 2024-05-14T15:41:03.294 INFO:journalctl@ceph.mds.c.smithi193.stdout:May 14 15:41:03 smithi193 bash[74218]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-c 2024-05-14T15:41:03.456 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.c.service' 2024-05-14T15:41:03.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:41:03.491 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-14T15:41:03.491 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-14T15:41:03.491 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.f 2024-05-14T15:41:03.803 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:41:03 smithi193 systemd[1]: Stopping Ceph mds.f for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:41:04.303 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:41:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f[65905]: 2024-05-14T15:41:03.804+0000 7fe4c898d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:41:04.304 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:41:03 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f[65905]: 2024-05-14T15:41:03.804+0000 7fe4c898d700 -1 mds.f *** got signal Terminated *** 2024-05-14T15:41:06.914 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:41:06 smithi193 podman[74353]: 2024-05-14 15:41:06.663727164 +0000 UTC m=+2.950697063 container died 2edd9b4dfe167bf08a7fbc6b45722f256f74a8c061a0da408c0a1eb083ad03f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-14T15:41:07.303 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:41:07 smithi193 podman[74353]: 2024-05-14 15:41:07.01315737 +0000 UTC m=+3.300127290 container cleanup 2edd9b4dfe167bf08a7fbc6b45722f256f74a8c061a0da408c0a1eb083ad03f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-14T15:41:07.304 INFO:journalctl@ceph.mds.f.smithi193.stdout:May 14 15:41:07 smithi193 bash[74353]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-f 2024-05-14T15:41:07.441 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.f.service' 2024-05-14T15:41:07.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:41:07.473 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-14T15:41:07.473 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-14T15:41:07.474 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.i 2024-05-14T15:41:07.776 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:41:07 smithi193 systemd[1]: Stopping Ceph mds.i for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:41:08.053 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:41:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i[66521]: 2024-05-14T15:41:07.775+0000 7f9df5a0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:41:08.054 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:41:07 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i[66521]: 2024-05-14T15:41:07.775+0000 7f9df5a0a700 -1 mds.i *** got signal Terminated *** 2024-05-14T15:41:10.935 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:41:10 smithi193 podman[74475]: 2024-05-14 15:41:10.663646488 +0000 UTC m=+2.967951132 container died 037ab062289e9af0206bd1bebf7162f21de743761035f79f29c2ac9f6755b28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-14T15:41:11.260 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:41:11 smithi193 podman[74475]: 2024-05-14 15:41:11.008906821 +0000 UTC m=+3.313211457 container cleanup 037ab062289e9af0206bd1bebf7162f21de743761035f79f29c2ac9f6755b28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-14T15:41:11.260 INFO:journalctl@ceph.mds.i.smithi193.stdout:May 14 15:41:11 smithi193 bash[74475]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-i 2024-05-14T15:41:11.399 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.i.service' 2024-05-14T15:41:11.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:41:11.430 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-14T15:41:11.430 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-14T15:41:11.430 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.l 2024-05-14T15:41:11.553 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:41:11 smithi193 systemd[1]: Stopping Ceph mds.l for 4a7148fe-1204-11ef-bc98-c7b262605968... 2024-05-14T15:41:12.053 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:41:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l[65600]: 2024-05-14T15:41:11.753+0000 7f46d8416700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T15:41:12.053 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:41:11 smithi193 ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l[65600]: 2024-05-14T15:41:11.753+0000 7f46d8416700 -1 mds.l *** got signal Terminated *** 2024-05-14T15:41:16.030 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:41:15 smithi193 podman[74597]: 2024-05-14 15:41:15.76621837 +0000 UTC m=+4.122260881 container died c93296aea431feea17dd75d6ed279514687dc5f18baa45bf418cae022b1f3b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-14T15:41:16.303 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:41:16 smithi193 podman[74597]: 2024-05-14 15:41:16.120673934 +0000 UTC m=+4.476716434 container cleanup c93296aea431feea17dd75d6ed279514687dc5f18baa45bf418cae022b1f3b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d324431b4d2338fa16c3bb003d46b4793ae98b04c9b497ba1bc6cc9b14e61430, name=ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-vshankar-testing-20240509.053109-debug-e2b755a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-14T15:41:16.304 INFO:journalctl@ceph.mds.l.smithi193.stdout:May 14 15:41:16 smithi193 bash[74597]: ceph-4a7148fe-1204-11ef-bc98-c7b262605968-mds-l 2024-05-14T15:41:16.537 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a7148fe-1204-11ef-bc98-c7b262605968@mds.l.service' 2024-05-14T15:41:16.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T15:41:16.568 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-14T15:41:16.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T15:41:16.772 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:41:19.305 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T15:41:19.494 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:41:21.991 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T15:41:22.179 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:41:24.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:41:24.501 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:41:24.532 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T15:41:24.557 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-14T15:41:24.559 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi005/crash 2024-05-14T15:41:24.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash -- . 2024-05-14T15:41:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T15:41:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-05-14T15:41:24.611 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi144/crash 2024-05-14T15:41:24.612 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash -- . 2024-05-14T15:41:24.653 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T15:41:24.653 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-05-14T15:41:24.655 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi193/crash 2024-05-14T15:41:24.656 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash -- . 2024-05-14T15:41:24.693 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T15:41:24.693 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-05-14T15:41:24.694 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-14T15:41:24.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-14T15:41:24.731 INFO:tasks.cephadm:Compressing logs... 2024-05-14T15:41:24.731 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T15:41:24.774 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T15:41:24.776 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T15:41:24.798 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T15:41:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:: No such file or directory 2024-05-14T15:41:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log 2024-05-14T15:41:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T15:41:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-14T15:41:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-14T15:41:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-14T15:41:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-14T15:41:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-14T15:41:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-14T15:41:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.x.log: 87.1% 90.2% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T15:41:24.804 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T15:41:24.810 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T15:41:24.810 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-05-14T15:41:24.810 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log 2024-05-14T15:41:24.810 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-14T15:41:24.810 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: gzip/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.5.log: /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-14T15:41:24.811 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.z.log: 87.1% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-14T15:41:24.812 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-14T15:41:24.813 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T15:41:24.813 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-14T15:41:24.813 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T15:41:24.813 INFO:teuthology.orchestra.run.smithi005.stderr: 84.2% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T15:41:24.813 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T15:41:24.814 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.11.log: 92.2% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-14T15:41:24.814 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T15:41:24.815 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-14T15:41:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-14T15:41:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-14T15:41:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-14T15:41:24.817 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-14T15:41:24.817 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T15:41:24.817 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log 2024-05-14T15:41:24.819 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T15:41:24.819 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-14T15:41:24.819 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-14T15:41:24.822 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-14T15:41:24.822 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-14T15:41:24.822 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-14T15:41:24.822 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-14T15:41:24.823 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-14T15:41:24.823 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T15:41:24.823 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-14T15:41:24.823 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T15:41:24.823 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-14T15:41:24.823 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-14T15:41:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-14T15:41:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-14T15:41:24.826 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-14T15:41:24.829 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T15:41:24.853 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-14T15:41:24.853 INFO:teuthology.orchestra.run.smithi193.stderr: 91.1% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T15:41:24.878 INFO:teuthology.orchestra.run.smithi193.stderr: 94.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-14T15:41:24.879 INFO:teuthology.orchestra.run.smithi193.stderr: 94.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-14T15:41:24.907 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.b.log: /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-14T15:41:24.969 INFO:teuthology.orchestra.run.smithi144.stderr: 92.1% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-14T15:41:24.990 INFO:teuthology.orchestra.run.smithi144.stderr: 83.8% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T15:41:24.999 INFO:teuthology.orchestra.run.smithi144.stderr: 87.1% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T15:41:25.010 INFO:teuthology.orchestra.run.smithi144.stderr: 91.0% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T15:41:25.046 INFO:teuthology.orchestra.run.smithi144.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T15:41:25.119 INFO:teuthology.orchestra.run.smithi144.stderr: 90.3% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T15:41:25.241 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-14T15:41:25.571 INFO:teuthology.orchestra.run.smithi193.stderr: 92.8% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-14T15:41:25.841 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-14T15:41:30.793 INFO:teuthology.orchestra.run.smithi193.stderr: 96.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-14T15:41:34.274 INFO:teuthology.orchestra.run.smithi144.stderr: 91.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-14T15:41:46.128 INFO:teuthology.orchestra.run.smithi193.stderr: 91.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-14T15:41:47.038 INFO:teuthology.orchestra.run.smithi193.stderr: 91.3% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-14T15:41:48.446 INFO:teuthology.orchestra.run.smithi005.stderr: 91.7% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-14T15:41:52.559 INFO:teuthology.orchestra.run.smithi005.stderr: 91.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-14T15:41:53.244 INFO:teuthology.orchestra.run.smithi005.stderr: 91.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-14T15:41:55.583 INFO:teuthology.orchestra.run.smithi193.stderr: 91.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-14T15:41:56.488 INFO:teuthology.orchestra.run.smithi005.stderr: 91.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-14T15:41:56.490 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-14T15:41:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m31.702s 2024-05-14T15:41:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:user 1m49.590s 2024-05-14T15:41:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m3.279s 2024-05-14T15:41:59.664 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-14T15:41:59.666 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-05-14T15:41:59.666 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m34.876s 2024-05-14T15:41:59.666 INFO:teuthology.orchestra.run.smithi193.stderr:user 1m52.391s 2024-05-14T15:41:59.667 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m3.526s 2024-05-14T15:43:18.511 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-14T15:43:58.419 INFO:teuthology.orchestra.run.smithi144.stderr: 96.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-14T15:44:23.685 INFO:teuthology.orchestra.run.smithi144.stderr: 95.4% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-14T15:44:59.346 INFO:teuthology.orchestra.run.smithi144.stderr: 91.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-14T15:45:08.851 INFO:teuthology.orchestra.run.smithi144.stderr: 91.6% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-14T15:46:17.512 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-14T15:49:36.656 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/4a7148fe-1204-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-14T15:49:36.658 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-05-14T15:49:36.658 INFO:teuthology.orchestra.run.smithi144.stderr:real 8m11.867s 2024-05-14T15:49:36.658 INFO:teuthology.orchestra.run.smithi144.stderr:user 5m23.007s 2024-05-14T15:49:36.659 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m13.158s 2024-05-14T15:49:36.659 INFO:tasks.cephadm:Archiving logs... 2024-05-14T15:49:36.659 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi005/log 2024-05-14T15:49:36.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T15:49:54.871 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi144/log 2024-05-14T15:49:54.872 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T15:50:47.220 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi193/log 2024-05-14T15:50:47.222 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T15:51:05.376 INFO:tasks.cephadm:Removing cluster... 2024-05-14T15:51:05.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --force 2024-05-14T15:51:05.579 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:51:06.020 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --force 2024-05-14T15:51:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:51:06.808 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a7148fe-1204-11ef-bc98-c7b262605968 --force 2024-05-14T15:51:07.000 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 4a7148fe-1204-11ef-bc98-c7b262605968 2024-05-14T15:51:07.459 INFO:tasks.cephadm:Removing cephadm ... 2024-05-14T15:51:07.460 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T15:51:07.479 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T15:51:07.496 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T15:51:07.511 INFO:tasks.cephadm:Teardown complete 2024-05-14T15:51:07.512 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-14T15:51:07.524 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-14T15:51:07.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T15:51:07.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T15:51:07.538 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T15:51:07.651 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-14T15:51:07.662 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-14T15:51:07.662 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T15:51:07.665 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T15:51:07.668 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T15:51:07.680 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-05-14T15:51:07.681 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-05-14T15:51:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-14T15:51:07.685 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T15:51:07.685 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-05-14T15:51:07.686 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +691us[ +707us] +/- 44ms 2024-05-14T15:51:07.686 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -971us[ -971us] +/- 76ms 2024-05-14T15:51:07.686 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -209us[ -194us] +/- 66ms 2024-05-14T15:51:07.686 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:51:07.688 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T15:51:07.688 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-14T15:51:07.688 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +676us[ +690us] +/- 44ms 2024-05-14T15:51:07.688 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -984us[ -969us] +/- 76ms 2024-05-14T15:51:07.688 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -203us[ -189us] +/- 66ms 2024-05-14T15:51:07.688 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:51:07.701 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T15:51:07.702 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-05-14T15:51:07.702 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +648us[ +644us] +/- 44ms 2024-05-14T15:51:07.702 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -1008us[-1011us] +/- 76ms 2024-05-14T15:51:07.702 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -228us[ -231us] +/- 66ms 2024-05-14T15:51:07.702 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T15:51:07.703 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-14T15:51:07.714 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-14T15:51:07.714 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-14T15:51:07.724 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T15:51:07.822 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T15:51:07.931 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T15:51:07.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T15:51:08.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:51:08.029 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T15:51:08.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:51:08.058 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T15:51:08.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:51:08.085 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-14T15:51:08.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-14T15:51:08.106 INFO:teuthology.task.internal:Duration was 2758.425999 seconds 2024-05-14T15:51:08.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-14T15:51:08.117 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-14T15:51:08.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T15:51:08.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T15:51:08.122 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T15:51:08.161 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T15:51:08.162 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T15:51:08.198 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T15:51:08.594 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-14T15:51:08.594 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-05-14T15:51:08.594 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T15:51:08.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-05-14T15:51:08.619 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T15:51:08.654 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-05-14T15:51:08.654 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T15:51:08.680 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-14T15:51:08.680 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T15:51:08.683 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T15:51:08.696 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T15:51:08.745 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-14T15:51:08.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T15:51:08.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T15:51:08.762 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T15:51:09.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-14T15:51:09.964 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-14T15:51:09.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T15:51:09.990 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T15:51:10.020 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T15:51:10.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-14T15:51:10.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T15:51:10.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T15:51:10.065 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T15:51:10.084 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-05-14T15:51:10.095 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-05-14T15:51:10.114 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-05-14T15:51:10.141 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T15:51:10.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:51:10.156 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T15:51:10.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:51:10.186 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T15:51:10.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T15:51:10.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-14T15:51:10.213 INFO:teuthology.task.internal:Transferring archived files... 2024-05-14T15:51:10.215 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi005 2024-05-14T15:51:10.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T15:51:10.280 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi144 2024-05-14T15:51:10.280 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T15:51:10.317 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884/remote/smithi193 2024-05-14T15:51:10.317 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T15:51:10.354 INFO:teuthology.task.internal:Removing archive directory... 2024-05-14T15:51:10.355 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T15:51:10.357 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T15:51:10.360 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T15:51:10.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-14T15:51:10.418 INFO:teuthology.task.internal:Not uploading archives. 2024-05-14T15:51:10.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-14T15:51:10.445 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-14T15:51:10.445 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T15:51:10.448 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T15:51:10.450 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T15:51:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: 265834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 15:51 /home/ubuntu/cephtest 2024-05-14T15:51:10.463 INFO:teuthology.orchestra.run.smithi144.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 15:51 /home/ubuntu/cephtest 2024-05-14T15:51:10.466 INFO:teuthology.orchestra.run.smithi193.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 15:51 /home/ubuntu/cephtest 2024-05-14T15:51:10.467 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-14T15:51:10.480 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-14T15:51:10.504 INFO:teuthology.nuke:Checking targets against current locks 2024-05-14T15:51:10.532 DEBUG:teuthology.nuke:shortname: smithi005 2024-05-14T15:51:10.532 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T15:51:10.555 DEBUG:teuthology.nuke:shortname: smithi144 2024-05-14T15:51:10.556 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T15:51:10.561 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 14:57:07.434175', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFhUIHoGJpf4PEVsOBwGVRXX/J30I2pAaQ+g7bp70PFPF9v4/4PBrrNsOcLn4GyVaW2dqbCD1OfL5u+1YMEmnQ8='} 2024-05-14T15:51:10.581 DEBUG:teuthology.nuke:shortname: smithi193 2024-05-14T15:51:10.581 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T15:51:10.585 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 14:57:07.431505', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRl4etdRzBgBfNEyMDJttwPsxBLFttqNA+87n/JJuaWyi7lGXTVeIohMJtspAxcrNT0H54EG8wqxKuCRxc5vFk='} 2024-05-14T15:51:10.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-14_07:04:04-fs-wip-vshankar-testing-20240509.053109-debug-testing-default-smithi/7705884', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 14:57:07.432933', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLFa8bh5LHZxueTxQq8bSyPOLQsmxgbb6E99sDhMcIc//Rh3En1ZhQlKt+Co23GpBoffNr024DBJ7xff8Uq/aU='} 2024-05-14T15:51:10.618 INFO:teuthology.orchestra.console.smithi005:Power off 2024-05-14T15:51:10.619 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T15:51:10.642 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2024-05-14T15:51:10.642 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:10.659 INFO:teuthology.orchestra.console.smithi193:Power off 2024-05-14T15:51:10.659 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T15:51:10.673 INFO:teuthology.orchestra.console.smithi144:Power off 2024-05-14T15:51:10.673 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T15:51:10.683 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-05-14T15:51:10.683 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:10.690 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-05-14T15:51:10.695 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-05-14T15:51:10.696 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:10.705 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-14T15:51:10.718 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-14T15:51:14.692 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:14.706 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:14.719 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:14.803 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-05-14T15:51:14.815 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-05-14T15:51:14.827 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-05-14T15:51:18.806 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:18.816 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:18.828 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T15:51:18.917 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-05-14T15:51:18.925 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-05-14T15:51:18.936 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-05-14T15:51:19.018 INFO:teuthology.orchestra.console.smithi005:Power off completed 2024-05-14T15:51:19.026 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-05-14T15:51:19.036 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-05-14T15:51:19.143 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 2758.4259989261627 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e2b755a64f0d72176b57299d02e880cf0620d780 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d35da2cbaf73443eb60ada0bf16c922a status: fail success: false 2024-05-14T15:51:19.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T15:51:19.235 INFO:teuthology.run:FAIL